source:date(source:first())
. The wrongful line used source[source:first()]
. This mistake didn't prevent indicator from working, but forced indicator to read all the available data. I apologize for the inconvenience. Note: This article uses the latest API version, so the indicator developed will work only in the August 27 2010 release version. Please read notes at the end of the article to know how to make the indicator working in the current production release. I will remove this note as soon as the current beta version comes to the production.
Note: This point is, probably, the most complex in the indicator development. So, if you can handle this, there is nothing you cannot do.
Sometimes an indicator requires data of a bigger time frame, for example, we could use for close of the previous day for calculations. However, it’s possible to try and find this data in the price history the indicator is applied to, but… Just imagine. To get yesterday’s high/low, the whole yesterday’s data must be loaded. If the current history is a minute history, in the worst case we have to have at least 2880 bars to be able find that data (1440 minute bars for today and then 1440 minute bars for the whole yesterday). This is a huge amount to load and then to calculate. But the system has access to the day data, so why don’t we use it? Let’s do it.
However, there are a number of points we should handle carefully. Ok. Let’s develop an indicator which shows yesterday’s close price on the chart and discuss each point in detail.
Point 1. What is “yesterday”?
The handling of the date in indicators is pretty simple. A date value is a number, which is just a number of days past midnight of Dec, 30 1899. The format is also known as Windows Ole Automation date format. The integer part of this number is a number of days and the fractional part is a port of the day. So, to find a calendar midnight date and time, we must just floor the value (cut away the fractional part). To shift the date to N days ago, we must just subtract N from the date. So, if FXCM used the “calendar clock” for the trading days (i.e. the trading day starts at midnight), the finding of yesterday for bar x of the source would be pretty simple:
math.floor(source:date(x)) - 1
.Unfortunately, FXCM’s trading day does not match the calendar trading day. Currently, the trading day starts at 17:00EST yesterday and lasts until 17:00EST today. And for different configurations this date can differ. Hopefully, the indicator core helps us to handle calculation of the candle borders for a particular date. There is a
core.getcandle
function which returns the date and time of the candle of the specified time frame to which the specified date and time belong. As you can see, this function requires two parameters which let us “tune” this function for a certain configuration: an offset of the trading day and an offset of the trading week against calendar. You must use host:execute("getTradingDayOffset")
and host:execute("getTradingWeekOffset")
host’s calls to get the settings of the trading day and trading week for particular host application and connection.So, the function to get yesterday’s date properly can look as follows:
- Code: Select all
function getYesterday(date)
local offset;
local weekoffset;
offset = core.host:execute("getTradingDayOffset");
weekoffset = core.host:execute("getTradingWeekOffset");
local today, yesterday;
today = core.getcandle("D1", date, offset, weekoffset);
yesterday = today - 1;
return yesterday;
end
Good? Not yet. There is another point we should handle. There are “non-trading” periods when neither trading is possible nor prices are received. So, in fact, during a week only Monday trough Friday candles exist. There are no “Saturday” candles at all. So, when our minute belongs to a Monday day candle, the function will return us a Sunday candle which just does not exist! So, for Monday’s data we must take previous Friday’s close price.
To handle that correctly, there is another useful function:
core.isnontrading
which checks whether the date belongs to a non-trading period (Friday 17:00EST-Sunday 17:00EST) and returns the date and time when this non-trading period is finished. So, to find the proper “yesterday” candle we have to check whether the calendar yesterday falls into non-trading period and shift the “yesterday” date into past to skip this non-trading period.Let’s modify our function a bit:
- Code: Select all
function getYesterday(date)
local offset;
local weekoffset;
offset = core.host:execute("getTradingDayOffset");
weekoffset = core.host:execute("getTradingWeekOffset");
local today, yesterday;
today = core.getcandle("D1", date, offset, weekoffset);
yesterday = today - 1;
local nontrading, nontradingend;
nontrading, nontradingend = core.isnontrading(yesterday, offset);
if nontrading then
-- if the calendar yesterday date is non-trading (Friday 17:00 EST - Sunday 17:00 EST)
-- shift three days back against the non-trading period end (Sunday 17:00 EST).
yesterday = nontradingend - 3;
end
return yesterday;
end
So, now, having the
getYesterday()
function, we can easily find how many days we need. The oldest day is getYesterday(source:date(source:first()))
, i.e. the “yesterday” bar which corresponds to the oldest available bar. And what about the newest one? There are two cases. The source collection could be either a historical (i.e. a collection loaded until the specified date, which will be never changed for the chart window) or “alive” (i.e. loaded till “now” and continuously growing as new bars come). According to these two cases, the “to” date could be either a “yesterday” day of the latest available bar (getYesterday(source:date(source:size() - 1))
or … now as well. To specify “now”, we use 0
value. Point 2. How to get the day price data?
Ok, now we know which data we need. The next step is to get this data. Marketscope provides a
host
interface which lets you execute an application-specific functions. One of these functions is “getHistory”
. By this call Marketscope loads the specified instrument in the specified time frame and in the specified time range. See host:execute(“getHistory”, …)
for details. - Code: Select all
local days; -- a variable to keep days
-- the function loads the proper day history
function loadData()
if source:size() < source:first() then
return ;
end
local from, to;
-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" is the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() – 1));
end
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
end
It is important that after the
“getHistory”
call, Marketscope only starts to load the data. It takes some time, but Marketscope does not wait until the data is completely loaded to avoid hanging of the indicator. When data loading is finished, Marketscope will notify the indicator by calling the AsyncOperationFinished
function. Moreover, Marketscope does not care about the indicator status and will continue calling indicator to be updated. So, there are two points to do.
First, we must protect the indicator from updating while the data is loaded.
To protect the indicator from being recalculated, we can just return from the update function while data is being loaded. Just add the “loading” global variable so that all functions could “know” that data is being loaded.
- Code: Select all
local days; -- day source data
local loading; -- the loading day data flag
function Prepare()
...
days = nil;
loading = false;
...
end
function Update(period, mode)
if loading then
-- do nothing if data is being loaded
return ;
end
if days == nil then
loadData();
return ;
end
...
end
function AsyncOperationFinished(cookie, success, error)
if cookie == 1 then
assert(success, error);
loading = false;
end
return 0;
end
function loadData()
if loading then
return ;
end
local from, to;
-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" is the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date[source:size() - 1]);
end
loading = true;
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
end
Well, now we need to force the indicator update when data is loaded. There is function
instance:updateFrom()
which forces the indicator to be recalculated from the specified period. Also, when AsyncOperationFinished
returns the core.ASYNC_REDRAW
value, Marketscope redraws the indicator in the chart window. So, since we haven’t calculated anything yet, we have to update our indicator right from the oldest data. Just fix
AsyncOperationFinished
a bit. - Code: Select all
function AsyncOperationFinished(cookie, success, error)
if cookie == 1 then
assert(success, error);
loading = false;
instance:updateFrom(source:first());
return core.ASYNC_REDRAW;
end
return 0;
end
Almost finished. The last question is how to get the data. We can calculate the day date. Of course, we can just loop through the day history and find the date, but… It’s a bit long, isn’t it? Please, take into consideraion this article, there is a good recipe there.
So, we can build the first version of our indicator. Let’s do so and try it.
- Code: Select all
function Init()
indicator:name("Show Yesterday Close");
indicator:description("The indicator shows yesterday's close value on the chart");
indicator:requiredSource(core.Tick);
indicator:type(core.Indicator);
indicator.parameters:addColor("clr", "Indicator Line Color", "", core.rgb(255, 255, 0));
end
local source; -- indicator source data
local first; -- first available source data
local days; -- day source data
local loading; -- the loading day data flag
local loadedFrom; -- the date from which the days was loaded
local SYC; -- yesterday close output
local host;
local offset;
local weekoffset;
function Prepare()
local name;
name = profile:id() .. "(" .. instance.source:name() .. ")";
instance:name(name);
source = instance.source;
first = source:first();
loading = false;
days = nil;
host = core.host;
offset = host:execute("getTradingDayOffset");
weekoffset = host:execute("getTradingWeekOffset");
SYC = instance:addStream("SYC", core.Line, name, "SYC", instance.parameters.clr, first);
end
function Update(period, mode)
if period < first then
return ;
end
if loading then
return ;
end
-- load the data if there is no day
if days == nil then
SYC:setBookmark(1, source:first());
loadData();
return ;
end
-- try to find the value in the days collection
local yesterday;
yesterday = getYesterday(source:date(period));
if yesterday < loadedFrom then
SYC:setBookmark(1, period);
loadData();
return ;
end
local i;
i = findDateFast(days, yesterday, false);
if i >= 0 then
SYC[period] = days.close[i];
end
end
function AsyncOperationFinished(cookie, success, error)
if cookie == 1 then
assert(success, error);
loading = false;
local i = SYC:getBookmark(1);
if i < source:first() then
i = source:first();
end
instance:updateFrom(i);
return core.ASYNC_REDRAW;
end
return 0;
end
-- the function loads the proper day history
function loadData()
if source:size() < source:first() then
return ;
end
if loading then
return ;
end
local from, to;
if days == nil then
-- initial data loading
-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" is the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() - 1));
end
loading = true;
loadedFrom = from;
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
else
from = getYesterday(source:date(source:first()));
to = days:date(0);
loading = true;
loadedFrom = from;
host:execute("extendHistory", 1, days, from, to);
end
end
-- returns the beginning of the "yesterday" date for
-- the specified data
function getYesterday(date)
local today, yesterday;
-- get beginning of the today's candle and round it up to the second
today = core.getcandle("D1", date, offset, weekoffset);
today = math.floor(today * 86400 + 0.5) / 86400;
-- get calendar yesterday
yesterday = today - 1;
local nontrading, nontradingend;
nontrading, nontradingend = core.isnontrading(yesterday, offset);
if nontrading then
-- if the calendar yesterday date is non-trading (Friday 17:00 EST - Sunday 17:00 EST)
-- shift three days back against the non-trading period end (Sunday 17:00 EST).
yesterday = nontradingend - 3;
end
return yesterday;
end
-- ------------------------------------------------------------------------------
-- Find the specified date in the specified stream
--
-- The function uses the binary search algorithm, so it requires only O(n) = log2(n) operations
-- to find (or to do not find) the value.
--
-- The function compares the date and time rounded to the whole seconds.
--
-- Parameters:
-- stream The price stream to find the date in
-- date Date and time value to be found
-- precise The search mode
-- In case the value of the parameter is true, the function
-- Searches for the the exact date and returns not found in the
-- date is not found.
-- In case the value of the parameter is false, the function
-- returns the period with the biggest time value which is smaller
-- than the value of the date parameter.
-- Returns:
-- < 0 The value is not found
-- >= 0 The index of the the period in the stream
-- ----------------------------------------------------------------------------------
function findDateFast(stream, date, precise)
local datesec = nil;
local periodsec = nil;
local min, max, mid;
datesec = math.floor(date * 86400 + 0.5)
min = 0;
max = stream:size() - 1;
while true do
mid = math.floor((min + max) / 2);
periodsec = math.floor(stream:date(mid) * 86400 + 0.5);
if datesec == periodsec then
return mid;
elseif datesec > periodsec then
min = mid + 1;
else
max = mid - 1;
end
if min > max then
if precise then
return -1;
else
return min - 1;
end
end
end
end
Now apply the indicator to the chart.
It works!!!! But stop, stop, stop… When we try to load more data into past it apparently stopped to work!
Point 3. How to get the day price data?
What happened?! Ok. Remember, we loaded the data starting from yesterday for the oldest of the existing candles. When we started to scroll our chart back, more data was loaded. But who updated our collection of day bars? Of course, nobody did it. So, we have to force Marketscope to load new data when the source collection is extended into past. Good news is that there is another cool call of the host interface:
host:execute(“extendHistory”)
which helps us just load more data into already loaded history instead of loading the whole history again. Because the original source history can be extended only into past, it is too easy to check whether new data is loaded. When it happens, the “yesterday” value we are trying to find is earlier than the date we previously loaded the day history from. So, do the following modification in our code:
a) Keep the “from” date we loaded the day history earlier.
b) In case the day collection already exists, extend it instead of loading older data.
So, add new
loadedFrom
global variable. And change the Update
and loadData
functions. There is new indicator version:
- Code: Select all
function Init()
indicator:name("Show Yesterday Close");
indicator:description("The indicator shows yesterday's close value on the chart");
indicator:requiredSource(core.Tick);
indicator:type(core.Indicator);
indicator.parameters:addColor("clr", "Indicator Line Color", "", core.rgb(255, 255, 0));
end
local source; -- indicator source data
local first; -- first available source data
local days; -- day source data
local loading; -- the loading day data flag
local loadedFrom; -- the oldest yesterday bar we already loaded
local SYC; -- yesterday close output
local host;
local offset;
local weekoffset;
function Prepare()
local name;
name = profile:id() .. "(" .. instance.source:name() .. ")";
instance:name(name);
source = instance.source;
first = source:first();
loading = false;
days = nil;
host = core.host;
offset = host:execute("getTradingDayOffset");
weekoffset = host:execute("getTradingWeekOffset");
SYC = instance:addStream("SYC", core.Line, name, "SYC", instance.parameters.clr, first);
end
function Update(period, mode)
if period < first then
return ;
end
if loading then
return ;
end
-- load the data if there is no day
if days == nil then
loadData();
return ;
end
-- try to find the value in the days collection
local yesterday;
yesterday = getYesterday(source:date(period));
-- check whether such day bar is not loaded yet
if yesterday < loadedFrom then
loadData();
return ;
end
-- find the day data
local i;
i = findDateFast(days, yesterday, false);
if i >= 0 then
SYC[period] = days.close[i];
end
end
function AsyncOperationFinished(cookie, success, error)
if cookie == 1 then
assert(success, error);
loading = false;
instance:updateFrom(source:first());
return core.ASYNC_REDRAW;
end
return 0;
end
-- the function loads the proper day history
function loadData()
if source:size() < source:first() then
return ;
end
if loading then
return ;
end
local from, to;
if days == nil then
-- initial data loading
-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" is the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() - 1));
end
loading = true;
loadedFrom = from;
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
else
from = getYesterday(source:date(source:first()));
to = days:date(0);
loading = true;
loadedFrom = from;
host:execute("extendHistory", 1, days, from, to);
end
end
-- returns the beginning of the "yesterday" date for
-- the specified data
function getYesterday(date)
local today, yesterday;
-- get beginning of the today's candle and round it up to the second
today = core.getcandle("D1", date, offset, weekoffset);
today = math.floor(today * 86400 + 0.5) / 86400;
-- get calendar yesterday
yesterday = today - 1;
local nontrading, nontradingend;
nontrading, nontradingend = core.isnontrading(yesterday, offset);
if nontrading then
-- if the calendar yesterday date is non-trading (Friday 17:00 EST - Sunday 17:00 EST)
-- shift three days back against the non-trading period end (Sunday 17:00 EST).
yesterday = nontradingend - 3;
end
return yesterday;
end
-- ------------------------------------------------------------------------------
-- Find the specified date in the specified stream
--
-- The function uses the binary search algorithm, so it requires only O(n) = log2(n) operations
-- to find (or to do not find) the value.
--
-- The function compares the date and time rounded to the whole seconds.
--
-- Parameters:
-- stream The price stream to find the date in
-- date Date and time value to be found
-- precise The search mode
-- In case the value of the parameter is true, the function
-- Searches for the the exact date and returns not found in the
-- date is not found.
-- In case the value of the parameter is false, the function
-- returns the period with the biggest time value which is smaller
-- than the value of the date parameter.
-- Returns:
-- < 0 The value is not found
-- >= 0 The index of the the period in the stream
-- ----------------------------------------------------------------------------------
function findDateFast(stream, date, precise)
local datesec = nil;
local periodsec = nil;
local min, max, mid;
datesec = math.floor(date * 86400 + 0.5)
min = 0;
max = stream:size() - 1;
while true do
mid = math.floor((min + max) / 2);
periodsec = math.floor(stream:date(mid) * 86400 + 0.5);
if datesec == periodsec then
return mid;
elseif datesec > periodsec then
min = mid + 1;
else
max = mid - 1;
end
if min > max then
if precise then
return -1;
else
return min - 1;
end
end
end
end
Install the indicator and try it. Wow. Now, when the previous version stopped working during the scroll, the new version works anyway! Congratulation!