Herman,
That was one of the best descriptions of the various methods that can
be used for speed optimizations I have seen. It really shows the
variety of ways to think about RT execution problems and solutions in
AFL. It also highlights how having the low level support in AFL for
certain functions can make it easier for those of us who want to do
more than a one size fits all solution to trading.
The beginner should not be concerned about these kinds of things, and
many beginners will never progress to serious or complex system
development. For those who do stick with it over the years, the
desire and ability to do more with our favorite program grows. When
the time comes to explore and use AB in unique ways that gives us a
trading advantage, having more control over low level functions make
it a straight forward and rewarding journey.
However, we do not yet have all those things in place, and our journey
may times ends up in a frustrated tangle causing us to wast hundreds
of hours, possibly abandon our plans, or jump through system coding
hoops to approximate what we want to do.
I eagerly await each and every new low level AFL feature offered in
new versions. I can almost always use each one to simplify my
existing code and make it more reliable in the process -- or implement
a long desired ability in my system. As you have highlighted, it is
not just the processing of bars, but also expanding on the user
interface that benefits from these methods.
I completely agree with you about the desirability for a more flexible
SBR functionality. It would certainly reduce some of the hoops I am
having to jump through now.
My motto is also get the job done in the KISS way!
Best regards,
Dennis
On Jul 3, 2009, at 7:34 AM, Herman wrote:
See capture below...
It may be possible (i haven't tried it) by partitioning code and
running them in their own panes with their own unique
SetbarsRequired().
In my naive opinion, a preferred method would be if AB had a
SetBarsRequired() that could be called repeatedly, in effect this
would allocated code to different panes without displaying separate
panes. For example each Sections_() could have its own
SetbarsRequired(); Eventually I think we will see this feature
implemented like we saw _Section_()s, TimeFrames(), etc. that can be
called repeatedly and control the program environment.
It would be nice to be able to create/use a virtual pane to isolated
and control the environment for afl.
For personal reasons I like to keep things as easy as possible ;-)
hence I prefer to stick to 32bit XP so that I don't run into
problems with drivers and other programs i use. The days that I
enjoyed installing an OS and learn new features have long gone. At
today's level of technology it should be possible to use a computer,
like many other sophisticated tools, by just using the GUI. Without
know 'how' it works.
I have found that spending time on optimizing the program design and
code is a logical first step that often gives greater benefits than
changing hardware. An exception is when you need more than 4GBytes
of memory. But proper written Indicator code may not need more than
that.
It is essential to know how AmiBroker processes arrays and how to
control bars required. Most of the time the only way to figure this
out is to write test programs.
wrt prioritizing code for speed:
A) Normally code executes on each new quote received. This is the
highest order of execution and should be used only for critical
stuff, for example polling the last Price to see if it meets you
setup requirements.
B) Next you can use the RequestTimedRefresh() to execute code on
second (or greater) bounderies. You execute code only when
Status("redrawaction") is true.
C) You can execute code dynamically, for example only when the last
price (or MA etc) has changed sufficiently to warrant a new
execution of a lengthy process.
D) Many function in AT/RT trading only deal with last values. You
can often collect (Static variables) a limited number of last values
and calculate your own functions. This works for example for MA()
and other simple formula that span just few periods. It is faster to
calc a Ma(C,5) using numbers than using arrays that use thousands of
bars.
E) Results that do not change in the timeframe used should be pre-
calculated. They can stored in Static variables, Composites, or files.
F) I have heard people use array-cloning to reduce overhead - i have
never used that.
You can try poll RT prices for a watchlist using a simple loop:
Below is a capture of RT afl that polls 498 tickers in Real-Time,
calculates values, sorts (OSAKA plugin) them by rank, displays the
top n tickers in a bar chart, displays a Button panel for ordering
and to display order status. Clicking on a Buy cell places the
order, changes its color, and fills it with an order status msg.
This idea can be used to trade individual or custom (RT!) indices.
The market is now closed so this will execute slower if RT data was
coming in, however the 65 Millisecond execution time (IQFeed) shows
that a lot can be done with proper programming :-) btw, note that
this works because of the minimal use of arrays.
I am not a professional programmer, I only use afl and simply tinker
until it works. My conclusion: afl is VERY FAST!
herman
<int_1.jpg>
Thursday, July 2, 2009, 9:08:04 PM, you wrote:
> Hi Herman,
> I was starting to probe the subject of 'optimal RT performance' in
Dennis's
> thread. Trading systems has to be at least as important as
desiging them and all trading is RT!
> It is hard for me to stay on the page because a matrix starts to
appear, in
> my concsiousness, and I can see the interconnecting threads going
in all directions, but I'll try.
> I don't think we have squeezed all of the juice out of the orange
by a long
> way but your comments seem to mark out the playing field.
> So far I am just starting to play around with the following ideas:
> - review the OS I am using (any real benefits in using Linux etc)
> - the machines I am using (we have discussed this quite a bit ...
the 64 bit story continues? .. RAM and cache)
> - a dedicated machine (any point in going further than Mike
indicates re
> using task manager to prioritize ..... MS comes with a lot of junk
I don't use
> ... maybe I will talk to my techie about custom installs for my
trading machines ... strip the OS right down
> - Paul's idea of farming out some dirty work to C++ plugins and
managing memory allocation there
> - AB develops a brain and slaps me when I am doing less than
optimal things with my machine or code
> - managing blocks of AFL to run at different times etc .....
Siddharthas code
> example in Dennis's topic points to one simple way I think.
> - twisting the CPU' arm (in more sophisticated ways than using
Windows Task Manager
> - more to come?
> I don't expect you to answer these questions ... I am just
reviewing my
> understanding, of the potential, or the scope, to date.
> Over to your comments:
> Very interesting.
> Have you published anything on the subjects you listed? (if you
have covered this at the UKB I will just go there).
>> Anyway, we all have to figure out what works for each of us.
> It sure helps if we pool our philosophies, knowledge and
experience first though.
>> you can partition and prioritize RT code, and execute sections of
>code on a schedule that doesn't conflict with placing trades. This
>allows you to run code code in RT that would lock up any other way.
> Briefly ... in laypersons terms .. how are you doing that?
>>Using getrtdataforeign() and simple formulas I can scan, in RT,
100 >(easy) to 500 (pushing it) tickers in real time, and display
the top >candidates using a gfx chart.
> Very impressive .... so far I have been too scared to even try RT
scanning,
> sorting etc .... I find it scary enough in EOD.
> How are you doing that?
>>You may also be able to partition code based on BarsRequired and
run >formulas in a separate panes, each with the minimum number of
bars >required. Use Static Arrays to share results.
> Any further hints on that?
>>Also, many real time trading routines only need a few prices you
can >keep track of yourself - separate these from the arrays
processing >pane.
> You seem to be implying that we can use panes to 'partition' code
and control
> it's execution. What is the basic execution model that AB uses to
manage panes, in relation to each other?
