Bruce Where would I find Ed Gilbert's doc?
--- In [email protected], "bruce1r" <[EMAIL PROTECTED]> wrote: > > Ken - > > I'm too involved with something else right now, but let me see if I > can offer quick suggestion. First - > > 1. Tomasz is pointing out the solutions in (N^2) time are never > practical past some limit. That means that the execution time goes up > with the square of the number of items - ticker in this case. There > are a couple of programming tricks that you can play, but I don't > think that they are going to get you where you want to go - > > For example, programming tricks can be used to make the N^2 comparison > matrix "triangular". This reduces the comparisons by half. > > You might use Pad and Align to a ticker with a short history to cut > the time further. > > But, this is still going to leave you in a long timeframe. > > 2. It looks like you are trying to add unbounded indicators and use > the ordinal values to normalize them so that they can be combined. > Use of the custom backtester would still require that you generate > output for each indicator and then combine them. > > Another approach might be to go out of the box a little and question > your basic assumption. Here's what I mean. > > Ordinal values can be used to convert unbounded ranges (such as ROC) > to bounded values. But they can do some strange things to outliers. > For example, consider these points. Say they are for tickers > A,B,C,D,E on a particular day - > > 0, 20, 21, 22, 200 > > The point 22 is ranked #2 (higher value better) when it is not near > the top. > > ON THE OTHER HAND, range value can be used also to convert unbounded > data to bounded. THEY REQUIRE A PRE-SCAN TO KNOW THE MIN AND MAX. > For the range above, it would convert to the following percentages - > > 0, 10, 10.5, 11, 100 > > This has some advantages for certain data distributions, but some > disadvantages for others. For data where the probability of outliers > is low, it yields similar results. > > SO, HERE'S WHAT YOU MIGHT DO. > > 1. Take a watchlist and start a Exploration pass. When > Status("stocknum") == 0, loop through the list and find the global Min > and Max for each bar across all of the tickers for a given indicator > and store it in an ATC in the H and L fields. For RSI and ROC, you > would have 2 ATC's - say ~MINMAX_ROC and ~MINMAX_RSI. This is 1 pass > of all N tickers. > > 2. Continuing on for stocknum 0 and for 1 - N, calculate the ROC and > RSI and convert it to a percentage of the MIN and MAX range that you > stored in the ATC's for each bar - > > rangepcnt = ( tickscore - tickglobalmin ) / ( tickmax - tickglobalmin > ) * 100; > > 3. Now you can combine the range values because they are normalized. > If you divide by the number of indicators, you'll end up with a > combined percentage. > > Now, while this is not an ordinal rank, it works perfectly well for > scoring and is a solution in 2*N time. BTW - this reference won't mean > much to most here, but should to you - Ed Gilbert detailed this in > Trade doc almost a decade ago. > > -- BruceR > > > --- In [email protected], "Tomasz Janeczko" <groups@> wrote: > > > > Hello, > > > > No, look again. The code I provided gives the sort is ON BAR BY BAR > basis. > > > > Best regards, > > Tomasz Janeczko > > amibroker.com > > ----- Original Message ----- > > From: Ken Close > > To: [email protected] > > Sent: Sunday, July 06, 2008 9:08 PM > > Subject: RE: [amibroker] Paul Ho: Memory Challenges with Great > Ranking Tool > > > > > > Tomasz: > > > > Thanks for all the help you give to so many people, me included. > > > > However, while I did as you suggested with the custombacktester, > and looked into the output file it produces, I am at a loss to know > how to use the data it contains. It is not all of the data that I need. > > > > I want the ordinal ranking of multiple indicators, add them all > together, per bar and per symbol, and use the final sum, of the > ORDINAL ranks, as the ranking value for all symbols. > > > > This output represents what I want (but it is only for two > indicators). I want to turn this into my "recipe" which will have > approximately 8 to 10 indicators. > > > > > > > > I ran the custom backtest, opened the output.html file, and see > that the symbols are sorted by the ranking value and it is indeed an > ordinal value. But, the sort is done only once (probably as a lastbar > basis) and Paul Ho sorting algorithm gives me ordinal values for each > bar for each symbol (displayed above using a lastbar basis). > > > > You say Paul's code is inefficient, and maybe it is because it > sorts all symbols by all bars. Can you suggest a change to the > specific code that would do what I want, but more efficiently? > > > > Again, thanks for all that you do. > > > > Ken > > > > > > > > > -------------------------------------------------------------------- ---------- > > From: [email protected] [mailto:[EMAIL PROTECTED] > On Behalf Of Tomasz Janeczko > > Sent: Sunday, July 06, 2008 1:39 PM > > To: [email protected] > > Subject: Re: [amibroker] Paul Ho: Memory Challenges with Great > Ranking Tool > > > > > > Hello, > > > > The code is inefficient because it repeats the sorting N*N times > where N is number of symbols, while > > only N times is enough. > > > > Ranking is a process that is done during first pass of backtest. > It is implemented efficiently. > > We can use this built-in process easily using custom backtest > procedure as shown here: > > > > Note that this formula will not produce output in AA directly. > Instead it will produce a HTML > > file (output.html) that you can later import to AA using AA, > File->Import > > > > Also please be warned that produced files are huge and attempt to > load such big HTML file > > into Internet Explorer instead will easily hang IE. > > > > PositionScore = ROC( C, 14 ) + 1000; // WHAT YOU WANT TO RANK > > > > SetOption("MaxOpenPositions", 10 ); > > SetBacktestMode( backtestRegularRaw ); > > Buy=1; > > Sell=0; > > SetCustomBacktestProc(""); > > if( Status("action")==actionPortfolio ) > > { > > bo = GetBacktesterObject(); > > > > bo.PreProcess(); > > > > dt = DateTime(); > > > > fh = fopen("output.html", "w" ); > > > > > fputs ("<TABLE><TR><TH>Symbol</TH><TH>Date/Time</TH><TH>Rank</TH></TR>\n", > fh ); > > > > for( i = 0; i < BarCount; i++ ) > > { > > k = 1; > > for( sig = bo.GetFirstSignal( i ); sig; sig = > bo.GetNextSignal( i ) ) > > { > > Line = "<TR><TD>" + sig.Symbol + "</TD><TD>" + > > DateTimeToStr( dt[ i ] ) + "</TD><TD>" + k + > "</TD></TR>\n"; > > fputs( Line, fh ); > > k++; > > } > > } > > > > bo.PostProcess(); > > > > fputs( "</TABLE>", fh ); > > fclose( fh ); > > } > > > > > > Best regards, > > Tomasz Janeczko > > amibroker.com > > ----- Original Message ----- > > From: Ken Close > > To: [email protected] > > Sent: Sunday, July 06, 2008 5:35 PM > > Subject: [amibroker] Paul Ho: Memory Challenges with Great > Ranking Tool > > > > > > Paul: > > > > my initial euphoria has turned somewhat downward as I attempt to > apply the code below (just two indicators) to larger Watchlists. You > sounded (from other messages) like someone who knows the ins and outs > of memory management with AB, and perhaps can comment on how to keep > the code below from "bogging down". > > > > In spite of my many years with AB and its array processing, my > mind still has a problem wrapping around what this code is doing and > why (and whether) larger populated Watchlists will ever be able to work. > > > > I initially tested against the DJ-30 (30 symbols) and all went > well, fairly quickly, perhaps 10-15 seconds. > > > > I then tried the NDX (100 symbols) and things went more slowly > but finished. I noticed the symbols appearing in the AA window more > slowly. > > > > I have not been able to nor wanted to wait for the SP-500, as > the symbols appear more and more slowly and the est time counter was > saying something like 1 1/2 hours to complete 500 symbols. > > > > I was assuming that the code had to collect or process all > symbols before it could make comparisons among them---this is probably > false or else why would processed symbols start to appear in the AA > window while it is still accessing symbols. > > > > What suggestions can you make, given your understanding of the > code and AB, that would minimize the processing of large member > watchlists? > > > > Can adding a SetBarsRequired in the right place limit the number > of lookback bars that are processed, and thus speed up execution? > > > > As the number of indicators I wish to process into a "Total > Rank" score increases, I imagine that executing this code will get > slower and slower and may not be possible at all. Would you agree? > > > > Thanks for any added help. > > > > Ken > > > > > > > > > -------------------------------------------------------------------- -------- > > From: [email protected] > [mailto:[EMAIL PROTECTED] On Behalf Of Ken Close > > Sent: Saturday, July 05, 2008 10:47 AM > > To: [email protected] > > Subject: [amibroker] What a Great Ranking Tool > > > > > > Paul Ho has come up with a supurb ranking tool. I have expanded > it to two indicators. Feel free to expand the code structure to any > number of indicators. > > > > Possible next step: stick the Tot_Rank values into the OI field > for the symbols, then Plot the Ranks for a visual representation of > "where the symbol is over time". > > > > The possibilities are endless (or at least enlarged because of > Paul's code idea). Thanks Paul for your creative input. > > > > Ken > > > > // Ranking_Alt01.afl KSC 07/05/2008 > > > > // Original code by Paul Ho, Amibroker list 07/05/2008 > > > > // Modifications and expansions by Ken Close 07/05/2008 > > > > > > > > // Will ordinal rank every symbol in watchlist for every bar. > > > > > > > > > > > > mOwnROC = ROC(C, 14); > > > > mOwnRSI = RSIa(C, 14); > > > > mRoc = 0; > > > > mRSI = 0; > > > > list = CategoryGetSymbols(categoryWatchlist, 16); > > > > ROCcount[0] = rocrank[0] = 0; > > > > RSIcount[0] = RSIrank[0] = 0; > > > > for(i = 0; (sym = StrExtract(list, i)) != ""; i++) > > > > { > > > > SetForeign(sym); > > > > mRoc = ROC(C, 14); > > > > mRSI = RSIa(C, 14); > > > > RestorePriceArrays(); > > > > n = !IsNull(mRoc); > > > > m = !IsNull(mRSI); > > > > roccount += n; > > > > rsicount += m; > > > > rocrank = IIf(Nz(mRoc) > mOwnROC, Rocrank + n, rocrank); > > > > rsirank = IIf(Nz(mRsi) > mOwnRSI, Rsirank + m, rsirank); > > > > Totrank = rocrank + rsirank; > > > > } > > > > ROCn = ROC(C, 14); > > > > RSIn = RSIa(C, 14); > > > > Filter = 1; > > > > Buy = Sell = 0; > > > > AddColumn(ROCn, "ROCn",1.2); > > > > AddColumn(RSIn, "RSIn",1.2); > > > > AddColumn(mRoc, "MROC", 1.2); > > > > AddColumn(ROCrank, "ROCRank", 1.0); > > > > AddColumn(RSIrank, "rsirank",1.0); > > > > AddColumn(Totrank, "Totrank", 1.0); > > > > > > > > // To check the sorting, run on a watchlist, then click once on > the date column, > > > > // Then shift click on one of the indicators, ie, RSIn, and you > will see the > > > > // ordinal values in order. > > >
