Mike, Thank you yet again! The code works, and your insights are very helpful.
Each time I think I've coded something correctly, seems I actually have a number of basic bugs or sloppy implementation. Getting the hang of when to use array functions (iif) vs scalar (if) can also be confusing sometimes. --- In [email protected], "Mike" <sfclimb...@...> wrote: > > > Ozzy, > > You've got a few things going on: > > 1. You should not be using IIf in this scenario. Because all values are > scalers, you should just use a simple if statement. > > 2. Since you are initializing WinTest to 0, no WinPct will ever be less > than it and FinalFast will never be changed from its initial value of 0. > You must instead initialize WinTest to greater than 100. > > 3. You need to do some rudimentary validation on the input data before > blindly trying to work with it. Even after correcting the 2 issues > above, the last line of your file may be a blank line. This causes > StrExtract to return an empty string "" which StrToNum then converts to > a 0. Thus, your lowest %Win would be found to be 0 as would the > associated fast. > > Some additional thoughts: > > 1. No need to seperate out the first line read, do it all in the same > while loop. > > 2. Move the fclose(fh) inside the if (fh) block. > > 3. Good practice to move variables into the smallest scope within which > they will be used. > > Have a try with the following: > > FinalFast = 0; > > fh = fopen( "C:\\logFile2.txt", "r" ); > > if ( fh ) > { > WinTest = 101; > > while ( !feof( fh ) ) > { > Line1 = fgets( fh ); > WinsPctStr = StrExtract( Line1, 2 ); > > if ( WinsPctStr == "" ) > { > continue; > } > > WinsPct = StrToNum( WinsPctStr ); > > if ( WinsPct < WinTest ) > { > FinalFast = StrToNum( StrExtract( Line1, 0 ) ); > WinTest = WinsPct; > } > } > > fclose( fh ); > } > > _TRACE( NumToStr( FinalFast ) ); > > > Mike > > > --- In [email protected], "ozzyapeman" <zoopfree@> wrote: > > > > Having a bit of trouble designing a simple parser to extract values > from > > a csv file. My code only seems to work for some cases, but not others, > > and I can't seem to figure out why. > > > > I have a csv log file with three columns: FastMa, SlowMA, %Winners: > > > > 11,100,32 > > 22,100,35 > > 31,100,36 > > 43,100,37 > > 52,100,38 > > 23,125,32 > > 32,125,35 > > 44,125,36 > > 53,125,48 > > 12,150,31 > > 25,150,33 > > 34,150,32 > > 46,150,35 > > 54,150,36 > > 26,175,31 > > 36,175,34 > > 47,175,34 > > 55,175,36 > > 27,200,29 > > 37,200,33 > > 48,200,34 > > 57,200,36 > > > > If I want to extract the value of the FastMA that corresponds to the > > largest %Winner, the following code works correctly and prints the > value > > "53": > > > > > //----------------------------------------------------------------------\ > \ > > -------- > > // PARSER VERSION 1: Find value of FastMA that corresponds to largest > > %Winners > > > //----------------------------------------------------------------------\ > \ > > -------- > > > > > > fh = fopen( "C:\\logFile2.txt", "r" ); > > > > FinalFast = WinsPct = WinTest = fast = slow = 0; > > > > if(fh) > > { > > Line1 = fgets(fh); > > WinsPct = StrToNum( StrExtract(Line1,2) ); > > WinTest = WinsPct; // initialize WinTest > > > > while( !feof(fh) ) > > { > > Line1 = fgets(fh); > > > > fast = StrToNum( StrExtract(Line1,0) ); > > slow = StrToNum( StrExtract(Line1,1) ); > > WinsPct = StrToNum( StrExtract(Line1,2) ); > > > > FinalFast = IIf(WinsPct > WinTest, fast, FinalFast); > > > > _TRACE("FinalFast"+FinalFast); > > > > WinTest = IIf(WinsPct > WinTest, WinsPct, WinTest); > > > > _TRACE("WinTest"+WinTest); > > > > } > > } > > fclose(fh); > > > > printf(NumToStr(FinalFast) ); > > > > > > But if I want to extract the value of the FastMA that corresponds to > the > > smallest %Winners, and simply change all ">" to "<", the code should > > work. But it doesn't. Intead of printing "27", it prints "0". Why are > > the values being reset to initialization, when it did not happen with > > the case above? Below is the code. As mentioned, the only trivial > > difference are the inequalities. Otherwise, it's identical to the > above. > > > > Any feedback much appreciated. > > > > > > > //----------------------------------------------------------------------\ > \ > > -------- > > // PARSER VERSION 2: Find value of FastMA that corresponds to smallest > > %Winners > > > //----------------------------------------------------------------------\ > \ > > -------- > > > > > > fh = fopen( "C:\\logFile2.txt", "r" ); > > > > FinalFast = WinsPct = WinTest = fast = slow = 0; > > > > if(fh) > > { > > Line1 = fgets(fh); > > WinsPct = StrToNum( StrExtract(Line1,2) ); > > WinTest = WinsPct; // initialize WinTest > > > > while( !feof(fh) ) > > { > > Line1 = fgets(fh); > > > > fast = StrToNum( StrExtract(Line1,0) ); > > slow = StrToNum( StrExtract(Line1,1) ); > > WinsPct = StrToNum( StrExtract(Line1,2) ); > > > > FinalFast = IIf(WinsPct < WinTest, fast, FinalFast); > > > > _TRACE("FinalFast"+FinalFast); > > > > WinTest = IIf(WinsPct < WinTest, WinsPct, WinTest); > > > > _TRACE("WinTest"+WinTest); > > > > } > > } > > fclose(fh); > > > > printf(NumToStr(FinalFast) ); > > > > > > > > > > > > > > --- In [email protected], "ozzyapeman" zoopfree@ wrote: > > > > > > My bad. I had changed the location of the file in one area of the > code > > > and neglected the other. > > > > > > It works. > > > > > > Thanks a bunch!! > > > > > > > > > > > > --- In [email protected], "ozzyapeman" zoopfree@ wrote: > > > > > > > > Awesome, Mike. That's exactly the type of thing I was looking for. > > > > > > > > One problem, though. I try to run your code and only the column > > > > titles, "Fast,Slow,PCTWin" get written to the file. The actual > > values > > > > are not getting written, despite the fact that my optimization > > reports > > > > shows many values corresponding to % winners > 60. > > > > > > > > Looking at the trace output indicates the number of triggers are > > being > > > > properly hit. Your code also appears to be quite straightforward, > > and > > > > I cannot spot any obvious bug. > > > > > > > > Did you try running this? Does it work for you? Any idea where the > > bug > > > > might be? > > > > > > > > > > > > > > > > --- In [email protected], "Mike" <sfclimbers@> wrote: > > > > > > > > > > AddToComposite is for storing values on a bar by bar basis. I > > don't > > > > > think that this would be the right approach for your goals, > since > > you > > > > > are looking for a single scaler value representing total > > > performance for > > > > > a given period. Yes, you could write scripts to dig out the > > > performance > > > > > metrics and persist the subset of interest. However, it would be > > > easier > > > > > to just track the interesting ones as they occur. The following > > > AFL can > > > > > be run through the Optimizer and persist in a separate file the > > > > > parameter values for only those backtests that were of interest. > > I'll > > > > > leave the digging out of values in the second AFL to you, as it > > sounds > > > > > like you have an idea already of how you want to do that. Mike > > > > > fast = Optimize("Fast", 10, 10, 50, 10); slow = Optimize("Slow", > > 100, > > > > > 100, 200, 25); > > > > > Buy = Cross(MA(Close, fast), MA(Close, slow)); Sell = > > Cross(MA(Close, > > > > > slow), MA(Close, fast)); > > > > > SetCustomBacktestProc(""); > > > > > if (Status("ActionEx") == actionExOptimizeSetup) { > > > > > _TRACE("Optimize Begin"); fh = > > > > > fopen("C:\\temp\\logFile.csv", "w"); > > > > > if (fh) { fputs("Fast,Slow,PCTWin\n", fh); > > > > > fclose(fh); } } > > > > > if (Status("action") == actionPortfolio) { _TRACE("Portfolio > > > > > Backtest"); bo = GetBacktesterObject(); bo.Backtest(); > > > > > stats = bo.GetPerformanceStats(0); pctWinners = > > > > > stats.getValue("WinnersPercent"); > > > > > if (pctWinners > 60) { _TRACE("Trigger"); fh = > > > > > fopen("C:\\temp\\logFile.csv", "a"); > > > > > if (fh) { > > > > > fputs(StrFormat("%f,%f,%f\n", fast, slow, pctWinners), > > fh); > > > > > fclose(fh); > > > > > } } > > > > > } --- In [email protected], "ozzyapeman" <zoopfree@> > > wrote: > > > > > > > > > > > > Sorry if I was not clear enough. I will try to give more > > details. > > > > > > > > > > > > Basically I want to run an optimization on two variables, then > > > > > > somehow, automatically, store all values of optimized > variables > > that > > > > > > correspond to a certain metric range (let's say %Winners > > 60). > > > > > > Another AFL would then pull that range of optimized variables > > for a > > > > > > backtest. > > > > > > > > > > > > I would use the Walk Forward feature of AmiBroker for this, > > > except it > > > > > > only uses the 'best' value, not a range of values. My system > > > requires > > > > > > a range of values, not just the best one. I also want to > > permanently > > > > > > store the best values. > > > > > > > > > > > > I am stuck trying to figure out how to automatically pull the > > > > > > optimized variables from the optimization report. The custom > > > > > > backtester only allows me to pull the built-in metrics. > > > > > > > > > > > > So the basic question is - how do I extract the optimized > > variables > > > > > > after an optimization is run? Do I have to write some vbscript > > that > > > > > > exports the report to csv, then opens that report, then > somehow > > > parses > > > > > > through that report to find the correct column and range? Or > is > > > there > > > > > > (hopefully) a simpler way of extracting the values? > > > > > > > > > > > > Once the values are extracted, it is then fairly easy to > either > > > write > > > > > > them to a file or store them in static variables. But I am > > aiming to > > > > > > store them in composite symbols, as (a) they are 'permanent' > > like > > > > > > external files and (b) my sense is that it is more efficient > to > > pull > > > > > > values from a composite symbol than from an external file. > > > > > > > > > > > > As I run through optimizations across different historical > > > periods, I > > > > > > want to build a number of composite symbols that contain the > > 'best > > > > > > values' of optimized variables for use in 'walk forward' > > backtests, > > > > > > and then eventual live trading. > > > > > > > > > > > > Hopefully the above is clearer now. Please let me know if not. > > > > > > > > > > > > > > > > > > > > > > > > --- In [email protected], "Mike" sfclimbers@ wrote: > > > > > > > > > > > > > > You would have to be a little more clear on exactly what it > is > > you > > > > > are > > > > > > > trying to accomplish. Though, writing to a file directly, or > > using > > > > > > > static variables might be areas to explore. > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > --- In [email protected], "ozzyapeman" <zoopfree@> > > wrote: > > > > > > > > > > > > > > > > Ah. Well that would explain that. Thanks. > > > > > > > > > > > > > > > > Any ideas for a possible workaround? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --- In [email protected], "Mike" <sfclimbers@> > > wrote: > > > > > > > > > > > > > > > > > > Ozzy, > > > > > > > > > > > > > > > > > > The Stats object only contains values for built in > metrics > > as > > > > > > > > > described here (scroll to bottom): > > > > > > > > > > > > > > > > > > http://www.amibroker.com/guide/a_custombacktest.html > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > --- In [email protected], "ozzyapeman" > <zoopfree@> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hello, I've read Herman's excellent doc, "IntroToATC". > > > > > > > > > > > > > > > > > > > > I am trying to run an optimization, and then store the > > > values > > > > > of > > > > > > > the > > > > > > > > > > optimized variables in some composite symbols. I later > > > want to > > > > > > > pull > > > > > > > > > > values of a certain range and input them automatically > > in > > > > > > > another > > > > > > > > > AFL. > > > > > > > > > > However, I keep getting a syntax error that the fields > > > are not > > > > > > > > > > available, even though they clearly are. > > > > > > > > > > > > > > > > > > > > Hoping someone can point out my mistake, or give me > some > > > > > > > > > suggestions on > > > > > > > > > > what else to try. > > > > > > > > > > > > > > > > > > > > Here is the code. Any ideas? : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > //-------------------------------------------------------------- > > > > > > > ---- > > > > > > > > > -- > > > > > > > > > > // TRADING SYSTEM > > > > > > > > > > > > > > > //-------------------------------------------------------------- > > > > > > > ---- > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > FastMALength = Optimize("FastMALength", 10, 1, > > > 10, > > > > > > > > > > > > > > > 1); > > > > > > > > > > SlowMALength = Optimize("SlowMALength", 20, 20, > > 50, > > > > > > > > > 10); > > > > > > > > > > > > > > > > > > > > FastMA = MA( C, FastMALength ); > > > > > > > > > > SlowMA = MA( C, SlowMALength ); > > > > > > > > > > Buy = Cross( FastMA, SlowMA ); > > > > > > > > > > Sell = Cross( SlowMA, FastMA ); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > //-------------------------------------------------------------- > > > > > > > ---- > > > > > > > > > -- > > > > > > > > > > // CUSTOM OPTIMIZATION PROCEDURE (Store opt vars in > > > composite > > > > > > > > > symbols) > > > > > > > > > > > > > > > //-------------------------------------------------------------- > > > > > > > ---- > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > SetCustomBacktestProc( "" ); > > > > > > > > > > > > > > > > > > > > if ( Status( "action" ) == actionPortfolio ) > > > > > > > > > > { > > > > > > > > > > bo = GetBacktesterObject(); > > > > > > > > > > > > > > > > > > > > // run default backtest procedure > > > > > > > > > > bo.Backtest( 1 ); > > > > > > > > > > > > > > > > > > > > st = bo.getperformancestats( 0 ); > > > > > > > > > > > > > > > > > > > > // iterate through closed trades first > > > > > > > > > > for ( trade = bo.GetFirstTrade(); trade; trade = > > > > > > > bo.GetNextTrade > > > > > > > > > () ) > > > > > > > > > > { > > > > > > > > > > FastMALength = st.getvalue( "FastMALength" ); > > > > > > > > > > SlowMALength = st.getvalue( "SlowMALength" ); > > > > > > > > > > > > > > > > > > > > AddToComposite( FastMALength, "~OptFastMA", "X", > > > > > > > 1+2+8+16+64 ); > > > > > > > > > > AddToComposite( SlowMALength, "~OptSlowMA", "X", > > > > > > > 1+2+8+16+64 ); > > > > > > > > > > } > > > > > > > > > > bo.ListTrades(); > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
