Glad to see more posts on RMath-plug-in...this is a long overdue task on my to do list. Hopefully I will get some hints from here.
2 thinks I'm struggling to accomplish with it are: -send the investable universe - let's say a wathclist - to R which calculate a sub-universe and send it back to AB for further backtesting (something like every months I get a new investable universe - based on some calculation - which is then used within the backtest together with some buy/sell rules) -send historical metrics - like 1year sharpe ratio - for selected strategies to R which filter them based on some statistical filter and send the historical passing strategies and signals back to AB for final backtesting using the active strategies for every data point in history Any thoughts about that? Paolo --- In [email protected], "progster01" <progs...@...> wrote: > > Patrick, > > Vectors, matrices, and frames, oh my! > > Your post was very helpful and is much appreciated. > > Below is my first working demo, which incorporates your fixes and illustrates: > > Doing a calculation in R and returning it for display in a plot or > Exploration > > Selecting an array from an R-side Matrix result for return to AB > > Saving the R workspace so that all R-side variables can later be examined > > Building R command strings based on AB parameters > > Checking the class of an object > > As I move along, I hope to create more. I'll be accumulating them here: > http://www.codefortraders.com/phpBB3/viewforum.php?f=60 > > - Progster > > ----------- > > > /* > R_SMA_01.p2 by Progster > > Calculate SMA in R by using package:TTR . > > This code demonstrates: > > Doing a calculation in R and returning it for display in a plot or > Exploration > > Selecting an array from an R-side Matrix result for return to AB > > Saving the R workspace so that all R-side variables can later be > examined > > Building R command strings based on AB parameters > > You are allowed to bypass all calls to R by setting DoRStuff to N. > This allows inspection of native-only Close and MA for confidence > building. > > Thanks to Patrick who noted key concepts and provided corrected code! > > This test worked with: > > AB 5.3 > RAndFriendsSetup2092V3.0-17-2.exe: R 2.9.2, statconnDCOM 3.1-2B6 > RMath 1.0.0 per Tools -> Plug-ins ... > > Note: package:TTR must be installed separately > > After experimentation, use the Parameter Reset button to return to > "correct" default values > > */ > > // Uncomment to turn QuickAFL OFF > // Many of the displayed numbers will not seem intuitive if QuickAFL is > on. > // Alignment issue can appear if QuickAFL is on. > // So, to "get on with it" in the simplest fashion, turn QuickAFL OFF > when working with R > SetBarsRequired( sbrAll, sbrAll ); > > // Parameter settings > PlotCandles = ParamToggle( "PlotCandles", "N|Y", 1 ) ; > PlotNativeMA = ParamToggle( "PlotNativeMA", "N|Y", 1 ) ; > > MA_Length = Param( "MA_Length", 20, 1, 50, 1 ) ; > > DoRStuff = ParamToggle( "DoRStuff", "N|Y", 1 ) ; > > PlotRClose = ParamToggle( "PlotRClose", "N|Y", 1 ) ; > AddToRClose = Param( "AddToRClose", 0, -200, 200, 1 ) ; > RCloseShift = Param( "RCloseShift", 0, -20, 20, 1 ) ; > RMAShift = Param( "RMAShift", 0, -20, 20, 1 ) ; > ExplorationShift = Param( "ExplorationShift", 0, -20, 20, 1 ) ; > > Save_R_Workspace = ParamToggle( "Save_R_Workspace", "N|Y", 0 ) ; > > // Load needed non-default R libraries > if( DoRStuff ){ > EnableTextOutput( False ) ; > RMathDebugOn() ; > RMathProcedure( "library( TTR )" ) ; // package:TTR > (Technical Trading Rules) > RMathProcedure( "library( fSeries )" ) ; // package:fSeries > RMathDebugOff() ; > } > > // Build R command strings > Close_RStr = "MyClose <- MyClose + " + NumToStr(AddToRClose, 1.0) ; > printf( "Close_RStr: " + Close_RStr + "\n" ) ; // e.g. > MyClose <- MyClose + 0 > > // Dynamically build the R command string to calculate the specified SMA > MA_RStr = "MyMA <- SMA( MyClose, " + NumToStr(MA_Length, 1.0) + ")" ; > printf( "MA_RStr: " + MA_RStr + "\n" ) ; // e.g. MyMA > <- SMA(MyClose, 20) > > // Do R-side calculations > if( DoRStuff ){ > > RMathSetArray(Close, "MyClose" ) ; // send AB > Close array to R > > // To prove changes can be made on R side, and then returned > // RMathProcedure( " MyClose <- MyClose + 1 " ) ; // not > dynamic > > // default Close_RStr: "MyClose <- MyClose + 0" > RMathProcedure( Close_RStr ) ; > > // These work but they are are not dynamic > // RMathProcedure( " MyMA <- SMA(MyClose) " ) ; // works on R > side, per examination of saved workspace > // RMathProcedure( " MyMA <- SMA(MyClose, 3) " ) ; // works on R > side, per examination of saved workspace > > // Dynamic MA calc in R, varies per MA_Length parameter. > // default MA_RStr: "MyMA <- SMA(MyClose, 20)" > // Note: MyMA is not an Array, it is a Matrix! (that's the TTR > package implementation) > RMathProcedure( MA_RStr ) ; // works on R side, per examination > of saved workspace > > > // IMPORTANT: Nothing will come out right if you attempt to pass > back NA values from R to AB! > // One way to fix this is shown here, using substituteNA() > > // Deal with R-side NA values > RMathProcedure( " MA_Clean <- substituteNA(MyMA, type = <DQ>zeros<DQ>) > " ) ; // substitution to new name OK > > RMathProcedure( " MyMA <- substituteNA(MyMA, type = <DQ>zeros<DQ>) " ) > ; // self-substitution also OK > } > > // RMathDebugOn() ; > bi = BarIndex() ; // 0 to (N - 1) > > // from 1 to N, this is the one that will match the crosshairs, > PROVIDED QuickAFL is OFF > bn = bi + 1 ; > > arrayitem = SelectedValue( bi ) - bi[ 0 ]; > printf( "Close at selected bar:" + NumToStr( Close[ arrayitem ], 1.2 ) + > "\n" ) ; > > LBI = LastValue( bi ) ; > SBI = SelectedValue( bi ) ; > SBN = SelectedValue( bn ) ; > FVB = Status("firstvisiblebar"); > LVB = Status("lastvisiblebar"); > LVBI = Status("lastvisiblebarindex"); // note: case matters! > BTR = LBI - LVBI ; // actual bars to the Right of selected bar > > EnableTextOutput( True ) ; > printf( "\n" + "The following numbers make obvious sense only if QuickAFL > is OFF : \n" ) ; > printf( "LBI: " + NumToStr(LBI, 1.0) + "\n" ) ; > printf( "BarCount: " + NumToStr(BarCount, 1.0) + "\n" ) ; > printf( "SBI: " + NumToStr(SBI, 1.0) + "\n" ) ; // Selected Bar > Index > printf( "SBN: " + NumToStr(SBN, 1.0) + "\n" ) ; // Selected Bar > Number > printf( "FVB: " + NumToStr(FVB, 1.0) + "\n" ) ; > printf( "LVB: " + NumToStr(LVB, 1.0) + "\n" ) ; > printf( "LVBI: " + NumToStr(LVBI, 1.0) + "\n" ) ; > printf( "BTR: " + NumToStr(BTR, 1.0 ) + "\n\n" ) ; > EnableTextOutput( False ) ; > > // Return R-side calculations to AB > if( DoRStuff ){ > > rtnClose = RMathGetArray( "MyClose" ) ; > > // rtnMA = RMathGetArray( "MyMA" ) ; // bogus call! > MyMA is a Matrix, not an array! > > /* > RMathDebugOn() ; > RMathProcedure( " class(MyMA) " ) ; // display the R type > of MyMA in the RMath debug window > RMathDebugOff() ; > */ > > rtnMA = RMathGetArray( "MyMA[,1]" ) ; // NOTE: Matrix > notation !!! > rtnClean = RMathGetArray( "MA_Clean[,1]" ) ; // NOTE: Matrix > notation !!! > > } > > // Optionally save the R workspace image. > // If saved, you can later reopen it in an interactive session of R and > examine everything. > if(Save_R_Workspace){ > RMathDebugOn() ; > // Set desired output directory > RMathProcedure( " setwd( <DQ>C:/Users/Steve/Documents<DQ> ) " ) ; > // Save the workspace to specified file in the cwd > RMathProcedure( " save.image( file=<DQ>R_SMA_01.p2.Rdata<DQ>) " ) ; > // works! > RMathDebugOff() ; > } > > if(PlotCandles == 1) > Plot( Close, "Candle Close", colorGrey50, styleCandle, 0, 0, 0, 0 ) ; > > if(PlotNativeMA == 1) > Plot( MA(Close, MA_Length), "NativeMA", colorWhite, styleLine, 0, 0, > 0, 7 ) ; > > if( DoRStuff ){ > > if(PlotRClose){ > // Plot( rtnClose, "rtnClose", colorWhite, styleLine, 0, 0, 0, 8 > ) ; // Zorder of 8 puts this on top > Plot( Ref(rtnClose, RCloseShift ), "rtnClose", colorWhite, > styleLine, 0, 0, 0, 8 ) ; // Zorder of 8 puts this on top > } > > Plot(Ref(rtnMA, RMAShift), "rtnMA", colorRed, styleDashed, 0, 0, 0, 8 > ) ; > // Hack to prevent plot of leading zeros which are really NA (but > affects all zeros!) > // Plot(IIf( Ref(rtnMA, RMAShift) != 0, Ref(rtnMA, RMAShift), Null > ), "rtnMA", colorRed, styleDashed, 0, 0, 0, 8 ) ; > > // Plot(Ref(rtnClean, RMAShift), "rtnClean", colorYellow, > styleDashed, 0, 0, 0, 8 ) ; > // Hack to prevent plot of leading zeros which are really NA (but > affects all zeros!) > Plot(IIf( Ref(rtnClean, RMAShift) != 0, Ref(rtnClean, RMAShift), Null > ), "rtnClean", colorYellow, styleDashed, 0, 0, 0, 8 ) ; > } > > EnableTextOutput( True ) ; > printf( "Close: " + NumToStr(Close, 6.2 ) + "\n" ) ; > if( DoRStuff ) printf( "rtnMA: " + NumToStr(rtnMA, 6.2 ) + "\n" ) ; > EnableTextOutput( False ) ; > > // Show all results in the Exploration > Filter = 1 ; > > AddColumn( Close, "Close" ) ; > > if( DoRStuff ){ > AddColumn( rtnClose, "rtnClose" ) ; > > // Seems correct at ExplorationShift = 0 > AddColumn( Ref(rtnMA, ExplorationShift), "rtnMA" ) ; > AddColumn( Ref(rtnClean, ExplorationShift), "rtnClean" ) ; > > } > > > > > --- In [email protected], "vlanschot" <vlanschot@> wrote: > > > > Hi Progster, > > > > Appreciate your initiative, so let me give you some help to get you going > > (further). > > > > 1) Remember that R has multiple ways of handling data, including multiple > > arrays in matrix format. This is what the SMA function actually did. Using > > the RMathDebugOn(), you can see that the MyMA-series shows a format like > > this: > > > > [1230,] 99.0 > > [1231,] 100.0 > > etc. > > > > It means that it is a matrix. See my code below on how to call the 1st > > column of it (i.e. your array with data). > > > > 2) Remember that, unless you use R's timeseries functions, R will not > > recognize AB's "bars to the right" settings. It means that in charts you'll > > see a drop-off to zero for, say, the last 10 bars. This is why I use AB's > > Ref function below. > > > > 3) Please have a look at the package Zoo, as well as related timeseries > > packages. It allows you much more flexibility to link R's series to dates, > > etc. On that note, I often use AB's Status("firstvisiblebar"), etc to > > manipulate R's series by filling its matrix notation (i.e. [...,...]) with > > these "barnrs". You need to play around with it, but you'll get the idea. > > > > Here is the adjusted code which will correct the errors: > > > > RMathSetArray(Close, "MyClose" ) ; > > RMathProcedure( " MyClose <- MyClose + 1 " ) ; // to prove changes can be > > done on R side, AND returned > > RMathProcedure( " MyMA <- SMA(MyClose) " ) ; // seems to work on R side, > > based on examination of saved workspace > > > > RMathProcedure( " MA_Clean <- substituteNA(MyMA, type = <DQ>zeros<DQ>) " ) > > ; > > rtnClose = RMathGetArray( "MyClose" ) ; > > > > RMathDebugOn() ; > > FVB = Status("firstvisiblebar"); > > LVB = Status("lastvisiblebar"); > > > > rtnMA = RMathGetArray( "MyMA" ) ; > > rtnClean = RMathGetArray( "MA_Clean[,1]" ) ; > > > > Plot( rtnClose, "rtnClose", colorBlue ) ; > > Plot( Ref(rtnMA,-10), "rtnMA", colorRed ) ; > > Plot(Ref(rtnClean,-10), "rtnClean", colorGreen ) ; > > > > Finally, you are right: the link with R opens a new universe. Still, I > > don't have time to support the plugin to a larger extent. Again, your > > initiative is much appreciated in that regard. But at the end of the day, > > people have to be willing to spend time learning it, and thus R. It took me > > a few weeks, but now I can anything (except the coffee thing ;-)). > > > > > > best, > > > > PS > > > > --- In [email protected], "progster01" <progster@> wrote: > > > > > > Hello RMath plug-in users! > > > > > > Beginning work with the RMath plug-in for AB, I've had success with > > > simple tests such as passing Close to the R side, adding a number to it, > > > passing it back to the AB side, and then showing the results on a chart > > > on in an Exploration. > > > > > > However, adding just a little more complexity seems to introduce > > > problems. > > > > > > Below is a complete code that loads a couple of extra packages and then > > > attempts to generate a simple moving average (SMA) on the R side and pass > > > it back to the AB side. > > > > > > After saving the session on the R side (code provided, just un-comment it > > > and set the path as desired), the session can be reloaded in R > > > interactively and it can be confirmed that the SMA was generated > > > correctly, incl. a few NA values at the front of the array while it ramps > > > up. > > > > > > However, something is not right when passing the SMA back to AB, as the > > > result on the AB side is an array of zeros. This is true even after an > > > attempt to substitute 0 for NA (on the R side), on the theory that NA > > > does not pass pack to AB cleanly (?). > > > > > > So, what appears as if it should work, does not. > > > > > > Is the code incorrect? > > > > > > Does RMath have some limitation in this respect? > > > > > > Is there an aspect of series handling on the R side I need to learn more > > > about? > > > > > > Are there perhaps version incompatibilities in my installation? > > > > > > I'm appealing to other users of the RMath plugin here to see if anyone > > > can shed some light on this, or report that this code works or does not > > > work on their installation. > > > > > > I was hoping to make available a short suite of demos using RMath and AB, > > > but I seem to have driven directly into a hole! > > > > > > Public or private responses are welcome. So far, it seems there are very > > > few postings here related to RMath. The potential of the AB-R connection > > > is huge, IMO, but it's starting to seem that a bit of user-to-user > > > co-support may be important to have available. I'll certainly throw in > > > on this myself if I can get "over the hump". > > > > > > At this point, it doesn't look like there's any danger of swamping the > > > message-base here with RMath discussion. But if that becomes an issue, > > > we can take it to the CFT Forum or elsewhere. > > > > > > Thanks to anyone who can take a look and comment. > > > > > > - Progster > > > > > > /* > > > R_SMA_01.p1 by Progster > > > > > > Attempt to calculate SMA in R by using package:TTR . > > > > > > Problem is, the rtnMA array is all zeros on the AB side, > > > even though it is calculated correctly on the R side. > > > > > > This test failed with: > > > > > > AB 5.3 > > > RAndFriendsSetup2092V3.0-17-2.exe: R 2.9.2, statconnDCOM 3.1-2B6 > > > RMath 1.0.0 per Tools -> Plug-ins ... > > > > > > Note: package:TTR must be installed separately > > > > > > */ > > > > > > EnableTextOutput( False ) ; > > > RMathDebugOn() ; > > > RMathProcedure( "library( TTR )" ) ; // > > > package:TTR (Technical Trading Rules) > > > RMathProcedure( "library( fSeries )" ) ; // package:fSeries > > > RMathDebugOff() ; > > > > > > RMathSetArray(Close, "MyClose" ) ; > > > RMathProcedure( " MyClose <- MyClose + 1 " ) ; // > > > to prove changes can be done on R side, and returned > > > > > > RMathProcedure( " MyMA <- SMA(MyClose, n=10) " ) ; // > > > seems to work on R side, based on examination of saved workspace > > > > > > // RMathProcedure( " MyMA <- filter(MyClose, rep(1/5,5), > > > sides=1) " ) ; // > > > http://tolstoy.newcastle.edu.au/R/help/04/09/4302.html > > > > > > // RMathProcedure( " MyMA <- removeNA( MyMA ) " ) ; > > > > > > // With or without this call, rtnMA does not come out correct on > > > the AB side > > > RMathProcedure( " MA_Clean <- substituteNA(MyMA, type = <DQ>zeros<DQ>) " > > > ) ; // fSeries:substituteNA > > > > > > // RMathProcedure( " " ) ; > > > > > > rtnClose = RMathGetArray( "MyClose" ) ; > > > > > > rtnMA = RMathGetArray( "MA_Clean" ) ; > > > > > > // Finished on the R side - save image if desired. > > > // You can save the image here, then reload it in an R session > > > // to confirm the R side calculation of MA_Clean. > > > > > > /* > > > RMathDebugOn() ; > > > // Set desired output directory > > > RMathProcedure( " setwd( <DQ>C:/Users/Steve/Documents<DQ> ) " ) ; > > > // Save the workspace to specified file in the cwd > > > RMathProcedure( " save.image( file=<DQ>R_SMA_01.p1.Rdata<DQ>) " ) ; > > > // works! > > > RMathDebugOff() ; > > > */ > > > > > > Plot( rtnClose, "rtnClose", colorBlue ) ; > > > Plot( rtnMA, "rtnMA", colorRed ) ; > > > > > > EnableTextOutput( True ) ; > > > printf( "Close: " + NumToStr(Close, 6.4 ) + "\n" ) ; > > > printf( "rtnMA: " + NumToStr(rtnMA, 6.4 ) + "\n" ) ; > > > EnableTextOutput( False ) ; > > > > > > // Show all results in the Exploration > > > Filter = 1 ; > > > > > > AddColumn( Close, "Close" ) ; > > > AddColumn( rtnClose, "rtnClose" ) ; > > > AddColumn( rtnMA, "rtnMA" ) ; > > > > > >
