Hi Dennis, Thank you again for your reply and explanations.
First, I was curious about the order limit you talk about. I was always very skeptical about those kind of orders because it is difficult to backtest them. But I think your idea to have a 5-second window is interesting. I was trying to do that and set a 5-minute window in 15-minute bars, and I'd like to use what I learned (thanks to Graham) to work something with 1-minute bars. But so fars 1-minute seems like a tough nut to crack anyway so I don't know if I'll go into AB professional edition yet... Do you think it is worth all the headache? I mean: you HAVE to have it in AT at this speed, right? About the rollover problem, I will try to do as you said (try to see if newer contract have a longer history), but the problem I see is this one: not only the new ticker is exactly as the February ticker, but the volume and liquidity is also. Maybe what I say is not clear enough; I'll try to explain it the best I can. When I look at the January contract, it is very liquid with lot of action at the beginning of December, two weeks before the end of the contract. My idea of a rollover is somehow to merge the two contracts so it can stay liquid all the time. But the new ticker is absolutely not liquid at the beginning of December, so this is why I thought the rollover process did not work correctly. The new ticker is exactly as is the February contract ticker: not liquid before the end of the January contract. Not liquid before December 20 or so. I will try on more recent tickers, but I am skeptical about this. Anyway, if it works that would be wonderful! Did you learn this kind of code all by yourself in the last two years? I am trying to learn a bit more about AFL; I've read the manual (well, not entirely) and read Howard's book. I hope I could find some way to get better at this. Thank you, Louis 2008/6/23 Dennis Brown <[EMAIL PROTECTED]>: > Louis, > > When you go through the whole process, the rolled contract becomes the new > Feb ticker. The Jan ticker is price adjusted to match the Feb ticker on the > rollover date and pre-pended to it. The point is to get a longer history > than you can get with the newest contract. The Feb ticker database has > been altered at this point, so you can't compare it to itself and see any > difference. > > Essentially, if you do this process for every contract on the rollover > date, you end up with a rolling continuous contract price much like > adjusting a stock for splits or dividends. The test is to see if the > history goes back all the way to the beginning of the Jan ticker which would > be a month longer than the Feb ticker (unless they are both history limited > to the same date by the supplier because they are both a ways back in time > now). > > Try it on some contracts where you are sure that the older contract has a > longer history the the new one for a test. > > BR, > Dennis > > On Jun 23, 2008, at 8:58 PM, Louis Préfontaine wrote: > > Hi Dennis, > > I experienced some trouble trying to merge the futures with your rollover > code. > > I followed everything step by step but it didn't work as it was suppose to. > > To try the feature, I used Miny Crude Oil January 2008 (QMF8) and February > 2008 (QMG8). > > // 1. Create a new ticker for the new contract > // 2. Fill in the information section of the new ticker > // 3. Backfill the new ticker with as much data as available > > > I created both and backfilled both, and filled the information section. > > > // 4. Run this AFL on the old contract ticker and backfill it if needed > > I inserted the chart on the old contract (January). > > // 5. Turn off all filtering in the database settings > // 6. Set the chart timeframe to match the database base timeframe > > (Note sure what those mean, but I am on 24 hours trading and in 1-minute > bars) > > // 7. Fill in the parameter for the NEW contract ticker > // 8. Select the rollover date desired > > I clicked on "parameters" on the February ticker and chose the rollover > date to be December 17, 2007. > > // 9. Verify higher volume for new contract on that date in the chart > //10. Click the parameter to write out the new merged contract text file > > Done. > > //11. Use Import Wizard to import the data into the database > > Done. > //12. If you mess up, delete the new ticker from the database and start > over > // > > The result is a file which is *exactly *the same as the February > contract. I ran the February contract and just below the new file, and both > were absolutely the same, even before the December 2007 rollover. > > What do you think I could have done wrong? > > Thanks, > > Louis > 2008/6/23 Louis Préfontaine <[EMAIL PROTECTED]>: > >> Dennis, >> >> After looking carefully at your AFL code, I can say only two words: it >> rocks! Really, that is what I was looking for. I am trying to make it >> work for 1-minute data, but I wonder: what is the best rollover date? >> >> You seem to have some experience with futures. Would you say it is better >> to trade under 1-minute for the, or 1-minute is ok? Cause of course if you >> trade 5-seconds, you absolutely need to set it automatically to your >> broker. I can't imagine someone standing there all day long and trying to >> fill a trade in 5 seconds... You say you use it to anticipate, but isn't 1 >> -minute doing it the same way? >> >> >> Thanks, >> >> Louis >> >> >> >> 2008/6/23 Dennis Brown <[EMAIL PROTECTED]>: >> >>> Louis, >>> >>> I handle rollover via an AFL chart I wrote last week to make the problem >>> simple for me. I uploaded it to the AFL library. Some data suppliers >>> provide a Continuous Contract ticker for futures. However, unless they give >>> you as much history as you want for backtesting, it will be a problem. That >>> is why I had to write my own CC rollover. I want months of 5 second data, >>> and DTN only provides a week of history. However, I must make sure I do not >>> let a week go by without updating my database of all futures contracts I >>> want to save. I also have to collect data for months before I have a >>> history available. >>> >>> If you only want 1 minute bars, many offer a lot longer history. I could >>> not get good volume or range bar charts with more than 5 second database. >>> Tick is better and I will experiment with that at some future date. >>> However, 1 minute is the shortest chart timeframe I run for the time bars. >>> >>> I do not have an IB account. I will not run auto-trading for quite a >>> while into the future if ever. Right now, AB just tells me (audio and on >>> chart) when to anticipate an order so I can set it up in my broker account >>> screen, then tells me when to send the order (if it gets triggered). >>> >>> I still want to have final say so on any order sent, because I am aware >>> of more things than the program, like economic announcements, breaking news >>> events, patterns I have seen before, but not programmed into the charts, >>> etc. >>> >>> I will take things one step at a time. First thing is to have a >>> backtested edge that is not over fitting the data. >>> >>> Next is to test trade it with a small position and discover how human >>> emotions play into execution --very important! >>> >>> I am at the third stage of cleaning up my trading platform to make it >>> easier on myself. The computer is an extension of my brain, and good at >>> nagging me about opportunities I might miss because of too many bits of >>> information at the same time. >>> >>> I am also working on expanding my AFL to ever complex systems that take >>> over more functions that I do mentally with pattern recognition. >>> >>> I have been working on this system for two years already. I have come a >>> long way with my AFL skills. I think I have started over twice already as I >>> understood the problems better and how to organize the system to make it >>> more general and easier for me to maintain. It is a never ending >>> evolutionary process. >>> >>> Next, I will start having the AFL start filling out the fields on my >>> Broker Interface with things like the position size, limit price, etc. All >>> I have to do is press the send button at the right time. Then, I will have >>> AFL also press the send button, after I pre-approve the trade setup. I will >>> always maintain an override on trading. >>> >>> Eventually, I may sign up with a specialized Futures broker and use their >>> data feed and API to generate the trades. It all just depends on how things >>> progress. >>> >>> You see, one step at a time. If you can't make money the simple manual >>> way, then no point in getting fancy with programming fast ways to lose >>> money. Of course I am referring to Day Trading, not longer term swing >>> trading or investing where the execution timing is not so critical. If you >>> want to make 5 or more trades a day, one second delay in execution can make >>> the difference between a good profit or a loss overall. >>> >>> BR, >>> Dennis >>> >>> >>> On Jun 23, 2008, at 4:05 PM, Louis Préfontaine wrote: >>> >>> Thanks Dennis for your reply. I have traveled a long path since January >>> with AB, but now I feel like this is a very long way too. >>> >>> If there are a lot of errors on stocks, I might go with futures in the >>> future. However, I am kind of impressed by the rollover problem and how to >>> handle the end date for each contract. I am not sure how to deal with this. >>> >>> On a more practical level, do you know if there is any information on how >>> to set AB to run automatic trading on a demo account on IB? I searched but >>> didn't find out how. I am really starting from scratch here. >>> >>> Thanks, >>> >>> Louis >>> >>> 2008/6/23 Dennis Brown <[EMAIL PROTECTED]>: >>> >>>> Louis, >>>> >>>> It is too complicated for me also. I try to simplify everything I can, >>>> because there are so many complicated things to deal with. So I went to >>>> Futures trading which simulates very close to actual executions if you >>>> build >>>> your trading algorithms carefully. Once you are aware of the potential >>>> problems, it is easier to avoid getting trapped by them. For a final test, >>>> I trade it live for a couple of days and compare results realtime from the >>>> backtest (which runs live in indicator mode on my system) and what my >>>> broker >>>> account is doing. I limit my backtesting to a couple of days when I am >>>> trading realtime to keep things running fast. I tweak my trading >>>> algorithms >>>> until my backtests perform a little worse than my real trading. That way I >>>> have a bit of a cushion for the inevitable failures due to data feed >>>> problems, computer system malfunctions, and human error. >>>> >>>> BR, >>>> Dennis >>>> >>>> On Jun 23, 2008, at 2:56 PM, Louis Préfontaine wrote: >>>> >>>> OMG that seems very complicated. How do you do to do that? >>>> >>>> Louis >>>> >>>> 2008/6/23 Dennis Brown <[EMAIL PROTECTED]>: >>>> >>>>> Something to consider is that stocks have a large number of bad tick >>>>> data. Suppliers are trying to correct these as quickly as possible. >>>>> Futures data has very few bad ticks. That is the main reason I switched >>>>> to >>>>> only trading e-mini futures instead of stocks for many trades per day, and >>>>> why I limit my trading to just one future at a time. >>>>> >>>>> To backtest a HF system, you actually need two tickers, the raw data >>>>> ticker and the corrected data ticker. You generate trading signals from >>>>> the >>>>> raw data, and execute the order prices on the corrected data. And don't >>>>> forget to take into account the time lag between the raw data showing up, >>>>> and the time of execution on the corrected data. >>>>> >>>>> BR, >>>>> Dennis >>>>> >>>>> >>>>> On Jun 23, 2008, at 1:37 PM, Louis Préfontaine wrote: >>>>> >>>>> Hi Herman, >>>>> >>>>> I read this one >>>>> http://www.amibroker.org/userkb/2007/12/24/high-frequency-automated-trading-hfat-part-2/ >>>>> but it seemed to me like it was a bit complicated to get an uncorected >>>>> database. I mean, having to manually check where backfill ends and raw >>>>> begins, saving two different databases, etc. Seems like a lot of work... >>>>> and doing it everyday! Isn't there any easier way? >>>>> >>>>> Do you think this is an issue for 1-minute intraday, or only for tick, >>>>> or 5 seconds? Because it does not make sense to waste a lot of time on >>>>> corrected 1-minute data only to see it fails miserabily with raw data! >>>>> >>>>> Thanks, >>>>> >>>>> Louis >>>>> >>>>> 2008/6/23 Herman <[EMAIL PROTECTED]>: >>>>> >>>>>> >>>>>> I remember at least one case where I developed a HG system using >>>>>> historical 1-Minute bars that didn't work with real-time (uncorrected) >>>>>> data. >>>>>> >>>>>> >>>>>> Any system that trades frequently intraday can easily be verified by >>>>>> running it for a few days with life data. Also, the 2nd article on the >>>>>> UKB >>>>>> tell you how to save uncorrected (raw) data for later comparison with >>>>>> backfilled data. >>>>>> >>>>>> >>>>>> Of course different systems have different sensitivities to qualities. >>>>>> You can try and design so that your system is insensitive to the problems >>>>>> outlined on the UKB. >>>>>> >>>>>> >>>>>> best regards, >>>>>> >>>>>> herman >>>>>> >>>>>> >>>>>> For tips on developing Real-Time Auto-Trading systems visit: >>>>>> >>>>>> http://www.amibroker.org/userkb/ >>>>>> >>>>>> >>>>>> Monday, June 23, 2008, 10:04:25 AM, you wrote: >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> Well, I'm not really sure but it seems that one-minute bars have so >>>>>> many trades (assuming your contract is at least somewhat liquid) that >>>>>> these >>>>>> errors should pretty much cancel out and stop being a problem. >>>>>> >>>>>> >>>>>> >>>>>> ----- Original Message ----- >>>>>> >>>>>> From: Louis Préfontaine <[EMAIL PROTECTED]> >>>>>> >>>>>> To: [email protected] >>>>>> >>>>>> Sent: Monday, June 23, 2008 9:40 AM >>>>>> >>>>>> Subject: Re: [amibroker] Re: Questions for people using automatic >>>>>> transactions with IB >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> I wonder if the problems with HFAT can happen in « normal » trading. >>>>>> I mean: if I trade with 1-minute bars, I will build a system with data >>>>>> that >>>>>> has been revised at the end of the day. E.g. the data errors will not be >>>>>> there anymore. However, in real-time trading, those errors will be >>>>>> there.. >>>>>> So how can one deal with this, even without automatic trading? >>>>>> >>>>>> >>>>>> Louis >>>>>> >>>>>> >>>>>> >>>>>> 2008/6/23 reinsley <[EMAIL PROTECTED]>: >>>>>> >>>>>> >>>>>> thank you, I discovered the use of static var on that occasion and now >>>>>> >>>>>> this point is much better. >>>>>> >>>>>> >>>>>> Too many things are missing in my formula, I can't ask help as it >>>>>> >>>>>> could seem that i'am asking for a ready-made solution. I have to keep >>>>>> >>>>>> my cards when I'll be trapped. >>>>>> >>>>>> I need to scambled for the archive first. >>>>>> >>>>>> >>>>>> Regards >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >> > > >
