Getting large enough block of data for both, in- and out- of sample sets plus valiadation sample wil be difficult.
ANN is not entirely black-box if you have access to the hidden layer. Each node represents a single interaction, which can be understood. However, it is a tedious process not very amenable to real-time trading. There are very many books on NNN. I have read quite a few, but can not think of any specific one right now. Sometimes NNN is also referred to as "Lazy learning". You may want to Google that term as well. The basic principle is very simple: when you have a new data point, NNN picks N historical points from your historical data which are most similar to your new data point and computes the prediction for your datapoint based on the similarity-weighted average of outcomes from those historical points. Despite simplicity, it is very powerfull in dealing with interactions. It is very similar to the way human experience is formed. I think I saw a few open source NNN algorithms on the Web. ________________________________ From: Da Xu <[email protected]> To: [email protected] Sent: Fri, February 25, 2011 9:02:00 PM Subject: Re: [JBookTrader] Poll for new features Astor, Thank you very much for your message. Yes. Overfitting the data will be a problem. I guess I will have to train my ANN program using historical data with a long period and then do the forward testing. This might lower the risk of overfitting data. Since ANN algorithm is in fact a black box algorithm, we can do nothing to tell whether the failure is due to a structural shift or a just normal operation. I don't have any experience of NNN algorithm. I searched Wikipedia. It looks very interesting. Could you recommend me any paper, book or source code about NNN algorithm? Best, Da On Thu, Feb 24, 2011 at 10:20 PM, Astor <[email protected]> wrote: Da, I have built quite a few ANN models. Several of them had been (and I think still are) used to manage portfolios with AUM in tens of billions. Let me save you some time - if you are going to optimize thousands of parameters, you will most definitely overfit the data. Your model will perform spectacularly in the backtest and will fail in actal trading. > >I can give you a litany of other problems that you will encounter. You will >have >no clue whether your forecasts come from densely or sparsely populated sample >space, i.e. confidence intervals of your forecasts will vary greatly. When >your >model fails, you will not know if it is part of normal operation or if there >has >been a structural shift. > >If you are determined to explore non-linear interactions for prediction, you >will get much more consistent and much more intuitive results using N-nearest >neighbor algorithm. NNN has no parameters to optimize, requires no training >and can be updated in real time - which is very important for HFT. As long as >your original hypothesis for your model makes sense, you will be able to >understand the basis for the predictions. > >Good luck. > > > ________________________________ From: Da Xu <[email protected]> >To: [email protected] >Sent: Thu, February 24, 2011 6:41:18 PM > >Subject: Re: [JBookTrader] Poll for new features > > > >As I posted before, I woud like to develop artificial neural networks strategy >on JBT. We have to add thousands of parameters and change the optimization >class. I have been focusing on this project for a couple of months. Are there >anyone that is interested in ANN strategy? > >Da > > > >On Wed, Feb 23, 2011 at 3:19 PM, nonlinear <[email protected]> wrote: > >Do JBT users have a "wish list" for the new JBT features/functionality? If so, >please post them here. >>-- >>You received this message because you are subscribed to the Google Groups >>"JBookTrader" group. >>To post to this group, send email to [email protected]. >>To unsubscribe from this group, send email to >>[email protected]. >>For more options, visit this group at >>http://groups.google.com/group/jbooktrader?hl=en. >> >-- >You received this message because you are subscribed to the Google Groups >"JBookTrader" group. >To post to this group, send email to [email protected]. >To unsubscribe from this group, send email to >[email protected]. >For more options, visit this group at >http://groups.google.com/group/jbooktrader?hl=en. > > >-- >You received this message because you are subscribed to the Google Groups >"JBookTrader" group. >To post to this group, send email to [email protected]. >To unsubscribe from this group, send email to >[email protected]. >For more options, visit this group at >http://groups.google.com/group/jbooktrader?hl=en. > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en. -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
