Hi Brian: Sorry for taking so long to respond, thanks for checking this out.
I will also run my routines over the 25/26 Feb also to see if I get similar results and will email you what i get. I might be making to big a deal over this but I'm new to trading but a bit of a fusspot over having correct data. Also the system I am trying to write is highly dependent on volume, Cheers Mark --- In [email protected], "brian_z111" <brian_z...@...> wrote: > > Mark, > > I checked my files. > I have 6 mths tick data ending Feb08. > > I looked at two files - 25 and 26/Feb. > > They exceed Xcel max rows. > I remember now that I had to work on them in txt format. > > I also have the ASX file format doc ... do you have that .. I can > email it? > > My best guess is: > > - you would get better advice from Weblink or ASX :-) > - but! as long as you delete all errors, and cancellation pairs, then > you can import all the quotes and AB will filter the view for you, as > per your settings. > - you will see some wierd and wonderful prices, in the pre, but that > is normal for 24 hr stock charts and you soon learn to ignore low vol > wierd prices > - in 24 hr view I put a horizontal line on the chart to mark the > formal open and close of market hours > > > > If you want to go into detail: > > - the premarket totals around 3% vol in my file > - it contains housekeeping stuff e.g. warrants, executed puts and > calls etc > - the prices in the pre have no relevance to the current day or the > prev day e.g. puts executing for $25 -29 when BHP traded between > 22.40-22.90 .... BHP hadn't traded at >$25 in the 20 days prior. > > - a lot of the pre quotes are dated for the previous day > > - I think the pre market quotes are not important ... you can keep > them, just in case, if you want to ... you will probably never look > at them .. dumping them isn't high risk ... I'm 99% sure they are of > no use (based on my two files anyway). > > - errors are limited to (???) quotes with no price or vol ... delete > all of them ... most of the errors at in the pre or afters > > - cancellations are not present in large numbers ... they are C > trades matched to the trade above by trade number ... delete the > pairs .. most are in market hours .. a few in the pre > > > > From the format doc: > > A trade cancellation consists of the following CSV record: > > C, Symbol, Date, TS Number, Cancelled TS Number, Cancelled Date, > Cancel Reason > Where: Symbol -- Unique security identifier, no more than 6 > characters. The first three characters indicate the issuer or > underlying security. > Date -- The date that the trade cancellation occurred. > TS Number -- Trade sequence number. This is a 6 digit identifier for > the trade cancellation, which is unique for the current day. > Cancelled TS Number -- The TS number of the trade being cancelled. > This, coupled with the cancelled date field, can be used to uniquely > identify the trade. > Cancelled Date -- The date on which the trade being cancelled > occurred. > Cancel Reason -- The reason for the cancellation, being one of the > following: > Incorrect Broker Number > Incorrect Volume > Incorrect Price > Incorrect Security Code > Other > Data Entry > Stop Cancellation Request > > Example: > > XT is a cross trade > Yahoo has wrapped the 2nd line > the C line cancels trade 12334 above > I checked approx 50 and they are all matching pairs on two lines > > T BHP 20080226 100544 12334 39.26 30000 > 20080229 XT > C BHP 20080226 12334 12334 20080226 > > > - none of the C's in my files gave a reason ... the intitial trades > were nearly all crossed trades though > > - trading continues beyond 1600hrs for 15 - 30 mins ... you can keep > this in the database if you want ... it is good data ... in fact best > that you do keep it > > - the after market quotes include a lot of errors > > > > > > > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > Yes, that is the area I am referring to. > > > > Importing 24 hours data would be the best, but then you need to be > > 100% certain about the stuff that is in the after hours timeslot, > > because that is where the data is non-typical. > > > > > > I'll get my file out and check ... my file is approx one year > old.... > > might be different to yours. > > > > It's worth it since it has relevance to other Aussie traders. > > > > Better still if we could leave some brief notes, on a method, in > the > > file section but that isn't a pre-requisite if time presses. > > > > I'll come back to you ... if not today then within a few days. > > > > > > --- In [email protected], "elegalpublishing" > > <mark.a.brand@> wrote: > > > > > > Hi Brian: > > > > > > <snip> > > > I'm not certain but I think a trade for 10000 as a buy might be > > > corrected as a sell the next day, so both orders should be > cancelled > > > for a 5 star database. > > > </snip> > > > > > > Thanks for that, there are two types of transactions in the > Weblink > > > tick data, "T" normal trade and "C" for cancelled trades. Would > you > > > expect the situation you refer to above be covered by the > > cancellation > > > transactions. > > > > > > Thanks > > > Mark > > > > > > PS. I will send the scripts to your email. > > > > > > --- In [email protected], "brian_z111" <brian_z111@> > wrote: > > > > > > > > Have a quick look to make sure you are not double counting vol > or > > > > data ... I don't have a file in front of me but from memory > they > > > > trades have a ref number or order number .... I'm not certain > but > > I > > > > think a trade for 10000 as a buy might be corrected as a sell > the > > > > next day, so both orders should be cancelled for a 5 star > > database. > > > > > > > > > > > > --- In [email protected], "brian_z111" <brian_z111@> > > wrote: > > > > > > > > > > I guess we are better off to trust the ASX file .... so if we > > > > assume > > > > > their corrections are spot on, and you enter all clean 24 > hour > > data > > > > > by date (sounds like you have done that) then just use AB's > > built > > > > in > > > > > database settings >> intraday settings to view 24 hours or > > market > > > > > hours only ... this works fine for me and you can have the > best > > of > > > > > both worlds. > > > > > > > > > > > > > > > > gawk and bash under cygwin, i can send the scripts to you > if > > you > > > > are > > > > > > interested. > > > > > > > > > > Thanks that would be good - might help me with my programming > > > > > education. > > > > > > > > > > brian_z111|asat|yahoo.com > > > > > > > > > > Interesting example in the US, last night, of how the goal > > posts > > > > are > > > > > shifting .... formerly a daily bar was market hour data only > > and > > > > > nothing else mattered. > > > > > > > > > > Citigroup (C) traded up from 3.48 to 4.52 in overnight > > trading ... > > > > > then drifted a bit to open at 4.29 and then traded sideways > > during > > > > > market hours to finish at 4.17 > > > > > > > > > > This only happens in the hot US stocks but it changes the > > meaning > > > > of > > > > > daily bars. > > > > > > > > > > C is in the top ten US stocks for vol at the moment and this > is > > > > where > > > > > the O'nite action seems to happen. > > > > > > > > > > (one the US opens this list will be populated with high vol > > stocks) > > > > > > > > > > http://finance.yahoo.com/actives?e=us > > > > > > > > > > The open gap on the intraday chart was actually o'nite trade. > > > > > > > > > > http://finance.yahoo.com/q/bc?s=C&t=5d > > > > > > > > > > > > > > > > > > > > --- In [email protected], "elegalpublishing" > > > > > <mark.a.brand@> wrote: > > > > > > > > > > > > Hi Brian: > > > > > > > > > > > > Thanks a lot for your response, its good because you made > me > > > > > actually > > > > > > count the out of hours volume which i had not actually done > > > > prior. > > > > > As > > > > > > you found it is about 2% which I can live with. > > > > > > > > > > > > <snip> > > > > > > I seem to recall adding the out of hours vol and it came > to > > > > > approx > > > > > > <2% of daily vol ... is that what you get? > > > > > > </snip> > > > > > > > > > > > > One example i did was 4M on 114M total volume > > > > > > > > > > > > <snip> > > > > > > are you massaging the data into 1 min bars before > importing > > > > into > > > > > AB? > > > > > > </snip> > > > > > > > > > > > > yes > > > > > > > > > > > > <snip> > > > > > > - briefly what language are you using and how are you going > > about > > > > > it? > > > > > > </snip> > > > > > > > > > > > > gawk and bash under cygwin, i can send the scripts to you > if > > you > > > > are > > > > > > interested. > > > > > > > > > > > > <snip> > > > > > > If that is what you are doing then it could be a good idea > > > > because > > > > > > the pre-market data contains trade corrections from > previous > > > > days > > > > > > and I found I had to delete all of that because it wasn't > > > > palatable > > > > > to > > > > > > AB's import method. > > > > > > </snip> > > > > > > > > > > > > I clean all this before import into AB. > > > > > > > > > > > > <snip> > > > > > > Have you successfully imported any of the data into AB in > > the > > > > > tick > > > > > > format? > > > > > > </snip> > > > > > > > > > > > > Yes, no problems with that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is solved for me - I will just ignore out of hours > > volume. > > > > > > > > > > > > Thanks again Brian. > > > > > > > > > > > > Best Regards > > > > > > Mark > > > > > > > > > > > > > > > > > > > > >
