Thinking about it a bit more it might be solvable if InfluxDB would support custom fill() functions, like e.g. requested in #6891.
On Tuesday, June 21, 2016 at 8:33:43 PM UTC+2, [email protected] wrote: > > Ok, let's consider this data: > > time lvl price side size > 1412578805000000000 1 91.38 B 359 > 1412578818000000000 1 91.64 A 119 > 1412578821000000000 1 91.09 B 119 > There data points T1, T2, T3. > > At time T1 there is only one data point available. One bid with a price of > 91.38 and a size of 359. > > Since, no ask price/size is available at T1 there resulting row is: > time bidPrice bidSize askPrice askSize > 1412578805000000000 91.38 359 > at T2 an ask price is coming in. The bid price of T1 is still the latest > available bid. There the result should be: > 1412578818000000000 91.38 359 91.64 119 > > Finally, at T3 a new bid is coming in; with the ask from T2 still the > latest one. The result should then be: > 1412578821000000000 91.09 119 91.64 119 > Or as complete table: > time bidPrice bidSize askPrice askSize > 1412578805000000000 91.38 359 > 1412578818000000000 91.38 359 91.64 119 > 1412578821000000000 91.09 119 91.64 119 > A good pointer is also this stackoverflow asof-join-for-timeseries-data > <http://stackoverflow.com/questions/12322289/kdb-like-asof-join-for-timeseries-data-in-pandas> > thread. > > Hope this helps, > Marc > > On Tuesday, June 21, 2016 at 8:15:45 PM UTC+2, Sean Beckett wrote: >> >> I think I don't understand your example, then. I don't have a background >> in finance so I need something simpler. >> >> Can you shorten it to just two A/B points and the output you'd want? I'm >> not understanding how the 10 original points can be combined to get 10 new >> points with the same timestamps, but I think a smaller example will help. >> Also, why is "sym" different in the two outputs? >> >> >> >> On Tue, Jun 21, 2016 at 10:48 AM, <[email protected]> wrote: >> >>> Hi Sean, >>> >>> it is organized by time, it is just "composed" of the data from the two >>> tags 'A' and 'B' which are combined/merged/joined/... based on their time. >>> Basically, as they had been at any given time T. >>> The question for me is, if this is doable with InfluxDB at all. Maybe by >>> not using tags for bid and ask, but own measurements or something. I am >>> completely open to changes to the data model, since I am currently just >>> "playing around" with InfluxDB to find out the strengths and weaknesses; >>> and ultimately if it fits to my use cases. >>> >>> But, currently I have the impression that I have to pull all the data >>> (which is a lot) into some sort of pre-processor for my model to generate >>> the necessary data. >>> >>> >>> On Tuesday, June 21, 2016 at 5:51:50 PM UTC+2, Sean Beckett wrote: >>>> >>>> There are no JOIN commands in InfluxQL. Your output has a timestamp as >>>> a column but it is not organized by timestamps, it's organized by bidPrice >>>> and askPrice. InfluxQL only supports queries that output a continuous >>>> timeline. >>>> >>>> On Mon, Jun 20, 2016 at 11:41 PM, <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I currently trying to build the Top of the book >>>>> <https://en.wikipedia.org/wiki/Order_book_(trading)> out of full >>>>> market depth and struggling a bit. >>>>> >>>>> I am currently have one measurement called book, with two tags sym and >>>>> side (bid or ask): >>>>> >>>>> > show series >>>>> key >>>>> book,side=A,sym=DE0007236101 >>>>> book,side=B,sym=DE0007236101 >>>>> >>>>> which contains data like: >>>>> >>>>> > select * from book where sym='DE0007236101' and lvl=1 limit 10 >>>>> >>>>> name: book >>>>> ---------- >>>>> time lvl noOrders price seqNo side >>>>> size sym >>>>> 1412578805000000000 1 1 91.38 499 B >>>>> 359 DE0007236101 >>>>> 1412578818000000000 1 1 91.64 985 A >>>>> 119 DE0007236101 >>>>> 1412578821000000000 1 1 91.09 1034 B >>>>> 119 DE0007236101 >>>>> 1412578831000000000 1 1 91.12 1222 B >>>>> 187 DE0007236101 >>>>> 1412578843000000000 1 1 91.33 1404 A >>>>> 64 DE0007236101 >>>>> 1412578857000000000 1 2 91.31 1584 B >>>>> 144 DE0007236101 >>>>> 1412578871000000000 1 1 91.39 1811 A >>>>> 18 DE0007236101 >>>>> 1412578892000000000 1 1 91.39 1920 A >>>>> 119 DE0007236101 >>>>> 1412578900000000000 1 3 91.39 1938 A >>>>> 516 DE0007236101 >>>>> 1412578901000000000 1 4 91.39 1939 A >>>>> 601 DE0007236101 >>>>> >>>>> Which shows that bid and asks are not arriving at the time time. This >>>>> means for any given time T the latest valid bid and ask (for level 1) >>>>> have >>>>> to be joined. >>>>> The result I would expect is: >>>>> >>>>> time bidOrders bidPrice bidSize askPrice askSize askOrders sym >>>>> 1412578805000000000 1 91.38 359 DE0007236101 >>>>> 1412578818000000000 1 91.38 359 91.64 119 1 DE0007236102 >>>>> 1412578821000000000 1 91.09 119 91.64 119 1 DE0007236103 >>>>> 1412578831000000000 1 91.12 187 91.64 119 1 DE0007236104 >>>>> 1412578843000000000 1 91.12 187 91.33 64 1 DE0007236105 >>>>> 1412578857000000000 2 91.31 144 91.33 64 1 DE0007236106 >>>>> 1412578871000000000 2 91.31 144 91.39 18 1 DE0007236107 >>>>> 1412578892000000000 2 91.31 144 91.39 119 1 DE0007236108 >>>>> 1412578900000000000 2 91.31 144 91.39 516 3 DE0007236109 >>>>> 1412578901000000000 2 91.31 144 91.39 601 4 DE0007236110 >>>>> Unfortunately, I have not been able to formulate this "temporal join" >>>>> on the same measurement in any way. >>>>> >>>>> I would appreciate any help input on this... >>>>> >>>>> Thanks, >>>>> Marc >>>>> >>>>> -- >>>>> Remember to include the InfluxDB version number with all issue reports >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "InfluxDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/influxdb/395d6d3d-3b46-425d-b322-5babd6cc8c3b%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/influxdb/395d6d3d-3b46-425d-b322-5babd6cc8c3b%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Sean Beckett >>>> Director of Support and Professional Services >>>> InfluxDB >>>> >>> -- >>> Remember to include the InfluxDB version number with all issue reports >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "InfluxDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/influxdb. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/influxdb/7520883e-ec0a-4e85-b10f-60d6d706984a%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/influxdb/7520883e-ec0a-4e85-b10f-60d6d706984a%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Sean Beckett >> Director of Support and Professional Services >> InfluxDB >> > -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/b0342a0a-c249-475b-9644-0e09c8eb7ea5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
