Currently the 'binary' data type is synonymous with 'varchar' and is used for naming convention compatibility with popular data bases.
'binary' is intended to store Raw or BLOB data, such as images of arbitrary binary files. Saying "external form" means the actual binary representation in char list using 3!:1 etc. However, it might make sense to store J nouns of any shape and type there. This would allow to store and extract the values without additional transformation into external format. Maybe a different type name should used like 'native' or so. > From: Bill Harris <[email protected]> > > On Wed, January 20, 2010 3:09 pm, Oleg Kobchenko wrote: > > There is type "binary" (similar to varchar), which > > is intended for things like images etc. > > > > So you should be able to store values like J nouns > > (of any shape and type, even boxed, in external binary form) > > or linear array of scalar type (e.g. list of doubles), > > and then save and restore accordingly. > testt3=: Create__testd 'id3' > InsertCols__testd 'id3'; 0 : 0 > timeseries binary > ) > Is that what you would expect? Did I not understand what binary means? Yes, but as of now you should convert it to/from binary rep 3!:x. > Bill Lam asked: > > > Did you try > > > Insert__testd 'id3';<<i. 3 4 > > I just did (it does seem obvious), and it works -- sort of. Inserting > <<i.3 4 5 puts all the numbers in, but the shape of the returned object > when I retrieve it is 1 1 60. So I can probably save one time series per > field in a record, but I can't save an array of time series in a cell in a > record without a bit of extra care. As of now you can convert a boxed list into a binary representation char list. > > While you explore this, it would be good to make it into > > an illustrative example to append to JDB test scripts. > > Agreed. Assuming my manager doesn't have a problem with it, I could do that. Here I mean to make a generic small example like the examples in JDB, so you don't have to deal with concrete data. You need to show case various uses of binary data, including J nouns of different shapes and sizes. For example it could be like a small JOD stored in JDB. > > The above won't allow any in-DB time series analysis, since > > the data is opaque that way. An alternative is to > > store the data relationally, in hope that the inverted > > representation will already be optimal enough. This would > > allow some TS logic in queries. There isn't much functional > > support in existing J-QL, but this use case may present > > a good opportunity. > > Say more about "storing the data relationally." Do you mean store each > datapoint in the series as a record? I figured that would be hard to deal > with (select * from series1 where time in (start, end) in informal SQL), > and, as the times don't necessarily have meaning between series, I figured > that wouldn't help much. Perhaps you have something different in mind. It could be a header table and series-data table with a single column FK, so each row will have just an index and not a time value. If you need time you can store and derive it from the header table. > What I'll likely be querying is something like > > select seriestype1 from table where attribute1 = x and attribute2 = y; > > to get an array of series that I can then process outside of the SQL > paradigm. Interestingly, KDB+ while storing time series data relationally, will return lists of numbers if aggregation operation is omitted. E.g. Q: select max(data.values) by series.name where series.name in ('S1','S2') name max ------------- S1 123 S2 456 Q: select data.values by series.name where series.name in ('S1','S2') name values ----------------------- S1 2 3 4 5 6 6 7 ... S2 6 5 4 3 2 1 2 ... > > Do you have a roster of time series analysis you plan to use? > > I'll be collecting energy-related data. This is interesting: (smart) meter data? > > E.g. select a fastest growing quantity in a given month, maybe. > > I could certainly dummy up something as an example. Yes, you could be as abstract as you need. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
