My understanding of JD is that its like JDB which is like a relational
database. Relational databases typically return flat structures. If the
query is returning a flat array (table of data) then converting to JSON
should be greatly simplified.
I may suggest that JSON may not be the right format for large queries as
it's heavy with the key/value pair repeating for every row.
I'll often return just the data in javascript arrays [['abc',1],['xyz',2]]
instead of [{"name":"abc","id":1},{"name":"xyz","id":2}]
If the consumer is javascript then consuming a TAB or CSV delimited format
is straightforward as well and can likely be easily generated from JD
On Fri, May 16, 2014 at 12:32 PM, Devon McCormick <[email protected]>wrote:
> Playing around with the online JSON displayer at
> http://json.bloople.net/#_output , I see that the problem goes both ways:
> there are JSON objects not directly representable in J - like heterogeneous
> vectors - and J arrays not directly representable in JSON - like matrixes,
> complex and rational numbers, as well as the empty numeric vector.
>
> Since JSON is the more general of the two, one could adopt a convention
> like that suggested by Raul to allow any J array to be converted into JSON.
> The inverse operation would be more problematic. However, any particular
> case of something like a heterogeneous vector could be dealt with ad-hoc:
> trying to generalize this seems like unnecessary work.
>
>
> On Thu, May 15, 2014 at 9:31 PM, Raul Miller <[email protected]>
> wrote:
>
> > You might want to try something like:
> >
> > {
> > "type": "literal",
> > "shape": [5,5],
> > "ravel": "ABCDEFGHIJKLMNOPQRSTUVWXY"
> > }
> >
> > This makes for a fairly straightforward way of representing boxes and
> > whatever else.
> >
> > That said, I'm not sure if you are going to want to support every kind of
> > data which J has (complex number? rational numbers? sparse arrays?
> > symbols?)
> >
> > Thanks,
> >
> > --
> > Raul
> >
> >
> > On Thu, May 15, 2014 at 9:22 PM, Marshall Lochbaum <[email protected]
> > >wrote:
> >
> > > We do want the JSON output to be readable in Javascript and the like
> > > (and in fact JD already has a format using 3!:1 if the output only
> needs
> > > to be read by J). When I referred to dec_json as an inverse of enc_json
> > > in my original message, I was trying to communicate that no data should
> > > be lost in conversion to JSON.
> > >
> > > I suppose I underestimated how rare it is to have your data in J
> already
> > > and want to send it to other languages. We will likely modify
> > > convert/json so that it makes an object with shape and ravel attributes
> > > as you suggest.
> > >
> > > Marshall
> > >
> > > On Fri, May 16, 2014 at 08:28:36AM +0800, bill lam wrote:
> > > > If both client and server are in J, using json is over-skilled. Exact
> > > data type mapping between json and J is impossible, in particular J
> does
> > > not support array of array without using boxing while json does not
> > > understand boxing. IIRC json only support rank-1 array.
> > > >
> > > > A simple workaround to your problem would be passing both the shape
> and
> > > the razed data.
> > > >
> > > > On 16.05.2014, at 4:53, Marshall Lochbaum <[email protected]>
> > wrote:
> > > >
> > > > > JD has code to set up a server which receives JD queries and
> returns
> > > > > their results. We would like to support JSON output in addition to
> > the
> > > > > serialized J and text formats which are currently supported.
> > > > >
> > > > > Obviously sending only the first row of a result is no good, and
> > > sending
> > > > > json which is decoded into a bunch of boxes is quite inefficient.
> > > > >
> > > > > It sounds like you (and possibly the creators of convert/json) have
> > > > > spent more time working with JSON data in J than the other way
> > around.
> > > > > If the answer to my question is that the feature that I'm looking
> for
> > > > > doesn't currently exist, then that's still an answer and I will
> start
> > > > > working on modifying the convert/json code.
> > > > >
> > > > > Marshall
> > > > >
> > > > > On Thu, May 15, 2014 at 04:19:22PM -0400, Raul Miller wrote:
> > > > >> There are many ways of representing J's nouns in json, and many
> ways
> > > > >> of representing json in J's nouns. But to convert between them in
> a
> > > > >> consistent fashion, you have to live some simplifications.
> > > > >>
> > > > >> For example, a J noun consists of type, shape and a list of
> values.
> > So
> > > > >> to fully represent a J noun in JSON all three of these should be
> > > > >> emitted (and, for boxed arrays, this should be done recursively).
> > But
> > > > >> for most purposes this is an unnecessary complication.
> > > > >>
> > > > >> Similarly, if we went with this representation, we then have two
> > > > >> different classes of json to be read back in. "Everything" and
> "the
> > > > >> subset which could have been a representation of a J noun".
> > > > >>
> > > > >> Going the other direction, we can have a noun which is an exact
> > > > >> representation of the value represented by arbitrary JSON
> (something
> > > > >> other than a list of characters) and we have nouns in general. And
> > the
> > > > >> translation of a noun which is an exact representation of a JSON
> > data
> > > > >> structure would be translated to JSON using a different mechanism
> > than
> > > > >> the type/shape/list-of-values mechanism.
> > > > >>
> > > > >> Do you see how this pans out?
> > > > >>
> > > > >> Personally, I don't even parse the json which I'm currently
> working
> > > > >> with. Instead, I treat it as a sequence of lines, where each line
> > is a
> > > > >> key/value pair. The key is between the first pair of double quotes
> > on
> > > > >> the line, the value is between the second pair of double quotes.
> If
> > I
> > > > >> was using JSON to transport arrays I would, of course, do this
> > > > >> differently.
> > > > >>
> > > > >> So... getting back to your case: what is the problem you are
> trying
> > to
> > > > >> solve? If you start with the solution before you have identified
> > your
> > > > >> problem, you'll typically wind up solving the wrong problem.
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> --
> > > > >> Raul
> > > > >>
> > > > >>
> > > > >> On Thu, May 15, 2014 at 3:30 PM, Marshall Lochbaum <
> > > [email protected]> wrote:
> > > > >>> The addon convert/json ignores all but the first element of a
> > > > >>> (non-boxed) array when using enc_json.
> > > > >>>
> > > > >>> enc_json 3 2 1
> > > > >>> 3
> > > > >>> enc_json 3 2 1;4 3 2
> > > > >>> [3,4]
> > > > >>> enc_json <"_1 ]3 2 1
> > > > >>> [3,2,1]
> > > > >>> enc_json <"_1 ]i.3 3
> > > > >>> [0,3,6]
> > > > >>>
> > > > >>> Is there a reason for this behavior? I realize that encoding 3 2
> 1
> > as
> > > > >>> "[3,2,1]" is ambiguous (since (<"_1]3 2 1) has the same
> encoding),
> > > but
> > > > >>> encoding as "3" seems even worse since it destroys data rather
> than
> > > > >>> formatting.
> > > > >>>
> > > > >>> Is there a way around it without modifying the convert/json code?
> > Of
> > > > >>> course applying <"0 L:0 beforehand will suffice, but it is rather
> > > > >>> inefficient and decodes to a boxed structure. The best possible
> > > solution
> > > > >>> would be one that encodes any J noun unambiguously into
> reasonable
> > > JSON,
> > > > >>> but either of the following would be useful:
> > > > >>> - A method of conversion where dec_json is an exact inverse to
> > > end_json,
> > > > >>> but the intermediate JSON may not make sense (e.g. it has extra
> > > > >>> decorations for J arrays).
> > > > >>> - A way of making any J noun a into another noun a1 such that
> > > > >>> a -: dec_json enc_json a1 .
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Marshall
> > > > >>>
> > > ----------------------------------------------------------------------
> > > > >>> For information about J forums see
> > > http://www.jsoftware.com/forums.htm
> > > > >>
> > ----------------------------------------------------------------------
> > > > >> For information about J forums see
> > > http://www.jsoftware.com/forums.htm
> > > > >
> > ----------------------------------------------------------------------
> > > > > For information about J forums see
> > http://www.jsoftware.com/forums.htm
> > > >
> ----------------------------------------------------------------------
> > > > For information about J forums see
> http://www.jsoftware.com/forums.htm
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
>
>
>
> --
> Devon McCormick, CFA
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
