Hi Andrea,

I was just giving some context for the links Jim posted. Bottom line, our functions will work on json strings, using jayway backed by gson (both active libraries I believe, and gson already on the geotools classpath), but are optimized for our serialized forms. If you want to use them as a template, you probably don't want those optimizations included.

Thanks,

Emilio

On 1/24/19 11:25 AM, Andrea Aime wrote:
Hi,
it's good that you have low level optimizations, but... it does not help with the work I'm going to do, where I just need to handle JSON as strings, and select attributes with some possibly well maintained dependency. Or did I miss something?

Cheers
Andrea


On Thu, Jan 24, 2019 at 4:24 PM Emilio Lahr-Vivaz <[email protected] <mailto:[email protected]>> wrote:

    Hi Andrea, Jim,

    (resending after subscribing with the correct email)

    For some more reference - we use jayway with gson as a provider,
    but only as a fallback. The filter functions we have are optimized
    for reading kryo-serialized json. In GeoMesa, you can indicate
    that a string field is json by adding a user data hint: "json" ->
    "true". When we detect that hint, we will persist the json string
    by parsing it and serializing the parsed form with kryo. Our json
    functions will then select out the appropriate json path without
    deserializing and parsing the string again. To support this, we
    implemented our own json path parser. The functions will still
    work against regular strings, but will require parsing the string
    into json to evaluate.

    For reference, here's our json path parser:

    
https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathParser.scala

    And here's our kryo json serialization, which is based on the bson
    format and supports selecting by a parsed json path:

    
https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/KryoJsonSerialization.scala

    Thanks,

    Emilio

    On 1/24/19 10:03 AM, Andrea Aime wrote:
    Hi Jim,
    it terms of useful, it does, but I'd like to avoid depending at
    runtime on a library that has dead dependencies, as discussed
    above... we
    already have many, and believe we should try to put some effort
    in to reduce them instead of growing them.
    Pity because I like jawyway json-path for testing... I commented
    here, but I'm not really expecting an answer:
    https://github.com/json-path/JsonPath/issues/513

    Cheers
    Andrea


    On Thu, Jan 24, 2019 at 3:59 PM Jim Hughes <[email protected]
    <mailto:[email protected]>> wrote:

        Hi Andrea,

        We've done something similar in GeoMesa.  We needed to
        implement a JsonPathPropertyAccessor[1] and a
        JsonPathFilterFunction[2].  We used Jayway for that and maybe
        json-simple2 somewhere along the way.

        The unit tests cover lots of little corner cases we've
        already fussed through.

        Does any of this look useful?

        Cheers,

        Jim

        1.
        
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessor.scala
        
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessorTest.scala
        2.
        
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunction.scala
        
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunctionTest.scala



        On 1/23/19 4:29 PM, Torben Barsballe wrote:
        For completeness, we actually have a few more JSON libraries
        in GeoTools/GeoServer in addition to what you listed:

          * Google GSON for MongoDB and ArcGIS REST (in GeoTools)
            and for WMS TopoJSON format and the GeoGig community
            module (in GeoServer), which is active.
          * Jettison 1.0.1 (via XStream) for the JSON portion of the
            REST API (Only in GeoServer). While the project is
            active, we're stuck on a 10-year old version for
            backwards compatibility reasons.

        GSON is primarily intended for transforming Java to and from
        JSON , but it does have minimal JsonPath
        
<https://google.github.io/gson/apidocs/com/google/gson/stream/JsonReader.html#getPath-->
        support (With more robust integration added by the Jayway
        implementation that you already mentioned). There's also a
        seemingly abandoned fork
        
<https://github.com/johnnylambada/gson/commit/344bd2fd4911146e761926d5ee1b441ba6bb9cfa>
        that adds Json-Pointer support.
        Overall, it seems like GSON doesn't quite have the
        capabilities you are looking for, but it is already a
        dependency of some GeoTools modules so maybe consider it?

        More generally, I'd definitely say go for a JSON library we
        already depend upon (given that we have too many already),
        be it Jackson or something else.
        I think it's reasonable to add Jackson to gt-main. Having a
        JSON library as part of a core module might encourage others
        to use it rather than adding their own (although I rather
        doubt it).

        Torben





        On Wed, Jan 23, 2019 at 8:24 AM Ian Turton
        <[email protected] <mailto:[email protected]>> wrote:

            I'd say go for Jackson as a main dependency and then
            someone (or me) could try to bring the other JSON
            modules uptodate by using it, I might even merge the two
            at the same time, but not until the weekend :-)

            Ian

            On Wed, 23 Jan 2019 at 15:45, Andrea Aime
            <[email protected]
            <mailto:[email protected]>> wrote:

                Hi all,
                as you probably know GeoTools is now sometimes
                carrying around JSON in feature attributes
                (e.g., see JSON support in PostGIS data store).
                The JSON is represented as a String.

                Now, I'd like to add a filter function to parse and
                extract specific properties out of the JSON
                string... which
                requires new dependencies. And boy we have a mess of
                them in the classpath already:

                  * json-simple, used in geojson and mbstyles, but
                    dead, last release is 2012
                  * json-lib, used in GeoServer for geojson
                    generation, and also dead, last release is 2010
                  * jackson, used by Spring and directly by some
                    GeoServer community modules, with recent
                    releases and latest commits of only a few days ago


                There are basically two languages to extract stuff
                from JSON, json-path and json-pointer.
                json-path is the one I would have preferred as it's
                more powerful, but its situation library wise is not
                exactly in great shape, the jayway implementation
                uses yet another json java library, json-smart2,
                also dead.
                There are a couple of other options for json-path,
                either dead or with a bizzarre "do not evil" licence.

                json-pointer is instead supported directly by
                jackson-core, which is well alive, and comes with a
                300kb jar
                that has no external dependencies. This seems the
                best bet, just one hesitation... should I add a
                jackson-core
                to gt-main? On one side, it seems a large-ish
                dependency for just a function, on the other side
                having a single
                class new module also seems a bit overkill.

                Opinions?

                Cheers
                Andrea

                == GeoServer Professional Services from the experts!
                Visit http://goo.gl/it488V for more information. ==
                Ing. Andrea Aime @geowolf Technical Lead
                GeoSolutions S.A.S. Via di Montramito 3/A 55054
                Massarosa (LU) phone: +39 0584 962313 fax: +39 0584
                1660272 mob: +39 339 8844549
                http://www.geo-solutions.it
                http://twitter.com/geosolutions_it
                -------------------------------------------------------
                /Con riferimento alla normativa sul trattamento dei
                dati personali (Reg. UE 2016/679 - Regolamento
                generale sulla protezione dei dati “GDPR”), si
                precisa che ogni circostanza inerente alla presente
                email (il suo contenuto, gli eventuali allegati,
                etc.) è un dato la cui conoscenza è riservata al/i
                solo/i destinatario/i indicati dallo scrivente. Se
                il messaggio Le è giunto per errore, è tenuta/o a
                cancellarlo, ogni altra operazione è illecita. Le
                sarei comunque grato se potesse darmene notizia.
                This email is intended only for the person or entity
                to which it is addressed and may contain information
                that is privileged, confidential or otherwise
                protected from disclosure. We remind that - as
                provided by European Regulation 2016/679 “GDPR” -
                copying, dissemination or use of this e-mail or the
                information herein by anyone other than the intended
                recipient is prohibited. If you have received this
                email by mistake, please notify us immediately by
                telephone or e-mail./

                _______________________________________________
                GeoTools-Devel mailing list
                [email protected]
                <mailto:[email protected]>
                https://lists.sourceforge.net/lists/listinfo/geotools-devel



-- Ian Turton
            _______________________________________________
            GeoTools-Devel mailing list
            [email protected]
            <mailto:[email protected]>
            https://lists.sourceforge.net/lists/listinfo/geotools-devel



        _______________________________________________
        GeoTools-Devel mailing list
        [email protected]  
<mailto:[email protected]>
        https://lists.sourceforge.net/lists/listinfo/geotools-devel


        _______________________________________________
        GeoTools-Devel mailing list
        [email protected]
        <mailto:[email protected]>
        https://lists.sourceforge.net/lists/listinfo/geotools-devel



--
    Regards, Andrea Aime == GeoServer Professional Services from the
    experts! Visit http://goo.gl/it488V for more information. == Ing.
    Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di
    Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax:
    +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it
    http://twitter.com/geosolutions_it
    ------------------------------------------------------- /Con
    riferimento alla normativa sul trattamento dei dati personali
    (Reg. UE 2016/679 - Regolamento generale sulla protezione dei
    dati “GDPR”), si precisa che ogni circostanza inerente alla
    presente email (il suo contenuto, gli eventuali allegati, etc.) è
    un dato la cui conoscenza è riservata al/i solo/i destinatario/i
    indicati dallo scrivente. Se il messaggio Le è giunto per errore,
    è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
    sarei comunque grato se potesse darmene notizia. This email is
    intended only for the person or entity to which it is addressed
    and may contain information that is privileged, confidential or
    otherwise protected from disclosure. We remind that - as provided
    by European Regulation 2016/679 “GDPR” - copying, dissemination
    or use of this e-mail or the information herein by anyone other
    than the intended recipient is prohibited. If you have received
    this email by mistake, please notify us immediately by telephone
    or e-mail./




--

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- /Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail./


_______________________________________________
GeoTools-Devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Reply via email to