Hi Andrea, Jim,

(resending after subscribing with the correct email)

For some more reference - we use jayway with gson as a provider, but only as a fallback. The filter functions we have are optimized for reading kryo-serialized json. In GeoMesa, you can indicate that a string field is json by adding a user data hint: "json" -> "true". When we detect that hint, we will persist the json string by parsing it and serializing the parsed form with kryo. Our json functions will then select out the appropriate json path without deserializing and parsing the string again. To support this, we implemented our own json path parser. The functions will still work against regular strings, but will require parsing the string into json to evaluate.

For reference, here's our json path parser:

https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathParser.scala

And here's our kryo json serialization, which is based on the bson format and supports selecting by a parsed json path:

https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/KryoJsonSerialization.scala

Thanks,

Emilio

On 1/24/19 10:03 AM, Andrea Aime wrote:
Hi Jim,
it terms of useful, it does, but I'd like to avoid depending at runtime on a library that has dead dependencies, as discussed above... we already have many, and believe we should try to put some effort in to reduce them instead of growing them. Pity because I like jawyway json-path for testing... I commented here, but I'm not really expecting an answer: https://github.com/json-path/JsonPath/issues/513

Cheers
Andrea


On Thu, Jan 24, 2019 at 3:59 PM Jim Hughes <jn...@ccri.com <mailto:jn...@ccri.com>> wrote:

    Hi Andrea,

    We've done something similar in GeoMesa.  We needed to implement a
    JsonPathPropertyAccessor[1] and a JsonPathFilterFunction[2].  We
    used Jayway for that and maybe json-simple2 somewhere along the way.

    The unit tests cover lots of little corner cases we've already
    fussed through.

    Does any of this look useful?

    Cheers,

    Jim

    1.
    
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessor.scala
    
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessorTest.scala
    2.
    
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunction.scala
    
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunctionTest.scala



    On 1/23/19 4:29 PM, Torben Barsballe wrote:
    For completeness, we actually have a few more JSON libraries in
    GeoTools/GeoServer in addition to what you listed:

      * Google GSON for MongoDB and ArcGIS REST (in GeoTools) and for
        WMS TopoJSON format and the GeoGig community module (in
        GeoServer), which is active.
      * Jettison 1.0.1 (via XStream) for the JSON portion of the REST
        API (Only in GeoServer). While the project is active, we're
        stuck on a 10-year old version for backwards compatibility
        reasons.

    GSON is primarily intended for transforming Java to and from JSON
    , but it does have minimal JsonPath
    
<https://google.github.io/gson/apidocs/com/google/gson/stream/JsonReader.html#getPath-->
    support (With more robust integration added by the Jayway
    implementation that you already mentioned). There's also a
    seemingly abandoned fork
    
<https://github.com/johnnylambada/gson/commit/344bd2fd4911146e761926d5ee1b441ba6bb9cfa>
    that adds Json-Pointer support.
    Overall, it seems like GSON doesn't quite have the capabilities
    you are looking for, but it is already a dependency of some
    GeoTools modules so maybe consider it?

    More generally, I'd definitely say go for a JSON library we
    already depend upon (given that we have too many already), be it
    Jackson or something else.
    I think it's reasonable to add Jackson to gt-main. Having a JSON
    library as part of a core module might encourage others to use it
    rather than adding their own (although I rather doubt it).

    Torben





    On Wed, Jan 23, 2019 at 8:24 AM Ian Turton <ijtur...@gmail.com
    <mailto:ijtur...@gmail.com>> wrote:

        I'd say go for Jackson as a main dependency and then someone
        (or me) could try to bring the other JSON modules uptodate by
        using it, I might even merge the two at the same time, but
        not until the weekend :-)

        Ian

        On Wed, 23 Jan 2019 at 15:45, Andrea Aime
        <andrea.a...@geo-solutions.it
        <mailto:andrea.a...@geo-solutions.it>> wrote:

            Hi all,
            as you probably know GeoTools is now sometimes carrying
            around JSON in feature attributes
            (e.g., see JSON support in PostGIS data store).
            The JSON is represented as a String.

            Now, I'd like to add a filter function to parse and
            extract specific properties out of the JSON string... which
            requires new dependencies. And boy we have a mess of them
            in the classpath already:

              * json-simple, used in geojson and mbstyles, but dead,
                last release is 2012
              * json-lib, used in GeoServer for geojson generation,
                and also dead, last release is 2010
              * jackson, used by Spring and directly by some
                GeoServer community modules, with recent releases and
                latest commits of only a few days ago


            There are basically two languages to extract stuff from
            JSON, json-path and json-pointer.
            json-path is the one I would have preferred as it's more
            powerful, but its situation library wise is not
            exactly in great shape, the jayway implementation uses
            yet another json java library, json-smart2, also dead.
            There are a couple of other options for json-path, either
            dead or with a bizzarre "do not evil" licence.

            json-pointer is instead supported directly by
            jackson-core, which is well alive, and comes with a 300kb jar
            that has no external dependencies. This seems the best
            bet, just one hesitation... should I add a jackson-core
            to gt-main? On one side, it seems a large-ish dependency
            for just a function, on the other side having a single
            class new module also seems a bit overkill.

            Opinions?

            Cheers
            Andrea

            == GeoServer Professional Services from the experts!
            Visit http://goo.gl/it488V for more information. == Ing.
            Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S.
            Via di Montramito 3/A 55054 Massarosa (LU) phone: +39
            0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
            http://www.geo-solutions.it
            http://twitter.com/geosolutions_it
            -------------------------------------------------------
            /Con riferimento alla normativa sul trattamento dei dati
            personali (Reg. UE 2016/679 - Regolamento generale sulla
            protezione dei dati “GDPR”), si precisa che ogni
            circostanza inerente alla presente email (il suo
            contenuto, gli eventuali allegati, etc.) è un dato la cui
            conoscenza è riservata al/i solo/i destinatario/i
            indicati dallo scrivente. Se il messaggio Le è giunto per
            errore, è tenuta/o a cancellarlo, ogni altra operazione è
            illecita. Le sarei comunque grato se potesse darmene
            notizia. This email is intended only for the person or
            entity to which it is addressed and may contain
            information that is privileged, confidential or otherwise
            protected from disclosure. We remind that - as provided
            by European Regulation 2016/679 “GDPR” - copying,
            dissemination or use of this e-mail or the information
            herein by anyone other than the intended recipient is
            prohibited. If you have received this email by mistake,
            please notify us immediately by telephone or e-mail./

            _______________________________________________
            GeoTools-Devel mailing list
            GeoTools-Devel@lists.sourceforge.net
            <mailto:GeoTools-Devel@lists.sourceforge.net>
            https://lists.sourceforge.net/lists/listinfo/geotools-devel



-- Ian Turton
        _______________________________________________
        GeoTools-Devel mailing list
        GeoTools-Devel@lists.sourceforge.net
        <mailto:GeoTools-Devel@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/geotools-devel



    _______________________________________________
    GeoTools-Devel mailing list
    GeoTools-Devel@lists.sourceforge.net  
<mailto:GeoTools-Devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geotools-devel


    _______________________________________________
    GeoTools-Devel mailing list
    GeoTools-Devel@lists.sourceforge.net
    <mailto:GeoTools-Devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geotools-devel



--

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- /Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail./


_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Reply via email to