Hi, it's good that you have low level optimizations, but... it does not help with the work I'm going to do, where I just need to handle JSON as strings, and select attributes with some possibly well maintained dependency. Or did I miss something?
Cheers Andrea On Thu, Jan 24, 2019 at 4:24 PM Emilio Lahr-Vivaz <elahrvi...@ccri.com> wrote: > Hi Andrea, Jim, > > (resending after subscribing with the correct email) > > For some more reference - we use jayway with gson as a provider, but only > as a fallback. The filter functions we have are optimized for reading > kryo-serialized json. In GeoMesa, you can indicate that a string field is > json by adding a user data hint: "json" -> "true". When we detect that > hint, we will persist the json string by parsing it and serializing the > parsed form with kryo. Our json functions will then select out the > appropriate json path without deserializing and parsing the string again. > To support this, we implemented our own json path parser. The functions > will still work against regular strings, but will require parsing the > string into json to evaluate. > > For reference, here's our json path parser: > > > https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathParser.scala > > And here's our kryo json serialization, which is based on the bson format > and supports selecting by a parsed json path: > > > https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.2.0/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/KryoJsonSerialization.scala > > Thanks, > > Emilio > > On 1/24/19 10:03 AM, Andrea Aime wrote: > > Hi Jim, > it terms of useful, it does, but I'd like to avoid depending at runtime on > a library that has dead dependencies, as discussed above... we > already have many, and believe we should try to put some effort in to > reduce them instead of growing them. > Pity because I like jawyway json-path for testing... I commented here, but > I'm not really expecting an answer: > https://github.com/json-path/JsonPath/issues/513 > > Cheers > Andrea > > > On Thu, Jan 24, 2019 at 3:59 PM Jim Hughes <jn...@ccri.com> wrote: > >> Hi Andrea, >> >> We've done something similar in GeoMesa. We needed to implement a >> JsonPathPropertyAccessor[1] and a JsonPathFilterFunction[2]. We used >> Jayway for that and maybe json-simple2 somewhere along the way. >> >> The unit tests cover lots of little corner cases we've already fussed >> through. >> >> Does any of this look useful? >> >> Cheers, >> >> Jim >> 1. >> https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessor.scala >> >> https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathPropertyAccessorTest.scala >> 2. >> https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/main/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunction.scala >> >> https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/json/JsonPathFilterFunctionTest.scala >> >> >> On 1/23/19 4:29 PM, Torben Barsballe wrote: >> >> For completeness, we actually have a few more JSON libraries in >> GeoTools/GeoServer in addition to what you listed: >> >> - Google GSON for MongoDB and ArcGIS REST (in GeoTools) and for WMS >> TopoJSON format and the GeoGig community module (in GeoServer), which is >> active. >> - Jettison 1.0.1 (via XStream) for the JSON portion of the REST API >> (Only in GeoServer). While the project is active, we're stuck on a 10-year >> old version for backwards compatibility reasons. >> >> GSON is primarily intended for transforming Java to and from JSON , but >> it does have minimal JsonPath >> <https://google.github.io/gson/apidocs/com/google/gson/stream/JsonReader.html#getPath--> >> support (With more robust integration added by the Jayway implementation >> that you already mentioned). There's also a seemingly abandoned fork >> <https://github.com/johnnylambada/gson/commit/344bd2fd4911146e761926d5ee1b441ba6bb9cfa> >> that adds Json-Pointer support. >> Overall, it seems like GSON doesn't quite have the capabilities you are >> looking for, but it is already a dependency of some GeoTools modules so >> maybe consider it? >> >> More generally, I'd definitely say go for a JSON library we already >> depend upon (given that we have too many already), be it Jackson or >> something else. >> I think it's reasonable to add Jackson to gt-main. Having a JSON library >> as part of a core module might encourage others to use it rather than >> adding their own (although I rather doubt it). >> >> Torben >> >> >> >> >> >> On Wed, Jan 23, 2019 at 8:24 AM Ian Turton <ijtur...@gmail.com> wrote: >> >>> I'd say go for Jackson as a main dependency and then someone (or me) >>> could try to bring the other JSON modules uptodate by using it, I might >>> even merge the two at the same time, but not until the weekend :-) >>> >>> Ian >>> >>> On Wed, 23 Jan 2019 at 15:45, Andrea Aime <andrea.a...@geo-solutions.it> >>> wrote: >>> >>>> Hi all, >>>> as you probably know GeoTools is now sometimes carrying around JSON in >>>> feature attributes >>>> (e.g., see JSON support in PostGIS data store). >>>> The JSON is represented as a String. >>>> >>>> Now, I'd like to add a filter function to parse and extract specific >>>> properties out of the JSON string... which >>>> requires new dependencies. And boy we have a mess of them in the >>>> classpath already: >>>> >>>> - json-simple, used in geojson and mbstyles, but dead, last release >>>> is 2012 >>>> - json-lib, used in GeoServer for geojson generation, and also >>>> dead, last release is 2010 >>>> - jackson, used by Spring and directly by some GeoServer community >>>> modules, with recent releases and latest commits of only a few days ago >>>> >>>> >>>> There are basically two languages to extract stuff from JSON, json-path >>>> and json-pointer. >>>> json-path is the one I would have preferred as it's more powerful, but >>>> its situation library wise is not >>>> exactly in great shape, the jayway implementation uses yet another json >>>> java library, json-smart2, also dead. >>>> There are a couple of other options for json-path, either dead or with >>>> a bizzarre "do not evil" licence. >>>> >>>> json-pointer is instead supported directly by jackson-core, which is >>>> well alive, and comes with a 300kb jar >>>> that has no external dependencies. This seems the best bet, just one >>>> hesitation... should I add a jackson-core >>>> to gt-main? On one side, it seems a large-ish dependency for just a >>>> function, on the other side having a single >>>> class new module also seems a bit overkill. >>>> >>>> Opinions? >>>> >>>> Cheers >>>> Andrea >>>> >>>> == GeoServer Professional Services from the experts! Visit >>>> http://goo.gl/it488V for more information. == Ing. Andrea Aime >>>> @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 >>>> Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 >>>> 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it >>>> ------------------------------------------------------- *Con >>>> riferimento alla normativa sul trattamento dei dati personali (Reg. UE >>>> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si >>>> precisa che ogni circostanza inerente alla presente email (il suo >>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >>>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia. >>>> This email is intended only for the person or entity to which it is >>>> addressed and may contain information that is privileged, confidential or >>>> otherwise protected from disclosure. We remind that - as provided by >>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this >>>> e-mail or the information herein by anyone other than the intended >>>> recipient is prohibited. If you have received this email by mistake, please >>>> notify us immediately by telephone or e-mail.* >>>> _______________________________________________ >>>> GeoTools-Devel mailing list >>>> GeoTools-Devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/geotools-devel >>>> >>> >>> >>> -- >>> Ian Turton >>> _______________________________________________ >>> GeoTools-Devel mailing list >>> GeoTools-Devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/geotools-devel >>> >> >> >> _______________________________________________ >> GeoTools-Devel mailing >> listGeoTools-Devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/geotools-devel >> >> >> _______________________________________________ >> GeoTools-Devel mailing list >> GeoTools-Devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geotools-devel >> > > > -- > > Regards, Andrea Aime == GeoServer Professional Services from the experts! > Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime > @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 > Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 > 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it > ------------------------------------------------------- *Con riferimento > alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - > Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni > circostanza inerente alla presente email (il suo contenuto, gli eventuali > allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i > destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per > errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le > sarei comunque grato se potesse darmene notizia. This email is intended > only for the person or entity to which it is addressed and may contain > information that is privileged, confidential or otherwise protected from > disclosure. We remind that - as provided by European Regulation 2016/679 > “GDPR” - copying, dissemination or use of this e-mail or the information > herein by anyone other than the intended recipient is prohibited. If you > have received this email by mistake, please notify us immediately by > telephone or e-mail.* > > > -- Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- *Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.*
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
