Ciao Justing, please, read below.... Regards, Simone Giannecchini == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Simone Giannecchini @simogeo Founder/Director
GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 333 8128928 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. On Thu, Apr 14, 2016 at 6:18 PM, Justin Deoliveira <jdeol...@gmail.com> wrote: > Thanks for the input Simone. Comments inline below. > > On Thu, Apr 14, 2016 at 10:07 AM Simone Giannecchini > <simone.giannecch...@geo-solutions.it> wrote: >> >> See below.. >> >> Regards, >> Simone Giannecchini >> == >> GeoServer Professional Services from the experts! >> Visit http://goo.gl/it488V for more information. >> == >> Ing. Simone Giannecchini >> @simogeo >> Founder/Director >> >> GeoSolutions S.A.S. >> Via di Montramito 3/A >> 55054 Massarosa (LU) >> Italy >> phone: +39 0584 962313 >> fax: +39 0584 1660272 >> mob: +39 333 8128928 >> >> http://www.geo-solutions.it >> http://twitter.com/geosolutions_it >> >> ------------------------------------------------------- >> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 >> Le informazioni contenute in questo messaggio di posta elettronica e/o >> nel/i file/s allegato/i sono da considerarsi strettamente riservate. >> Il loro utilizzo è consentito esclusivamente al destinatario del >> messaggio, per le finalità indicate nel messaggio stesso. Qualora >> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo >> cortesemente di darcene notizia via e-mail e di procedere alla >> distruzione del messaggio stesso, cancellandolo dal Vostro sistema. >> Conservare il messaggio stesso, divulgarlo anche in parte, >> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità >> diverse, costituisce comportamento contrario ai principi dettati dal >> D.Lgs. 196/2003. >> >> The information in this message and/or attachments, is intended solely >> for the attention and use of the named addressee(s) and may be >> confidential or proprietary in nature or covered by the provisions of >> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New >> Data Protection Code).Any use not in accord with its purpose, any >> disclosure, reproduction, copying, distribution, or either >> dissemination, either whole or partial, is strictly forbidden except >> previous formal approval of the named addressee(s). If you are not the >> intended recipient, please contact immediately the sender by >> telephone, fax or e-mail and delete the information in this message >> that has been received in error. The sender does not give any warranty >> or accept liability as the content, accuracy or completeness of sent >> messages and accepts no responsibility for changes made after they >> were sent or for other risks which arise as a result of e-mail >> transmission, viruses, etc. >> >> >> On Thu, Apr 14, 2016 at 4:10 PM, Justin Deoliveira <jdeol...@gmail.com> >> wrote: >> > Hey folks. >> > >> > I just pushed a branch into my repo that contains a coverage reader for >> > solr >> > heatmaps: >> > >> > >> > >> > https://github.com/geotools/geotools/compare/master...jdeolive:solr-heatmap >> > >> > A few notes. >> > >> > First, I tried playing around with using simple interpolation for >> > smoothing. >> > While it provided something nicer looking than the raw heatmap it didn’t >> > seem to go far enough. Aside from the fact that it doesn’t seem the >> > scale >> > operations do anything when translate and scale parameters are identity? >> > Perhaps that was just user error on my part. >> > >> >> Strange, it should do the simple interpolation. It might be worth >> investingating further. >> (investigation ongoing....) >> >> Ok, had a quick look at the scale class in JAI-EXT ( I am assuming you >> are using geoserver master and have this activated) >> and apparently there is an overoptimistic optimization that bypasses >> the interpolation: >> >> https://github.com/geosolutions-it/jai-ext/blob/master/jt-scale/src/main/java/it/geosolutions/jaiext/scale/ScaleCRIF.java#L110 >> >> I'll talk to daniele to improve this. >> > Much appreciated! I have put this on his table. Let's see what he says. >> >> >> > That said I didn’t quite like adding any decisions regarding >> > visualization >> > into the coverage reader itself, so I decided to take all of that out >> > and >> > instead do it via rendering transformations. I think this gives us and >> > the >> > user the most flexibility. And also allows us to try out different >> > visualization techniques quite easily. >> > >> > In support of this idea on that branch there are two new processes: >> > >> > 1. ConvolveCoverageProcess - Exposes the Convolve operation with a few >> > of >> > it’s parameters >> >> Nice idea. >> There should ne no convolve preocess in GT, but there convolve in JAI >> and JAI-Ext >> (https://github.com/geosolutions-it/jai-ext/tree/master/jt-convolve). >> >> > 2. NormalizeCoverageProcess - “Normalizes” (please correct my >> > terminology >> > here) a raster by first calculating the max value, and then running it >> > through DivideByConst to provide a raster whose values fall between the >> > range [0,1]. >> >> This approach might interact badly with tiled requests since you would >> normalize the values using the Min/Max values in the local tile. >> Aside, I don't see how this per se would help with making the raster >> look smoother, no? Correct me if I am wrong. >> >> > Right. Tiled maps is not something I’ve really considered yet… mostly > because my experience with heatmaps is that they in general don’t work well > will tiled maps since you need to consider the data as whole (or at least > bigger than a single tile). Perhaps this is something meta tiling can > address… I just threw it in the mix as I know local approaches tend to be problematic for the viewer. I am not a Solr expert but if possible at all but one might attach to the raster the total number of documents in the index and then normalize using that or even ask solr to do it for you )not sure it can). This can still be problematic since in a large map if you go tiled you can make many requests and (even if metatiling is around) the time that passes between when you generate the first and the last tile in the map might see a different total number of doc in the index you are querying. I mean, this is something to investigate I believe. > > That said, the normalization isn’t about the smoothing/blurring effect, it’s > in regard to creating a color map to symbolize the heatmap, irrespective of > whether it’s smooth/blurred or not. More specifically my understanding is > that in order to provide a color map you need to know the value space you’re > working with? Ie the min/max values so you can create the color map entires > accordingly. And because of the dynamic nature of the heatmap grid cell > values (especially if an interpolation or blur filter is applied) the user > won’t really know these values ahead of time. Hence the need to normalize. I > hope that makes sense. Let me talk to daniele about this. I believe dynamic colormaps might help. >> >> >> > The rationale for (2) is to have a “stable” value space for the color >> > map, >> > otherwise you don’t really know what the min / max values are going to >> > be. >> > If there some existing code in the code base to handle this situation >> > that >> > doens’t involve adding any new code please let me know. >> > >> > Feedback welcome! >> >> Gave this feedback quickly, hope this helps. >> > Very much. Thank you! >> >> >> > >> > Thanks. >> > >> > -Justin >> > >> > >> > On Fri, Apr 8, 2016 at 8:31 AM Simone Giannecchini >> > <simone.giannecch...@geo-solutions.it> wrote: >> >> >> >> Let me know how it goes, >> >> we have some potential work in the pipeline for Solr and this is >> >> _very_ interesting for us. >> >> >> >> Regards, >> >> Simone Giannecchini >> >> == >> >> GeoServer Professional Services from the experts! >> >> Visit http://goo.gl/it488V for more information. >> >> == >> >> Ing. Simone Giannecchini >> >> @simogeo >> >> Founder/Director >> >> >> >> GeoSolutions S.A.S. >> >> Via di Montramito 3/A >> >> 55054 Massarosa (LU) >> >> Italy >> >> phone: +39 0584 962313 >> >> fax: +39 0584 1660272 >> >> mob: +39 333 8128928 >> >> >> >> http://www.geo-solutions.it >> >> http://twitter.com/geosolutions_it >> >> >> >> ------------------------------------------------------- >> >> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 >> >> Le informazioni contenute in questo messaggio di posta elettronica e/o >> >> nel/i file/s allegato/i sono da considerarsi strettamente riservate. >> >> Il loro utilizzo è consentito esclusivamente al destinatario del >> >> messaggio, per le finalità indicate nel messaggio stesso. Qualora >> >> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo >> >> cortesemente di darcene notizia via e-mail e di procedere alla >> >> distruzione del messaggio stesso, cancellandolo dal Vostro sistema. >> >> Conservare il messaggio stesso, divulgarlo anche in parte, >> >> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità >> >> diverse, costituisce comportamento contrario ai principi dettati dal >> >> D.Lgs. 196/2003. >> >> >> >> The information in this message and/or attachments, is intended solely >> >> for the attention and use of the named addressee(s) and may be >> >> confidential or proprietary in nature or covered by the provisions of >> >> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New >> >> Data Protection Code).Any use not in accord with its purpose, any >> >> disclosure, reproduction, copying, distribution, or either >> >> dissemination, either whole or partial, is strictly forbidden except >> >> previous formal approval of the named addressee(s). If you are not the >> >> intended recipient, please contact immediately the sender by >> >> telephone, fax or e-mail and delete the information in this message >> >> that has been received in error. The sender does not give any warranty >> >> or accept liability as the content, accuracy or completeness of sent >> >> messages and accepts no responsibility for changes made after they >> >> were sent or for other risks which arise as a result of e-mail >> >> transmission, viruses, etc. >> >> >> >> >> >> On Fri, Apr 8, 2016 at 4:25 PM, Justin Deoliveira <jdeol...@gmail.com> >> >> wrote: >> >> > Thanks a lot for the feedback guys. Simply using interpolation to do >> >> > the >> >> > smoothing makes a lot of sense! Thanks Simone. I am going to >> >> > experiment >> >> > a >> >> > bit more and I’ll come back with my findings. >> >> > >> >> > On Fri, Apr 8, 2016 at 7:09 AM Simone Giannecchini >> >> > <simone.giannecch...@geo-solutions.it> wrote: >> >> >> >> >> >> Ciao Justin, >> >> >> adding to Andrea's, please, read below... >> >> >> >> >> >> Regards, >> >> >> Simone Giannecchini >> >> >> == >> >> >> GeoServer Professional Services from the experts! >> >> >> Visit http://goo.gl/it488V for more information. >> >> >> == >> >> >> Ing. Simone Giannecchini >> >> >> @simogeo >> >> >> Founder/Director >> >> >> >> >> >> GeoSolutions S.A.S. >> >> >> Via di Montramito 3/A >> >> >> 55054 Massarosa (LU) >> >> >> Italy >> >> >> phone: +39 0584 962313 >> >> >> fax: +39 0584 1660272 >> >> >> mob: +39 333 8128928 >> >> >> >> >> >> http://www.geo-solutions.it >> >> >> http://twitter.com/geosolutions_it >> >> >> >> >> >> ------------------------------------------------------- >> >> >> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 >> >> >> Le informazioni contenute in questo messaggio di posta elettronica >> >> >> e/o >> >> >> nel/i file/s allegato/i sono da considerarsi strettamente riservate. >> >> >> Il loro utilizzo è consentito esclusivamente al destinatario del >> >> >> messaggio, per le finalità indicate nel messaggio stesso. Qualora >> >> >> riceviate questo messaggio senza esserne il destinatario, Vi >> >> >> preghiamo >> >> >> cortesemente di darcene notizia via e-mail e di procedere alla >> >> >> distruzione del messaggio stesso, cancellandolo dal Vostro sistema. >> >> >> Conservare il messaggio stesso, divulgarlo anche in parte, >> >> >> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per >> >> >> finalità >> >> >> diverse, costituisce comportamento contrario ai principi dettati dal >> >> >> D.Lgs. 196/2003. >> >> >> >> >> >> The information in this message and/or attachments, is intended >> >> >> solely >> >> >> for the attention and use of the named addressee(s) and may be >> >> >> confidential or proprietary in nature or covered by the provisions >> >> >> of >> >> >> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New >> >> >> Data Protection Code).Any use not in accord with its purpose, any >> >> >> disclosure, reproduction, copying, distribution, or either >> >> >> dissemination, either whole or partial, is strictly forbidden except >> >> >> previous formal approval of the named addressee(s). If you are not >> >> >> the >> >> >> intended recipient, please contact immediately the sender by >> >> >> telephone, fax or e-mail and delete the information in this message >> >> >> that has been received in error. The sender does not give any >> >> >> warranty >> >> >> or accept liability as the content, accuracy or completeness of sent >> >> >> messages and accepts no responsibility for changes made after they >> >> >> were sent or for other risks which arise as a result of e-mail >> >> >> transmission, viruses, etc. >> >> >> >> >> >> >> >> >> On Thu, Apr 7, 2016 at 7:56 PM, Andrea Aime >> >> >> <andrea.a...@geo-solutions.it> wrote: >> >> >> > On Wed, Apr 6, 2016 at 10:02 PM, Justin Deoliveira >> >> >> > <jdeol...@gmail.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> Hi folks, >> >> >> >> >> >> >> >> I’m working on a project to expose Solr’s heatmap capability >> >> >> >> through >> >> >> >> GeoServer. You can find details about Solr heatmaps here: >> >> >> >> >> >> >> >> >> >> >> >> https://cwiki.apache.org/confluence/display/solr/Spatial+Search >> >> >> >> (search for “Heatmap Faceting”. >> >> >> >> >> >> >> >> But the gist of it is this: If you have a spatial field that uses >> >> >> >> the >> >> >> >> recursive prefix tree type (ie. geohash) for indexing then it’s >> >> >> >> easy >> >> >> >> using >> >> >> >> Solr’s facetting infrastructure to generate a heatmap grid. What >> >> >> >> you >> >> >> >> get >> >> >> >> back from Solr is a 2D array representing the geohash grid, where >> >> >> >> each >> >> >> >> value >> >> >> >> is a count of documents that intersect that grid cell. Applying >> >> >> >> some >> >> >> >> symbolization you can get something that looks like this: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> https://raw.githubusercontent.com/voyagersearch/leaflet-solr-heatmap/master/sample.png >> >> >> >> >> >> >> >> The above screen shot comes from a leaflet plugin that visualizes >> >> >> >> the >> >> >> >> heatmap directly in the browser. I would like to add a similar >> >> >> >> looking >> >> >> >> visualization for GeoTools/GeoServer. >> >> >> >> >> >> >> >> My first thought was to expose this as a new type of coverage >> >> >> >> reader, >> >> >> >> since the data is simple grid it falls into the model quite >> >> >> >> easily. >> >> >> >> The >> >> >> >> major benefit of this approach is that becomes trivial to >> >> >> >> configure >> >> >> >> in >> >> >> >> GeoServer and easy to style using all of the existing raster >> >> >> >> symbology >> >> >> >> support. I’m interested to hear if others think this is a good >> >> >> >> approach. >> >> >> > >> >> >> > >> >> >> > I believe it is. You might want have a look at how the sde >> >> >> > coverage >> >> >> > reader >> >> >> > was setup, to share the same connection >> >> >> > info as a vector solr store (downside, you first have to setup a >> >> >> > vector >> >> >> > store). >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> If that sounds good my plan was to add this to the existing solr >> >> >> >> module. >> >> >> >> It won’t add any new dependencies aside from a dependency on the >> >> >> >> coverage >> >> >> >> module. >> >> >> >> >> >> >> >> @Andrea: you’re listed as the module maintainer… although if I >> >> >> >> recall >> >> >> >> correctly we agreed to co-maintain the module? >> >> >> > >> >> >> > >> >> >> > We did, just add yourself in the pom :-) >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> I have a prototype working so if that all sounds good I’ll push >> >> >> >> up a >> >> >> >> branch for folks to look at. >> >> >> > >> >> >> > >> >> >> > Great >> >> >> > >> >> >> >> >> >> >> >> One thing I am particular eager to get some feed back on is how >> >> >> >> to >> >> >> >> best >> >> >> >> achieve the blur affect that makes heatmaps look “ >> >> >> >> pretty”. At the moment what I have done is baked in a parameter >> >> >> >> to >> >> >> >> the >> >> >> >> coverage format that specifies a blur radius and then when >> >> >> >> reading >> >> >> >> the >> >> >> >> coverage I run it through the Convolve operation to achieve the >> >> >> >> desired >> >> >> >> affect. It would be ideal if this could be done at symbolization >> >> >> >> time. >> >> >> >> I’m >> >> >> >> wondering if we currently have any way to define a blur or some >> >> >> >> similar >> >> >> >> effect at rendering time with sld? Would a rendering transform >> >> >> >> work? >> >> >> > >> >> >> > >> >> >> > I don't think we have a blur operation, a rendering transform >> >> >> > would >> >> >> > likely >> >> >> > do the trick with the smallest effort. >> >> >> > In an ideal world, it would be nice to have a FTS level vendor >> >> >> > parameter >> >> >> > to >> >> >> > perform blurring just like we do color composition >> >> >> > but I guess that might be more complex to implement. >> >> >> >> >> >> Thinking... >> >> >> I had a quick look at the output of the SOLR heatmap and (as I >> >> >> expected) what we get from it are numbers. >> >> >> >> >> >> To make this look pretty you might want to perform smooting even by >> >> >> simply doing a bilinear or bicubic interpolation, you do not really >> >> >> need anything as fancy as a kernel operation. >> >> >> For this we could use the existing scale rendering transformation >> >> >> that >> >> >> also supports interpolation as a parameter; it should work by doing >> >> >> an >> >> >> identity transform with higher order interpolation (biliner or >> >> >> bicubic). >> >> >> >> >> >> Next step would be to apply a color map on top. >> >> >> >> >> >> > >> >> >> > Cheers >> >> >> > Andrea >> >> >> > >> >> >> > -- >> >> >> > == >> >> >> > GeoServer Professional Services from the experts! Visit >> >> >> > http://goo.gl/it488V for more information. >> >> >> > == >> >> >> > >> >> >> > Ing. Andrea Aime >> >> >> > @geowolf >> >> >> > Technical Lead >> >> >> > >> >> >> > GeoSolutions S.A.S. >> >> >> > Via di Montramito 3/A >> >> >> > 55054 Massarosa (LU) >> >> >> > phone: +39 0584 962313 >> >> >> > fax: +39 0584 1660272 >> >> >> > mob: +39 339 8844549 >> >> >> > >> >> >> > http://www.geo-solutions.it >> >> >> > http://twitter.com/geosolutions_it >> >> >> > >> >> >> > AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 >> >> >> > >> >> >> > Le informazioni contenute in questo messaggio di posta elettronica >> >> >> > e/o >> >> >> > nel/i >> >> >> > file/s allegato/i sono da considerarsi strettamente riservate. Il >> >> >> > loro >> >> >> > utilizzo è consentito esclusivamente al destinatario del >> >> >> > messaggio, >> >> >> > per >> >> >> > le >> >> >> > finalità indicate nel messaggio stesso. Qualora riceviate questo >> >> >> > messaggio >> >> >> > senza esserne il destinatario, Vi preghiamo cortesemente di >> >> >> > darcene >> >> >> > notizia >> >> >> > via e-mail e di procedere alla distruzione del messaggio stesso, >> >> >> > cancellandolo dal Vostro sistema. Conservare il messaggio stesso, >> >> >> > divulgarlo >> >> >> > anche in parte, distribuirlo ad altri soggetti, copiarlo, od >> >> >> > utilizzarlo >> >> >> > per >> >> >> > finalità diverse, costituisce comportamento contrario ai principi >> >> >> > dettati >> >> >> > dal D.Lgs. 196/2003. >> >> >> > >> >> >> > >> >> >> > >> >> >> > The information in this message and/or attachments, is intended >> >> >> > solely >> >> >> > for >> >> >> > the attention and use of the named addressee(s) and may be >> >> >> > confidential >> >> >> > or >> >> >> > proprietary in nature or covered by the provisions of privacy act >> >> >> > (Legislative Decree June, 30 2003, no.196 - Italy's New Data >> >> >> > Protection >> >> >> > Code).Any use not in accord with its purpose, any disclosure, >> >> >> > reproduction, >> >> >> > copying, distribution, or either dissemination, either whole or >> >> >> > partial, >> >> >> > is >> >> >> > strictly forbidden except previous formal approval of the named >> >> >> > addressee(s). If you are not the intended recipient, please >> >> >> > contact >> >> >> > immediately the sender by telephone, fax or e-mail and delete the >> >> >> > information in this message that has been received in error. The >> >> >> > sender >> >> >> > does >> >> >> > not give any warranty or accept liability as the content, accuracy >> >> >> > or >> >> >> > completeness of sent messages and accepts no responsibility for >> >> >> > changes >> >> >> > made after they were sent or for other risks which arise as a >> >> >> > result >> >> >> > of >> >> >> > e-mail transmission, viruses, etc. >> >> >> > >> >> >> > >> >> >> > ------------------------------------------------------- >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > ------------------------------------------------------------------------------ >> >> >> > >> >> >> > _______________________________________________ >> >> >> > GeoTools-Devel mailing list >> >> >> > GeoTools-Devel@lists.sourceforge.net >> >> >> > https://lists.sourceforge.net/lists/listinfo/geotools-devel >> >> >> > ------------------------------------------------------------------------------ Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! https://ad.doubleclick.net/ddm/clk/302982198;130105516;z _______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
