Hi Simone,
I have prepared a first draft for the proposal:
https://github.com/geoserver/geoserver/wiki/GSIP-140
It is not too concrete regarding implementation, but I think it correctly
outlines our requirement and how it would fit on the existing Geoserver
infrastructure.
Note that I have also sent a push request regarding the most basic problem
(i.e. impossibility to update the raster stats using REST API):
https://github.com/geoserver/geoserver/pull/1525
Best regards,
César
On 19 February 2016 at 11:46, Simone Giannecchini <
simone.giannecch...@geo-solutions.it> wrote:
> Ciao Cesar,
> the existing processes do not store aything.
>
> They can be used as a block-box/building block to compute what's
> needed and then use separate code to store it in the config.
>
> Regards,
> Simone Giannecchini
> ==
> GeoServer Professional Services from the experts!
> Visit http://goo.gl/it488V for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054 Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39 333 8128928
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
> Il loro utilizzo è consentito esclusivamente al destinatario del
> messaggio, per le finalità indicate nel messaggio stesso. Qualora
> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
> cortesemente di darcene notizia via e-mail e di procedere alla
> distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
> Conservare il messaggio stesso, divulgarlo anche in parte,
> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
> diverse, costituisce comportamento contrario ai principi dettati dal
> D.Lgs. 196/2003.
>
> The information in this message and/or attachments, is intended solely
> for the attention and use of the named addressee(s) and may be
> confidential or proprietary in nature or covered by the provisions of
> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
> Data Protection Code).Any use not in accord with its purpose, any
> disclosure, reproduction, copying, distribution, or either
> dissemination, either whole or partial, is strictly forbidden except
> previous formal approval of the named addressee(s). If you are not the
> intended recipient, please contact immediately the sender by
> telephone, fax or e-mail and delete the information in this message
> that has been received in error. The sender does not give any warranty
> or accept liability as the content, accuracy or completeness of sent
> messages and accepts no responsibility for changes made after they
> were sent or for other risks which arise as a result of e-mail
> transmission, viruses, etc.
>
>
> On Fri, Feb 19, 2016 at 11:03 AM, César Martínez Izquierdo
> <cesar....@gmail.com> wrote:
> > Hi Simone, is the WPS process just calculating the stats, or is it also
> > storing the results on the coverage?
> > If it is doing both things, then it already covers our use case.
> >
> > If not, it would be worthy indeed to handle it as an asynch import, using
> > the WPS process to get the results and storing the results on the
> coverage.
> >
> > Best regards,
> >
> > César
> >
> >
> > On 12 February 2016 at 14:11, Simone Giannecchini
> > <simone.giannecch...@geo-solutions.it> wrote:
> >>
> >> On Thu, Feb 11, 2016 at 5:55 PM, César Martínez Izquierdo
> >> <cesar....@gmail.com> wrote:
> >>>
> >>> Dear Simone, I agree on creating a proposal to mature the exact scope
> and
> >>> the best way to implement it on Geoserver.
> >>> Some first ideas:
> >>>
> >>> - WHAT:
> >>> 1) Read raster statistics (max & mean) if available on the raster
> data
> >>> source
> >>> 2) Compute min and max and store it on the CoverageDimension range
> (it
> >>> is already stored but never calculated AFAIK. It can be currently
> introduced
> >>> by hand on the web interface)
> >>>
> >>>
> >>>
> >>> - API options:
> >>> 1) As an WPS process
> >>
> >>
> >> The capability to compute stats on a raster is already available as a
> WPS.
> >> We might want to improve the process by adding a configurable
> subsampling
> >> step to reduce the load.
> >>
> >>>
> >>> 2) As an optional parameter to REST operations on coverages (POST or
> >>> GET)
> >>
> >>
> >> As Andrea indicated, it would be great to have this as an asynch task.
> >> It might be worth exploring the idea of putting the importer in the mix
> >> and handling this as an asynch import.
> >>
> >>>
> >>>
> >>> We can continue elaborating on the wiki.
> >>
> >>
> >> Cool, share the page when you have it.
> >>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> César
> >>>
> >>>
> >>>
> >>> On 10 February 2016 at 18:40, Simone Giannecchini
> >>> <simone.giannecch...@geo-solutions.it> wrote:
> >>>>
> >>>> Dear Cesar,
> >>>> adding to Andrea's, your proposal is certainly interesting and we
> would
> >>>> be glad to devote some our time helping out with this.
> >>>>
> >>>> I would suggest to start with putting together a proposal that we can
> >>>> start discussing on.
> >>>>
> >>>>
> >>>> Regards,
> >>>> Simone Giannecchini
> >>>> ==
> >>>> GeoServer Professional Services from the experts!
> >>>> Visit http://goo.gl/it488V for more information.
> >>>> ==
> >>>> Ing. Simone Giannecchini
> >>>> @simogeo
> >>>> Founder/Director
> >>>>
> >>>> GeoSolutions S.A.S.
> >>>> Via di Montramito 3/A
> >>>> 55054 Massarosa (LU)
> >>>> Italy
> >>>> phone: +39 0584 962313
> >>>> fax: +39 0584 1660272
> >>>> mob: +39 333 8128928
> >>>>
> >>>> http://www.geo-solutions.it
> >>>> http://twitter.com/geosolutions_it
> >>>>
> >>>> -------------------------------------------------------
> >>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
> >>>> Le informazioni contenute in questo messaggio di posta elettronica e/o
> >>>> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
> Il loro
> >>>> utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le
> >>>> finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio
> >>>> senza esserne il destinatario, Vi preghiamo cortesemente di darcene
> notizia
> >>>> via e-mail e di procedere alla distruzione del messaggio stesso,
> >>>> cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo
> >>>> anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per
> >>>> finalità diverse, costituisce comportamento contrario ai principi
> dettati
> >>>> dal D.Lgs. 196/2003.
> >>>>
> >>>> The information in this message and/or attachments, is intended solely
> >>>> for the attention and use of the named addressee(s) and may be
> confidential
> >>>> or proprietary in nature or covered by the provisions of privacy act
> >>>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data
> Protection
> >>>> Code).Any use not in accord with its purpose, any disclosure,
> reproduction,
> >>>> copying, distribution, or either dissemination, either whole or
> partial, is
> >>>> strictly forbidden except previous formal approval of the named
> >>>> addressee(s). If you are not the intended recipient, please contact
> >>>> immediately the sender by telephone, fax or e-mail and delete the
> >>>> information in this message that has been received in error. The
> sender does
> >>>> not give any warranty or accept liability as the content, accuracy or
> >>>> completeness of sent messages and accepts no responsibility for
> changes
> >>>> made after they were sent or for other risks which arise as a result
> of
> >>>> e-mail transmission, viruses, etc.
> >>>>
> >>>> On Wed, Feb 10, 2016 at 6:22 PM, Andrea Aime
> >>>> <andrea.a...@geo-solutions.it> wrote:
> >>>>>
> >>>>> On Tue, Feb 9, 2016 at 6:21 PM, César Martínez Izquierdo
> >>>>> <cesar....@gmail.com> wrote:
> >>>>>>
> >>>>>> We are interested on improving the behaviour of Geoserver regarding
> >>>>>> the management of raster statistics, which currently seems to be
> broken in
> >>>>>> several senses:
> >>>>>>
> >>>>>> a) It is not possible to update raster statistics using REST API
> (see
> >>>>>> message [1] on mailing list)
> >>>>>
> >>>>>
> >>>>> This one is easily fixed, given enough time to work it.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> b) Geoserver apparently fails to (or doesn't make any effort to)
> read
> >>>>>> raster statistics even if they are present of the raster data
> source (at
> >>>>>> least for GeoTiff)
> >>>>>
> >>>>>
> >>>>> Does GeoTiff have statistics built in? First time I hear about it?
> >>>>> Or are you talking about the GDAL statistics file (a xml file
> >>>>> normally), which is not part of any standard?
> >>>>> Indeed the code is not making any effort to read it.
> >>>>>
> >>>>> When you talk about statistics do you mean just min and max, or an
> >>>>> histogram?
> >>>>> And where would you expect to see the results of this calculation?
> >>>>> Right now I believe they are showing up only in WCS outputs as far
> as I know
> >>>>> (and assuming you're not talking about a histogram, which won't show
> up
> >>>>> anywhere).
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> c) Geoserver apparently is not able to compute raster statistics
> from
> >>>>>> the data source
> >>>>>
> >>>>>
> >>>>> It is not indeed, but it has the machinery necessary to do it. What
> is
> >>>>> worrysome about it is that some data
> >>>>> sources are several terabytes of data (or more), calculating full
> >>>>> statistics on them might require hours or days.
> >>>>> Seems like a job for a WPS process, that can be called
> asynchronously,
> >>>>> at least for large datasets.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> We intend to correct a) in the sort term (meaning: now), but we
> might
> >>>>>> also be interested on b) and c) on the long term depending on the
> required
> >>>>>> effort.
> >>>>>>
> >>>>>> For c), I envisage a parameter on REST requests for coverages which
> >>>>>> forces the (re)calculation of statistics. For instance:
> >>>>>> POST
> >>>>>> /workspaces/<ws>/coveragestores/<cs>/coverages[.<format>]?stats=yes
> (when
> >>>>>> uploading the coverage)
> >>>>>> GET
> >>>>>>
> /workspaces/<ws>/coveragestores/<cs>/coverages/<c>[.<format>]?stats=yes (for
> >>>>>> an already published coverage)
> >>>>>>
> >>>>>> Do you think it could be a good approach?
> >>>>>
> >>>>>
> >>>>> For small datasets yes, for larger ones a synchronous request might
> >>>>> take too much time
> >>>>>
> >>>>> Cheers
> >>>>> Andrea
> >>>>>
> >>>>> --
> >>>>> ==
> >>>>> GeoServer Professional Services from the experts! Visit
> >>>>> http://goo.gl/it488V for more information.
> >>>>> ==
> >>>>>
> >>>>> Ing. Andrea Aime
> >>>>> @geowolf
> >>>>> Technical Lead
> >>>>>
> >>>>> GeoSolutions S.A.S.
> >>>>> Via di Montramito 3/A
> >>>>> 55054 Massarosa (LU)
> >>>>> phone: +39 0584 962313
> >>>>> fax: +39 0584 1660272
> >>>>> mob: +39 339 8844549
> >>>>>
> >>>>> http://www.geo-solutions.it
> >>>>> http://twitter.com/geosolutions_it
> >>>>>
> >>>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
> >>>>>
> >>>>> Le informazioni contenute in questo messaggio di posta elettronica
> e/o
> >>>>> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
> Il loro
> >>>>> utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le
> >>>>> finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio
> >>>>> senza esserne il destinatario, Vi preghiamo cortesemente di darcene
> notizia
> >>>>> via e-mail e di procedere alla distruzione del messaggio stesso,
> >>>>> cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo
> >>>>> anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per
> >>>>> finalità diverse, costituisce comportamento contrario ai principi
> dettati
> >>>>> dal D.Lgs. 196/2003.
> >>>>>
> >>>>>
> >>>>>
> >>>>> The information in this message and/or attachments, is intended
> solely
> >>>>> for the attention and use of the named addressee(s) and may be
> confidential
> >>>>> or proprietary in nature or covered by the provisions of privacy act
> >>>>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data
> Protection
> >>>>> Code).Any use not in accord with its purpose, any disclosure,
> reproduction,
> >>>>> copying, distribution, or either dissemination, either whole or
> partial, is
> >>>>> strictly forbidden except previous formal approval of the named
> >>>>> addressee(s). If you are not the intended recipient, please contact
> >>>>> immediately the sender by telephone, fax or e-mail and delete the
> >>>>> information in this message that has been received in error. The
> sender does
> >>>>> not give any warranty or accept liability as the content, accuracy or
> >>>>> completeness of sent messages and accepts no responsibility for
> changes
> >>>>> made after they were sent or for other risks which arise as a result
> of
> >>>>> e-mail transmission, viruses, etc.
> >>>>>
> >>>>>
> >>>>> -------------------------------------------------------
> >>>>>
> >>>>>
> >>>>>
> ------------------------------------------------------------------------------
> >>>>> Site24x7 APM Insight: Get Deep Visibility into Application
> Performance
> >>>>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> >>>>> Monitor end-to-end web transactions and take corrective actions now
> >>>>> Troubleshoot faster and improve end-user experience. Signup Now!
> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> >>>>> _______________________________________________
> >>>>> Geoserver-devel mailing list
> >>>>> Geoserver-devel@lists.sourceforge.net
> >>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>> César Martínez Izquierdo
> >>> GIS developer
> >>> - - - - - - - - - - - - - - - - - - - -
> >>> SCOLAB: http://www.scolab.es
> >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>
> >>
> >
> >
> >
> > --
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > César Martínez Izquierdo
> > GIS developer
> > - - - - - - - - - - - - - - - - - - - -
> > SCOLAB: http://www.scolab.es
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >
> >
> ------------------------------------------------------------------------------
> > Site24x7 APM Insight: Get Deep Visibility into Application Performance
> > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > Monitor end-to-end web transactions and take corrective actions now
> > Troubleshoot faster and improve end-user experience. Signup Now!
> > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> > _______________________________________________
> > Geoserver-devel mailing list
> > Geoserver-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/geoserver-devel
> >
>
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
César Martínez Izquierdo
GIS developer
- - - - - - - - - - - - - - - - - - - -
SCOLAB: http://www.scolab.es
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
