Hi Jody,
I agree that we need a recalculation option.
>From our side, we are interested on implementing the proposal for coverages
but we don't have availability to do it on the short term (meaning that we
might work on it on the last quarter of the year or the 1Q of 2017, but I
can't warrant it). Of course, we would be happy if someone else wants to
take over the task on the short term.
Note that I wrote the proposal as a first draft, but I think it should be
further detailed before implementing it (probably including the API
parameters for recalculation).
César Martínez
On 1 June 2016 at 00:54, Jody Garnett <jody.garn...@gmail.com> wrote:
> This proposal came up in today's geosever meeting (
> https://github.com/geoserver/geoserver/wiki/GSIP-140) - were you
> interested in perusing this topic?
>
> In addition to calculating during import (as proposed) we should allow
> "info" to be recalculated via the REST API. This same limitation is already
> reported (GEOS-6637 <https://osgeo-org.atlassian.net/browse/GEOS-6637>)
> for vector content - adding a refresh=stats similar to refresh=attributes
> would be appropriate.
>
>
>
>
>
>
> --
> Jody Garnett
>
> On 3 March 2016 at 07:54, César Martínez Izquierdo <cesar....@gmail.com>
> wrote:
>
>> Hi Simone,
>>
>> I have prepared a first draft for the proposal:
>> https://github.com/geoserver/geoserver/wiki/GSIP-140
>>
>> It is not too concrete regarding implementation, but I think it correctly
>> outlines our requirement and how it would fit on the existing Geoserver
>> infrastructure.
>>
>> Note that I have also sent a push request regarding the most basic
>> problem (i.e. impossibility to update the raster stats using REST API):
>> https://github.com/geoserver/geoserver/pull/1525
>>
>> Best regards,
>>
>> César
>>
>>
>> On 19 February 2016 at 11:46, Simone Giannecchini <
>> simone.giannecch...@geo-solutions.it> wrote:
>>
>>> Ciao Cesar,
>>> the existing processes do not store aything.
>>>
>>> They can be used as a block-box/building block to compute what's
>>> needed and then use separate code to store it in the config.
>>>
>>> Regards,
>>> Simone Giannecchini
>>> ==
>>> GeoServer Professional Services from the experts!
>>> Visit http://goo.gl/it488V for more information.
>>> ==
>>> Ing. Simone Giannecchini
>>> @simogeo
>>> Founder/Director
>>>
>>> GeoSolutions S.A.S.
>>> Via di Montramito 3/A
>>> 55054 Massarosa (LU)
>>> Italy
>>> phone: +39 0584 962313
>>> fax: +39 0584 1660272
>>> mob: +39 333 8128928
>>>
>>> http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>>
>>> -------------------------------------------------------
>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>>> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
>>> Il loro utilizzo è consentito esclusivamente al destinatario del
>>> messaggio, per le finalità indicate nel messaggio stesso. Qualora
>>> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
>>> cortesemente di darcene notizia via e-mail e di procedere alla
>>> distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
>>> Conservare il messaggio stesso, divulgarlo anche in parte,
>>> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
>>> diverse, costituisce comportamento contrario ai principi dettati dal
>>> D.Lgs. 196/2003.
>>>
>>> The information in this message and/or attachments, is intended solely
>>> for the attention and use of the named addressee(s) and may be
>>> confidential or proprietary in nature or covered by the provisions of
>>> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
>>> Data Protection Code).Any use not in accord with its purpose, any
>>> disclosure, reproduction, copying, distribution, or either
>>> dissemination, either whole or partial, is strictly forbidden except
>>> previous formal approval of the named addressee(s). If you are not the
>>> intended recipient, please contact immediately the sender by
>>> telephone, fax or e-mail and delete the information in this message
>>> that has been received in error. The sender does not give any warranty
>>> or accept liability as the content, accuracy or completeness of sent
>>> messages and accepts no responsibility for changes made after they
>>> were sent or for other risks which arise as a result of e-mail
>>> transmission, viruses, etc.
>>>
>>>
>>> On Fri, Feb 19, 2016 at 11:03 AM, César Martínez Izquierdo
>>> <cesar....@gmail.com> wrote:
>>> > Hi Simone, is the WPS process just calculating the stats, or is it also
>>> > storing the results on the coverage?
>>> > If it is doing both things, then it already covers our use case.
>>> >
>>> > If not, it would be worthy indeed to handle it as an asynch import,
>>> using
>>> > the WPS process to get the results and storing the results on the
>>> coverage.
>>> >
>>> > Best regards,
>>> >
>>> > César
>>> >
>>> >
>>> > On 12 February 2016 at 14:11, Simone Giannecchini
>>> > <simone.giannecch...@geo-solutions.it> wrote:
>>> >>
>>> >> On Thu, Feb 11, 2016 at 5:55 PM, César Martínez Izquierdo
>>> >> <cesar....@gmail.com> wrote:
>>> >>>
>>> >>> Dear Simone, I agree on creating a proposal to mature the exact
>>> scope and
>>> >>> the best way to implement it on Geoserver.
>>> >>> Some first ideas:
>>> >>>
>>> >>> - WHAT:
>>> >>> 1) Read raster statistics (max & mean) if available on the raster
>>> data
>>> >>> source
>>> >>> 2) Compute min and max and store it on the CoverageDimension range
>>> (it
>>> >>> is already stored but never calculated AFAIK. It can be currently
>>> introduced
>>> >>> by hand on the web interface)
>>> >>>
>>> >>>
>>> >>>
>>> >>> - API options:
>>> >>> 1) As an WPS process
>>> >>
>>> >>
>>> >> The capability to compute stats on a raster is already available as a
>>> WPS.
>>> >> We might want to improve the process by adding a configurable
>>> subsampling
>>> >> step to reduce the load.
>>> >>
>>> >>>
>>> >>> 2) As an optional parameter to REST operations on coverages (POST
>>> or
>>> >>> GET)
>>> >>
>>> >>
>>> >> As Andrea indicated, it would be great to have this as an asynch task.
>>> >> It might be worth exploring the idea of putting the importer in the
>>> mix
>>> >> and handling this as an asynch import.
>>> >>
>>> >>>
>>> >>>
>>> >>> We can continue elaborating on the wiki.
>>> >>
>>> >>
>>> >> Cool, share the page when you have it.
>>> >>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> César
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 10 February 2016 at 18:40, Simone Giannecchini
>>> >>> <simone.giannecch...@geo-solutions.it> wrote:
>>> >>>>
>>> >>>> Dear Cesar,
>>> >>>> adding to Andrea's, your proposal is certainly interesting and we
>>> would
>>> >>>> be glad to devote some our time helping out with this.
>>> >>>>
>>> >>>> I would suggest to start with putting together a proposal that we
>>> can
>>> >>>> start discussing on.
>>> >>>>
>>> >>>>
>>> >>>> Regards,
>>> >>>> Simone Giannecchini
>>> >>>> ==
>>> >>>> GeoServer Professional Services from the experts!
>>> >>>> Visit http://goo.gl/it488V for more information.
>>> >>>> ==
>>> >>>> Ing. Simone Giannecchini
>>> >>>> @simogeo
>>> >>>> Founder/Director
>>> >>>>
>>> >>>> GeoSolutions S.A.S.
>>> >>>> Via di Montramito 3/A
>>> >>>> 55054 Massarosa (LU)
>>> >>>> Italy
>>> >>>> phone: +39 0584 962313
>>> >>>> fax: +39 0584 1660272
>>> >>>> mob: +39 333 8128928
>>> >>>>
>>> >>>> http://www.geo-solutions.it
>>> >>>> http://twitter.com/geosolutions_it
>>> >>>>
>>> >>>> -------------------------------------------------------
>>> >>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>>> >>>> Le informazioni contenute in questo messaggio di posta elettronica
>>> e/o
>>> >>>> nel/i file/s allegato/i sono da considerarsi strettamente
>>> riservate. Il loro
>>> >>>> utilizzo è consentito esclusivamente al destinatario del messaggio,
>>> per le
>>> >>>> finalità indicate nel messaggio stesso. Qualora riceviate questo
>>> messaggio
>>> >>>> senza esserne il destinatario, Vi preghiamo cortesemente di darcene
>>> notizia
>>> >>>> via e-mail e di procedere alla distruzione del messaggio stesso,
>>> >>>> cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>>> divulgarlo
>>> >>>> anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>>> utilizzarlo per
>>> >>>> finalità diverse, costituisce comportamento contrario ai principi
>>> dettati
>>> >>>> dal D.Lgs. 196/2003.
>>> >>>>
>>> >>>> The information in this message and/or attachments, is intended
>>> solely
>>> >>>> for the attention and use of the named addressee(s) and may be
>>> confidential
>>> >>>> or proprietary in nature or covered by the provisions of privacy act
>>> >>>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data
>>> Protection
>>> >>>> Code).Any use not in accord with its purpose, any disclosure,
>>> reproduction,
>>> >>>> copying, distribution, or either dissemination, either whole or
>>> partial, is
>>> >>>> strictly forbidden except previous formal approval of the named
>>> >>>> addressee(s). If you are not the intended recipient, please contact
>>> >>>> immediately the sender by telephone, fax or e-mail and delete the
>>> >>>> information in this message that has been received in error. The
>>> sender does
>>> >>>> not give any warranty or accept liability as the content, accuracy
>>> or
>>> >>>> completeness of sent messages and accepts no responsibility for
>>> changes
>>> >>>> made after they were sent or for other risks which arise as a
>>> result of
>>> >>>> e-mail transmission, viruses, etc.
>>> >>>>
>>> >>>> On Wed, Feb 10, 2016 at 6:22 PM, Andrea Aime
>>> >>>> <andrea.a...@geo-solutions.it> wrote:
>>> >>>>>
>>> >>>>> On Tue, Feb 9, 2016 at 6:21 PM, César Martínez Izquierdo
>>> >>>>> <cesar....@gmail.com> wrote:
>>> >>>>>>
>>> >>>>>> We are interested on improving the behaviour of Geoserver
>>> regarding
>>> >>>>>> the management of raster statistics, which currently seems to be
>>> broken in
>>> >>>>>> several senses:
>>> >>>>>>
>>> >>>>>> a) It is not possible to update raster statistics using REST API
>>> (see
>>> >>>>>> message [1] on mailing list)
>>> >>>>>
>>> >>>>>
>>> >>>>> This one is easily fixed, given enough time to work it.
>>> >>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> b) Geoserver apparently fails to (or doesn't make any effort to)
>>> read
>>> >>>>>> raster statistics even if they are present of the raster data
>>> source (at
>>> >>>>>> least for GeoTiff)
>>> >>>>>
>>> >>>>>
>>> >>>>> Does GeoTiff have statistics built in? First time I hear about it?
>>> >>>>> Or are you talking about the GDAL statistics file (a xml file
>>> >>>>> normally), which is not part of any standard?
>>> >>>>> Indeed the code is not making any effort to read it.
>>> >>>>>
>>> >>>>> When you talk about statistics do you mean just min and max, or an
>>> >>>>> histogram?
>>> >>>>> And where would you expect to see the results of this calculation?
>>> >>>>> Right now I believe they are showing up only in WCS outputs as far
>>> as I know
>>> >>>>> (and assuming you're not talking about a histogram, which won't
>>> show up
>>> >>>>> anywhere).
>>> >>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> c) Geoserver apparently is not able to compute raster statistics
>>> from
>>> >>>>>> the data source
>>> >>>>>
>>> >>>>>
>>> >>>>> It is not indeed, but it has the machinery necessary to do it.
>>> What is
>>> >>>>> worrysome about it is that some data
>>> >>>>> sources are several terabytes of data (or more), calculating full
>>> >>>>> statistics on them might require hours or days.
>>> >>>>> Seems like a job for a WPS process, that can be called
>>> asynchronously,
>>> >>>>> at least for large datasets.
>>> >>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> We intend to correct a) in the sort term (meaning: now), but we
>>> might
>>> >>>>>> also be interested on b) and c) on the long term depending on the
>>> required
>>> >>>>>> effort.
>>> >>>>>>
>>> >>>>>> For c), I envisage a parameter on REST requests for coverages
>>> which
>>> >>>>>> forces the (re)calculation of statistics. For instance:
>>> >>>>>> POST
>>> >>>>>>
>>> /workspaces/<ws>/coveragestores/<cs>/coverages[.<format>]?stats=yes (when
>>> >>>>>> uploading the coverage)
>>> >>>>>> GET
>>> >>>>>>
>>> /workspaces/<ws>/coveragestores/<cs>/coverages/<c>[.<format>]?stats=yes (for
>>> >>>>>> an already published coverage)
>>> >>>>>>
>>> >>>>>> Do you think it could be a good approach?
>>> >>>>>
>>> >>>>>
>>> >>>>> For small datasets yes, for larger ones a synchronous request might
>>> >>>>> take too much time
>>> >>>>>
>>> >>>>> Cheers
>>> >>>>> Andrea
>>> >>>>>
>>> >>>>> --
>>> >>>>> ==
>>> >>>>> GeoServer Professional Services from the experts! Visit
>>> >>>>> http://goo.gl/it488V for more information.
>>> >>>>> ==
>>> >>>>>
>>> >>>>> Ing. Andrea Aime
>>> >>>>> @geowolf
>>> >>>>> Technical Lead
>>> >>>>>
>>> >>>>> GeoSolutions S.A.S.
>>> >>>>> Via di Montramito 3/A
>>> >>>>> 55054 Massarosa (LU)
>>> >>>>> phone: +39 0584 962313
>>> >>>>> fax: +39 0584 1660272
>>> >>>>> mob: +39 339 8844549
>>> >>>>>
>>> >>>>> http://www.geo-solutions.it
>>> >>>>> http://twitter.com/geosolutions_it
>>> >>>>>
>>> >>>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>>> >>>>>
>>> >>>>> Le informazioni contenute in questo messaggio di posta elettronica
>>> e/o
>>> >>>>> nel/i file/s allegato/i sono da considerarsi strettamente
>>> riservate. Il loro
>>> >>>>> utilizzo è consentito esclusivamente al destinatario del
>>> messaggio, per le
>>> >>>>> finalità indicate nel messaggio stesso. Qualora riceviate questo
>>> messaggio
>>> >>>>> senza esserne il destinatario, Vi preghiamo cortesemente di
>>> darcene notizia
>>> >>>>> via e-mail e di procedere alla distruzione del messaggio stesso,
>>> >>>>> cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>>> divulgarlo
>>> >>>>> anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>>> utilizzarlo per
>>> >>>>> finalità diverse, costituisce comportamento contrario ai principi
>>> dettati
>>> >>>>> dal D.Lgs. 196/2003.
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> The information in this message and/or attachments, is intended
>>> solely
>>> >>>>> for the attention and use of the named addressee(s) and may be
>>> confidential
>>> >>>>> or proprietary in nature or covered by the provisions of privacy
>>> act
>>> >>>>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data
>>> Protection
>>> >>>>> Code).Any use not in accord with its purpose, any disclosure,
>>> reproduction,
>>> >>>>> copying, distribution, or either dissemination, either whole or
>>> partial, is
>>> >>>>> strictly forbidden except previous formal approval of the named
>>> >>>>> addressee(s). If you are not the intended recipient, please contact
>>> >>>>> immediately the sender by telephone, fax or e-mail and delete the
>>> >>>>> information in this message that has been received in error. The
>>> sender does
>>> >>>>> not give any warranty or accept liability as the content, accuracy
>>> or
>>> >>>>> completeness of sent messages and accepts no responsibility for
>>> changes
>>> >>>>> made after they were sent or for other risks which arise as a
>>> result of
>>> >>>>> e-mail transmission, viruses, etc.
>>> >>>>>
>>> >>>>>
>>> >>>>> -------------------------------------------------------
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> ------------------------------------------------------------------------------
>>> >>>>> Site24x7 APM Insight: Get Deep Visibility into Application
>>> Performance
>>> >>>>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>>> >>>>> Monitor end-to-end web transactions and take corrective actions now
>>> >>>>> Troubleshoot faster and improve end-user experience. Signup Now!
>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>>> >>>>> _______________________________________________
>>> >>>>> Geoserver-devel mailing list
>>> >>>>> Geoserver-devel@lists.sourceforge.net
>>> >>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> >>> César Martínez Izquierdo
>>> >>> GIS developer
>>> >>> - - - - - - - - - - - - - - - - - - - -
>>> >>> SCOLAB: http://www.scolab.es
>>> >>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> > César Martínez Izquierdo
>>> > GIS developer
>>> > - - - - - - - - - - - - - - - - - - - -
>>> > SCOLAB: http://www.scolab.es
>>> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Site24x7 APM Insight: Get Deep Visibility into Application Performance
>>> > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>>> > Monitor end-to-end web transactions and take corrective actions now
>>> > Troubleshoot faster and improve end-user experience. Signup Now!
>>> > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>>> > _______________________________________________
>>> > Geoserver-devel mailing list
>>> > Geoserver-devel@lists.sourceforge.net
>>> > https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>>> >
>>>
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>> César Martínez Izquierdo
>> GIS developer
>> - - - - - - - - - - - - - - - - - - - -
>> SCOLAB: http://www.scolab.es
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> _______________________________________________
>> Geoserver-devel mailing list
>> Geoserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>>
>>
>
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
César Martínez Izquierdo
GIS developer
- - - - - - - - - - - - - - - - - - - -
SCOLAB: http://www.scolab.es
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
