Hi,
Providing some additional feedbacks on my side, too.
On Mon, Jun 6, 2016 at 6:34 PM, Andrea Aime <andrea.a...@geo-solutions.it>
wrote:
> On Fri, Jun 3, 2016 at 1:19 AM, Devon Tucker <devonrtuc...@gmail.com>
> wrote:
>
>> Here are a few things that might help understanding, section by section:
>>>
>>> - *Description*: this is fine, makes sense, no questions (one thing,
>>> the recursive file loading during indexing is already the default
>>> behavior
>>> as far as I know)
>>> - *Index Generation: *quite a bit lost here, how are
>>> MosaicIndexConfiguration and GranuleCatalogManager going to be used by
>>> the
>>> existing classes? Or are they replacing it? I'd need to understand how
>>> the
>>> existing code is going to get reshuffled into a generic machiner plus
>>> default implementations of the two above objects? Also, can you provide
>>> an
>>> example of an alternate implementation (being the refactor targeted to
>>> extensibility)
>>> - *Harvesting: *same as above, how is the code going to be moved
>>> around, and examples of alternate implementations
>>>
>>>
>> I hope to clarify these parts soon. To be honest, these parts were done
>> earlier in the project and we've zig-zagged multiple times since then, so
>> even I need to revisit to solidify and clarify this section. To give some
>> perspective on the original motivations behind this part, at the time we
>> were:
>>
>> - Performing some pre-processing on rasters before indexing them (adding
>> overlays, creating alpha bands, generating elevation outlier footprints)
>>
>
> Image mosaic looks like the wrong place to do this, the importer would be
> much better (and already does some of those). See below for more
> observations regarding this.
>
I would agree with Andrea's feedbacks on the pre-processing topic.
>
>
>> - Collecting custom properties from rasters during indexing (resolution,
>> date stamps, various tiff metadata, sample date, etc.)
>>
> Where this information will be stored? In the mentioned
GranuleDescriptor's properties map? Shouldn't that achieved by
PropertyCollector?
>
> This is done by property collectors, why not just extend that?
>
>
>> - Looking at harvesting granules on demand either via. rest or the
>> geoserver UI.
>>
>
> The REST api already does that. There is no UI for it though.
>
Some more information about REST support for harvesting is available here
with some examples:
http://docs.geoserver.org/stable/en/user/rest/examples/curl.html#uploading-and-modifying-a-image-mosaic
>
>> - Exposing more of this configuration through a custom store and layer
>> config in GeoServer.
>>
>
> This can be arranged too, not sure it needs a refactor for that.
>
>
>> - Index generation code is very isolated as it stands. It's tough to
>> programmatically configure index generation.
>>
>
> Ok, this one makes sense... the others do not seem to actually require a
> refactor though.
>
>
>>
>>
>>>
>>> - *Delegate coverage acceptance/rejection to a predicate object*:
>>> makes sense I guess, so the reason to have this plugable is because you
>>> might roll a collector that has less limitations than the default ones?
>>>
>>> Exactly.
>>
>
> Ok but... why not keep the improved collectors in the mosaic instead? They
> would benefit every user there (instead of just the gt-dem users)
>
>
>>
>>> - *Pre-process Granule Footprint Before Indexing*: makes sense,
>>> thinking out loud is the footprint the only thing that needs
>>> reprocessing?
>>>
>>> No, it's not really the only thing that need reprocessing. In fact
>> PropertyCollectors have a tangentially related responsibility (and are used
>> in roughly the same spot). Maybe this functionality could be combined
>> somehow, and make monolithic some sort of monolithic visitor which would be
>> responsible for:
>>
>> - Collecting properties from granules
>> - Pre-processing the granule footprint if needed
>> - Populating the index feature
>>
>> And so on. I'll have to think about this one a bit more, but as the
>> proposal stands at least FootprintProcessor has a very clear responsibility.
>>
>
>
> See above about the usage of the importer. The indexing happens during
> layer configuration in the UI, expensive operations are best done during
> pre-processing.
> Normally what you are describing here is done as part of a "ingest"
> machinery, which is a off-line ETL of sorts that prepares the data for
> publication, but
> before it actually reaches GeoServer. The importer is one example of
> those, I've seen several others, all of them were asynch and offline.
>
I agree with this comment too.
>
>
>>
>>> - *Generalize Mosaicking per GranuleCollector and Update
>>> GranuleCollector to a tree-like hierarchy*: an example would be
>>> useful
>>>
>>> I can do an example for this. The motivation here is to delegate the
>> actual mosaicking to an object which may internally be delegating to other
>> mosaicking objects. For example, the default implementation would have the
>> logic for mosaicking per-resolution first, then resampling and mosaicking
>> those results.
>>
>
> +1
>
>
>>
>>> - *Enhance the GranuleDescriptor and GranuleCatalogVisitor
>>> interfaces: *same as above, examples of these "arbitrary properties"
>>> to be used would be useful
>>>
>>> I will provide examples for this as well. In our RnD it was necessary to
>> set the CRS on each granule descriptor.
>>
>
> Ok... but wouldn't it be better to keep the CRS information as an
> attribute in the index instead?
>
> Will these additional properties somehow stored/serialized or are them in
memory information computed at GranuleDescriptor initialization?
(Wondering if they should be part of the index too)
Cheers,
Daniele
> Cheers
> Andrea
>
> --
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054 Massarosa (LU)
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39 339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
> -------------------------------------------------------
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> GeoTools-Devel mailing list
> GeoTools-Devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
>
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==
Ing. Daniele Romagnoli
Senior Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
