On Wed, May 13, 2015 at 8:58 PM, Torben Barsballe <
tbarsba...@boundlessgeo.com> wrote:
> Hi Andrea
>
> I have implemented something very similar for importer, when working on an
> importer interface that has an upload functionality. Currently, Importer
> does provide a way to access all the files associated with a Task (ie.
> layer) using the ImportData object, specifically the SpatialFile subclass:
>
> File SpatialFile.getFile()
> FileSpatialFile.getPrjFile()
> List<File> SpatialFile.getSuppFiles()
>
> The way I implemented this was to do an import against all of the initial
> files, then move any files that got imported to the final destination (by
> looking at the ImportData of the completed tasks), and finally update the
> URL of the store to that destination.
>
I have seen those API, but it has downsides.
First, it's based on the assumption that any file with the same base name
as the main file is a supplementary
file, prj gets special treatment, aux are ignored. In general this does not
work, see my previous mail about how
complicated a mosaic can be.
Second, it is working only against data that you're ingesting, but I need
it to work also against target data,
to support indirect imports of single tiffs (for starters).
The use case is as follows, we have an already configured layer, a TIFF,
and we want to replace it with another
one, fresher. This would be a indirect import with UpdateMode.REPLACE.
In vector land this is done by wiping out the contents of the target store
and then write out the new, but for
raster data I cannot have that, as trying to use the writer would make me
lose the work of gdaladdo/gdaltranslate.
I guess one could make them post-transforms, but it would be bad too, since
any issue running them
will cause the original layer to be lost, and the new one to be broken.
This is not acceptable of course, if
the import fails anywhere, the original layer must be preserved (like we do
for an indirect import of
vector data, where everything runs in a transaction).
So the idea is that for rasters, and when the format of the source is the
same as the target one, we would use
these new interfaces we're talking about to figure out where the already
configured data is (where the existing
layer is), then disable the layer, move its files aside, move the newly
imported ones in place of the old ones,
enable back the layer, and finally dispose of the original ones (which we
kept just in case the move
of the newly imported ones failed).
In case instead the formats of input and target are not matching, for the
moment I'll have the importer
throw an exception (all indirect imports were doing that anyways).
>
> If you are applying the transformations to the files before determining
> whether or not to keep them, this feels like a case where the ImportData
> may be a better indicator of what files you want to be working with, since
> it only deals with a single Spatial file plus any supplementary fles,
> whereas some File-based stores may include several sets of shapefiles (This
> may not be an issue with raster stores, but I know shapefile stores at
> least can include any number of spatial files).
>
A directory store can, not the normal shapefile store. But as said, for
indirect imports in vector land we do not do any file moving anyways.
>
> Another thing that I ran into is the case where not all the shapefiles
> included in the initial store generated by the importer may get moved to
> the final store - they may get cancelled if the transformation failed, or
> if the importer threw an error, or they might be excluded from the import
> by the user. If we are applying the transform after the import completes
> then this is fine, we would only get the files that were actually imported
> into the store. However, if we are applying the transform before the import
> (At (1), see below), then the store will have all the files, even ones we
> may not be importing.
>
You say we are moving files, but I don't see where this is happening,
indeed when I'm uploading geotiffs I have the impression
they end up staying in a $DATA_DIR/uploads/upload<randomNumber> folder,
which is ugly, and does not match
the REST api behavior, where we have the pluggable PathMapper deciding
where uploaded files should reside (a difference
that we should amend, but I'm not sure I have enough resources to work it
in the current project)
Ah, also, I have a second need to determine the files that make up a
certain featuresource/raster data set: direct download.
We keep on being requested to allow also direct download of the original
data from GeoServer, the interfaces we
are talking about in this thread will allow to assemble a zip file for
direct download.
The link for direct download would likely appear inside the CSW metadata of
the layer in GeoServer.
Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
-------------------------------------------------------
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
