On Fri, Feb 16, 2024 at 3:27 PM Gabriel Roldan <
gabriel.rol...@camptocamp.com> wrote:
> Believe me I absolutely understand the convenience of just adding a
> parameter to the existing delete featuretype api. Heck, we would have saved
> all the time invested in this discussion an my customers would get what
> they want, very low hanging fruit.
>
Sorry Gabriel, I don't think you understood then... It's not at all about
convenience, but about continuity and symmetry.
The existing API has been around for a while, has been used, it has a
counterpart in coverages, and it's just lacking the
method to perform the delete along the same lines.
I see where you're coming from, but to me, that's a new REST API (v2). If
there is a separate API to handle native types,
so be it, but then the old API should be deprecated, and symmetry restored,
that is, the coverages world should also have one.
> The create feature type and for instance, all (most?) of the cataloginfo
> related endpoints in the core api, are RESTful-ish, in the sense they deal
> with REpresentational State Transfer of Catalog resources. As such, they
> are intend to be atomic and single purpose.
>
Sorry, we are again not on the same page. Atomicity is a very useful
property, but one that the existing REST API does not
offer in general, and when it does, it is by accident. The reason is
simple: the service and catalog models lack the concept of
a transaction that can be started at the beginning of a request, and
committed atomically at the end, or reverted fully if anything wrong
happens.
The current REST API has no such promise, because we lack the underlying
machinery to satisfy the promise.
Now, I guess that Cloud GeoServer with the PostgreSQL configuration backend
might wrap operations in a transaction, which is fine,
but not something we can claim for the API in general (not against the
random GeoServer).
Generally speaking, I care about what GeoServer can do, if we can help
downstreams projects it's even better, but not at the
expense of having an inconsistent situation when one can "enter the house
through the door, but be forced to leave through a window"
(load data through the existing feature resources, but remove it through
the native type resource).
I'd be more inclined to have delete as a parameter, and then in addition, a
native resource API as well (which has the merit
of describing types not currently published). A user can then decide which
path they prefer: multiple alternatives are
nothing new, for large data uploads using GeoServer only, the importer API
is the preferred alternative anyways, due to its async operation support.
The concept of "native resource" would likely have to be explained in
detail, we already have a layer and feature type, the native bit
would introduce
a third level (which is natural to those familiar with the internals, but
not so sure about random Joe).
> For the specific case of the create featuretype api, creating the schema
> on the target store is a desired byproduct, as it's currently designed, and
> there's no argument against its convenience.
>
That's your interpretation of it, the API evolved out of practical
considerations, not sure how much up front design was put into it.
>
> One can say the operation's single purpose is to create a FeatureTypeInfo,
> and as a convenience it'll create a LayerInfo and the underlying native
> schema.
> The main point being it's atomic, when it returns, the featuretype is
> either created or it isn't. Worse case scenario, something failed and the
> system left a dangling schema in the backing store, but no harm's done to
> the system.
>
If one uploads in replace mode, a lot of damage can be done... the old data
is removed before adding the new one.
All one needs is to upload a file that can be self described properly, but
cannot be properly read (corrupted shapefile for example).
- because it can't be atomic. At the end of the operation, both the
> published and native FTs shall be removed, or none. The Catalog doesn't
> handle transactions, and also not all DataStores do. Much less distributed
> ones. Hence, if an error occurs at any stage, you end up with an
> inconsistent state: the database table was removed but you couldn't remove
> the pusblished type, or the other way around. In the former case, you can't
> recover the data. In the later, the state is intermediate, and the
> operation didn't achieve its goal.
>
Skipped over and came back here. All reasoning based on atomicity
guarantees is faulty, it cannot be guaranteed in general unless we change
the Catalog and Geoserver interfaces.
> I've nothing against operation parameters as long as they're input values
> to the operation. What I'm against is flags that change its semantics.
>
POST creates (configuration and data) and DELETE destroys (configuration
and data). It's symmetric.
If you want to remove the ability to delete data, you need to get rid of
the ability to create it as well.
> Finally, in your current-api-breakdown reply, you mention the only
> existing related endpoint is to delete also the data for a coverage:
>
> > DELETE
> /workspaces/{workspaceName}/coveragestores/{storeName}?deleteType=none/all/metadata.
> With "metadata" the configuration files are all removed (e.g., the mosaic
> configuration files), with "all" the actual raster data is removed as well
>
> Well, this is the perfect example of what I'm proposing. Note how it's an
> operation on the CoverageStore and not on the Coverage.
> I concede that's probably related to the fact that CoverageStores support
> a single Coverage?
>
No, it's meant to go and destroy all coverages and associated data
contained in the mosaic
<https://github.com/geoserver/geoserver/blob/0ab715486a958632e9e7d54ad5746a4858189f3b/src/restconfig/src/main/java/org/geoserver/rest/catalog/CoverageStoreController.java#L198>,
if called that way.
If the same is called on the single coverage, it will delete only the data
files associated to that coverage, and that coverage alone.
E.g., if a NetCDF file happens to offer content for multiple coverages, it
will be removed only when the last coverage referencing it
it is removed as well.
These APIs are meant to be used by data managers that pay attention to what
they do ("admin" in GeoServer is tha same
as "root" on Linux, all powerful).
A separate resource makes a difference only in your mind, in my experience
I'm just as easily going to make a mistake in the path
or in the query string (see example in my previous mail).
> I think that exposes my point of view in a more civil way, and yes, we'll
> probably have to go through the GSIP process.
>
Thank you for the explanation, unfortunately it did not change my mind.
Further discussion is pointless, let's get the PSC members to vote, and
take responsibility for the result, whatever it is.
I'm happy to let the proposal pass anyways, if the rest of the PSC is happy
with the direction you're suggesting, but I want my disagreement to be on
record.
Cheers
Andrea
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services-us for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions Group
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
https://www.geosolutionsgroup.com/
http://twitter.com/geosolutions_it
-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see this:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
