Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Jeremy Palmer]

On Fri, Mar 20, 2020 at 6:04 PM matteo  wrote:

>
> +1000 from an user perspective the fid is really confusing. The user
> seems to be able to delete/edit the column but (of course) touching it
> will throw an error when saving.
>
> Moreover, it happens that during format conversion in the attribute
> table there are fid/pk/id columns, with same numbers.
>
>
I can imagine it is frustrating for users. It would be better not to have
the system fid at all.

We originally asked for this feature to be implemented in QGIS a few years
back - so we are to blame! Our issue was GeoPackage files were being
created from a data service with the source data primary key set as the
FID, and we couldn’t see this PK data in QGIS. This situation has now
changed, and the fid and source primary keys are both in the GPKG now.
However, the other issue with removing the fid column, is we have python
scripts using GDAL, creating GeoPackages with the layer creation options of
FID=id. This was so we can easily apply change-sets the database using the
GDAL Python API of `layer.GetFeature(fid)` rather than having to use
`layer.SetAttributeFilter(‘ID=XXX’)` (which also requires the creation of a
user index to be efficient). We also support PostGIS, MSSQL, and FileGDB
data sources in this same way. Happy if people have some thoughts on better
ways to do this :-) Maybe we can change QGIS only to display the GPKG FID
if it is not set to ‘fid”? I guess there might be other issues with that
approach in terms of editing...

Cheers
Jeremy
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[matteo]

Hi,

> I personally HATE HATE HATE these columns, and would rather I never
> saw them ever again. Does anyone else feel the same? If so, could we
> potentially just permanently hide these columns from QGIS and avoid
> all these dangerous issues for users?

+1000 from an user perspective the fid is really confusing. The user
seems to be able to delete/edit the column but (of course) touching it
will throw an error when saving.

Moreover, it happens that during format conversion in the attribute
table there are fid/pk/id columns, with same numbers.

Thanks for raining this

Cheers

Matteo
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Stephane Goldstein]

I used to frequently get errors when copying and pasting features across
geopackage files.
So now, before saving the edited file, I select all features and set the
fid to NULL, which avoid errors most of the times.

On Fri, Mar 20, 2020 at 5:07 AM Alexis R.L.  wrote:

> Greetings,
>
> Personnally when splitting layers and when there are FID conflits, I try
> to solve it manually. Sometimes it tells me that the fid of feature x
> cannot be change, which are not indistinguishable.
>
> And the next time you open the geopackage there is a change that the fid
> column will now be the next column over and the initial fid column with bad
> fids is still there. If you had critical information in your second column
> too bad, its now a fid and you lost that information forever.
>
> Maybe that issue was fixed already since I'm mostly using 3.4.4 on my main
> setup as of now (waiting for some PRs to get merged before updating).
>
> Thanks
>
> Alex
>
>
> Le jeu. 19 mars 2020 à 02:04, Nyall Dawson  a
> écrit :
>
>> Hi list,
>>
>> Just wondering what everyone's thoughts are about geopackage FID
>> columns. Personally, I find them an absolute nightmare to deal with,
>> resulting in annoying (and dangerous) issues when trying to save
>> geopackage edits, such as
>> - field type issues: converting certain formats to geopackage fails,
>> because existing fields with name "fid" are of an incompatible type
>> with geopackage. Solution: manually uncheck the "fid" field from the
>> "save as" dialog.
>> - unique constraint violations: we've mostly fixed this in processing,
>> but it's still unfortunately really common to get failures when saving
>> edits to geopackage because some operation has resulted in duplicate
>> fids. This can be a nightmare to fix, if it's even possible to do so.
>>
>> I personally HATE HATE HATE these columns, and would rather I never
>> saw them ever again. Does anyone else feel the same? If so, could we
>> potentially just permanently hide these columns from QGIS and avoid
>> all these dangerous issues for users?
>>
>> Nyall
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Alexis R.L.]

Greetings,

Personnally when splitting layers and when there are FID conflits, I try to
solve it manually. Sometimes it tells me that the fid of feature x cannot
be change, which are not indistinguishable.

And the next time you open the geopackage there is a change that the fid
column will now be the next column over and the initial fid column with bad
fids is still there. If you had critical information in your second column
too bad, its now a fid and you lost that information forever.

Maybe that issue was fixed already since I'm mostly using 3.4.4 on my main
setup as of now (waiting for some PRs to get merged before updating).

Thanks

Alex


Le jeu. 19 mars 2020 à 02:04, Nyall Dawson  a
écrit :

> Hi list,
>
> Just wondering what everyone's thoughts are about geopackage FID
> columns. Personally, I find them an absolute nightmare to deal with,
> resulting in annoying (and dangerous) issues when trying to save
> geopackage edits, such as
> - field type issues: converting certain formats to geopackage fails,
> because existing fields with name "fid" are of an incompatible type
> with geopackage. Solution: manually uncheck the "fid" field from the
> "save as" dialog.
> - unique constraint violations: we've mostly fixed this in processing,
> but it's still unfortunately really common to get failures when saving
> edits to geopackage because some operation has resulted in duplicate
> fids. This can be a nightmare to fix, if it's even possible to do so.
>
> I personally HATE HATE HATE these columns, and would rather I never
> saw them ever again. Does anyone else feel the same? If so, could we
> potentially just permanently hide these columns from QGIS and avoid
> all these dangerous issues for users?
>
> Nyall
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Tim Sutton]

I agree too - I also dont like that we get an FID column when we import another 
table that already has an id column….all sorts of chaos ensues….

https://github.com/qgis/QGIS/issues/30697

Regards

Tim

> On 19 Mar 2020, at 17:00, René-Luc Dhont  wrote:
> 
> I agree, so in my algorithms working with geopackage I always redefined  FID 
> with 
> 
> options = QgsVectorFileWriter.SaveVectorOptions()
> options.layerOptions = ['FID=id']
> 
> For me it is like a primary key in PostGIS, I need to manage it.
> Other point, Feature ID in QGIS is still a short integer.
> 
> Regards,
> René-Luc
> 
> Le 19/03/2020 à 10:28, Régis Haubourg a écrit :
>> +1000. I was so relieved of the fid/oid/mapinfo_id when QGIS arrived and 
>> allowed business logic primary keys. 
>> 
>> 
>> Le jeu. 19 mars 2020 à 07:52, Nathan Woodrow > > a écrit :
>> I agree.  
>> 
>> If there is an ID that is used internally as a unique primary key it should 
>> never be shown to the user. It should not be expected to be edited outside 
>> of the provider's control or be stable between edits. 
>> 
>> If you need a stable ID for reference you should make your own {{insert rant 
>> about people wanting to use increating ints as a reference ID}} 
>> 
>> On Thu, Mar 19, 2020 at 4:04 PM Nyall Dawson > > wrote:
>> Hi list,
>> 
>> Just wondering what everyone's thoughts are about geopackage FID
>> columns. Personally, I find them an absolute nightmare to deal with,
>> resulting in annoying (and dangerous) issues when trying to save
>> geopackage edits, such as
>> - field type issues: converting certain formats to geopackage fails,
>> because existing fields with name "fid" are of an incompatible type
>> with geopackage. Solution: manually uncheck the "fid" field from the
>> "save as" dialog.
>> - unique constraint violations: we've mostly fixed this in processing,
>> but it's still unfortunately really common to get failures when saving
>> edits to geopackage because some operation has resulted in duplicate
>> fids. This can be a nightmare to fix, if it's even possible to do so.
>> 
>> I personally HATE HATE HATE these columns, and would rather I never
>> saw them ever again. Does anyone else feel the same? If so, could we
>> potentially just permanently hide these columns from QGIS and avoid
>> all these dangerous issues for users?
>> 
>> Nyall
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org 
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org 
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org 
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

—









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com  to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link  to 
make finding time easy.

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[René-Luc Dhont]

I agree, so in my algorithms working with geopackage I always redefined  
FID with


options = QgsVectorFileWriter.SaveVectorOptions()
options.layerOptions = ['FID=id']

For me it is like a primary key in PostGIS, I need to manage it.
Other point, Feature ID in QGIS is still a short integer.

Regards,
René-Luc

Le 19/03/2020 à 10:28, Régis Haubourg a écrit :
+1000. I was so relieved of the fid/oid/mapinfo_id when QGIS arrived 
and allowed business logic primary keys.



Le jeu. 19 mars 2020 à 07:52, Nathan Woodrow > a écrit :


I agree.

If there is an ID that is used internally as a unique primary key
it should never be shown to the user. It should not be expected to
be edited outside of the provider's control or be stable between
edits.

If you need a stable ID for reference you should make your own
{{insert rant about people wanting to use increating ints as a
reference ID}}

On Thu, Mar 19, 2020 at 4:04 PM Nyall Dawson
mailto:nyall.daw...@gmail.com>> wrote:

Hi list,

Just wondering what everyone's thoughts are about geopackage FID
columns. Personally, I find them an absolute nightmare to deal
with,
resulting in annoying (and dangerous) issues when trying to save
geopackage edits, such as
- field type issues: converting certain formats to geopackage
fails,
because existing fields with name "fid" are of an incompatible
type
with geopackage. Solution: manually uncheck the "fid" field
from the
"save as" dialog.
- unique constraint violations: we've mostly fixed this in
processing,
but it's still unfortunately really common to get failures
when saving
edits to geopackage because some operation has resulted in
duplicate
fids. This can be a nightmare to fix, if it's even possible to
do so.

I personally HATE HATE HATE these columns, and would rather I
never
saw them ever again. Does anyone else feel the same? If so,
could we
potentially just permanently hide these columns from QGIS and
avoid
all these dangerous issues for users?

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org

List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:
https://lists.osgeo.org/mailman/listinfo/qgis-developer

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org 
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Régis Haubourg]

+1000. I was so relieved of the fid/oid/mapinfo_id when QGIS arrived and
allowed business logic primary keys.


Le jeu. 19 mars 2020 à 07:52, Nathan Woodrow  a écrit :

> I agree.
>
> If there is an ID that is used internally as a unique primary key it
> should never be shown to the user. It should not be expected to be edited
> outside of the provider's control or be stable between edits.
>
> If you need a stable ID for reference you should make your own {{insert
> rant about people wanting to use increating ints as a reference ID}}
>
> On Thu, Mar 19, 2020 at 4:04 PM Nyall Dawson 
> wrote:
>
>> Hi list,
>>
>> Just wondering what everyone's thoughts are about geopackage FID
>> columns. Personally, I find them an absolute nightmare to deal with,
>> resulting in annoying (and dangerous) issues when trying to save
>> geopackage edits, such as
>> - field type issues: converting certain formats to geopackage fails,
>> because existing fields with name "fid" are of an incompatible type
>> with geopackage. Solution: manually uncheck the "fid" field from the
>> "save as" dialog.
>> - unique constraint violations: we've mostly fixed this in processing,
>> but it's still unfortunately really common to get failures when saving
>> edits to geopackage because some operation has resulted in duplicate
>> fids. This can be a nightmare to fix, if it's even possible to do so.
>>
>> I personally HATE HATE HATE these columns, and would rather I never
>> saw them ever again. Does anyone else feel the same? If so, could we
>> potentially just permanently hide these columns from QGIS and avoid
>> all these dangerous issues for users?
>>
>> Nyall
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Nathan Woodrow]

I agree.

If there is an ID that is used internally as a unique primary key it should
never be shown to the user. It should not be expected to be edited outside
of the provider's control or be stable between edits.

If you need a stable ID for reference you should make your own {{insert
rant about people wanting to use increating ints as a reference ID}}

On Thu, Mar 19, 2020 at 4:04 PM Nyall Dawson  wrote:

> Hi list,
>
> Just wondering what everyone's thoughts are about geopackage FID
> columns. Personally, I find them an absolute nightmare to deal with,
> resulting in annoying (and dangerous) issues when trying to save
> geopackage edits, such as
> - field type issues: converting certain formats to geopackage fails,
> because existing fields with name "fid" are of an incompatible type
> with geopackage. Solution: manually uncheck the "fid" field from the
> "save as" dialog.
> - unique constraint violations: we've mostly fixed this in processing,
> but it's still unfortunately really common to get failures when saving
> edits to geopackage because some operation has resulted in duplicate
> fids. This can be a nightmare to fix, if it's even possible to do so.
>
> I personally HATE HATE HATE these columns, and would rather I never
> saw them ever again. Does anyone else feel the same? If so, could we
> potentially just permanently hide these columns from QGIS and avoid
> all these dangerous issues for users?
>
> Nyall
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Matthias Kuhn]

Hi Nyall,

Similar thoughts here.

Not sure what a format wins from forcing people into using an integer 
based primary key column.


I think ultimately it is the data producer's responsibility to assign 
unique and sensible identifiers with meaningful names (aka primary keys) 
to data.


All the fid_1, fid_2 etc columns after processing, all the troubles with 
shifted columns because field indexes changed, all the extra effort to 
clean up with these generated clutter ids ... So much wasted time that 
no one will ever give us back.


If it was for me, may it rest in peace.

Matthias


On 3/19/20 7:03 AM, Nyall Dawson wrote:

Hi list,

Just wondering what everyone's thoughts are about geopackage FID
columns. Personally, I find them an absolute nightmare to deal with,
resulting in annoying (and dangerous) issues when trying to save
geopackage edits, such as
- field type issues: converting certain formats to geopackage fails,
because existing fields with name "fid" are of an incompatible type
with geopackage. Solution: manually uncheck the "fid" field from the
"save as" dialog.
- unique constraint violations: we've mostly fixed this in processing,
but it's still unfortunately really common to get failures when saving
edits to geopackage because some operation has resulted in duplicate
fids. This can be a nightmare to fix, if it's even possible to do so.

I personally HATE HATE HATE these columns, and would rather I never
saw them ever again. Does anyone else feel the same? If so, could we
potentially just permanently hide these columns from QGIS and avoid
all these dangerous issues for users?

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] Geopackage FID columns: i HATE them!!!!

[Nyall Dawson]

Hi list,

Just wondering what everyone's thoughts are about geopackage FID
columns. Personally, I find them an absolute nightmare to deal with,
resulting in annoying (and dangerous) issues when trying to save
geopackage edits, such as
- field type issues: converting certain formats to geopackage fails,
because existing fields with name "fid" are of an incompatible type
with geopackage. Solution: manually uncheck the "fid" field from the
"save as" dialog.
- unique constraint violations: we've mostly fixed this in processing,
but it's still unfortunately really common to get failures when saving
edits to geopackage because some operation has resulted in duplicate
fids. This can be a nightmare to fix, if it's even possible to do so.

I personally HATE HATE HATE these columns, and would rather I never
saw them ever again. Does anyone else feel the same? If so, could we
potentially just permanently hide these columns from QGIS and avoid
all these dangerous issues for users?

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer