Re: [gdal-dev] ogr2ogr no longer works with non super user in PG after update

Thanks even for the feedback. I have just spotted the nomd option but I didn't get the time to test it. Regards Alexandre Le mar. 11 juin 2024 à 17:43, Even Rouault a écrit : > Hi, > > The issue has been reported in https://github.com/OSGeo/gdal/issues/9994 > and should be be fixed in 3.9.1

Re: [gdal-dev] ogr2ogr no longer works with non super user in PG after update

Hi, The issue has been reported in https://github.com/OSGeo/gdal/issues/9994 and should be be fixed in 3.9.1 per https://github.com/OSGeo/gdal/pull/10004 (testing of the 3.9 branch at https://github.com/OSGeo/gdal/tree/release/3.9 would be appreciated to confirm that it fixes error for

[gdal-dev] ogr2ogr no longer works with non super user in PG after update

Hello there, After updating our stack from GDAL 3.4.1 to GDAL 3.9.0, we have ogr2ogr calls which no longer succeed with the following message: ERROR: permission denied to create event trigger "ogr_system_tables_event_trigger_for_metadata". HINT: Must be superuser to create an event trigger.

Re: [gdal-dev] ogr2ogr, GMLAS driver, help

Hi, It is very difficult to create manually the table & field structure that will be generated by the GMLAS driver. It is likely that the attribute name generated by the GMLAs driver is not the one you expect. First try ogr2ogr to a not-yet-existing PotsgreSQL table to see if data is

[gdal-dev] ogr2ogr, GMLAS driver, help

Hello, i am importing an xml with ogr2ogr using GMLAS driver to a PostgreSQL server. I am also using an .xsd file to correctly import and manage the imported xml in the server. This way i was able to import every element into the correct column based on the xsd, except one, a type="xsd:time"

Re: [gdal-dev] ogr2ogr using PDF driver reports: ERROR 1: Max token stack size reached

Fix in https://github.com/OSGeo/gdal/pull/9213 for the "max token stack size reached" error. However opening the resulting converted file doesn't show something that can be immediately correlated with the content of the PDF. It is likely that some geometry transformations are missed by the

Re: [gdal-dev] ogr2ogr using PDF driver reports: ERROR 1: Max token stack size reached

Arjan, I'm trying to convert a PDF with vector data into a shape with: ogr2ogr test.shp Archeologische\ beleidskaart\ 2023.pdf  --config OGR_PDF_READ_NON_STRUCTURED YES However, It gives the error: ERROR 1: Max token stack size reached The sources are only converted partly, possibly

[gdal-dev] ogr2ogr using PDF driver reports: ERROR 1: Max token stack size reached

I'm trying to convert a PDF with vector data into a shape with: ogr2ogr test.shp Archeologische\ beleidskaart\ 2023.pdf  --config OGR_PDF_READ_NON_STRUCTURED YES However, It gives the error: ERROR 1: Max token stack size reached The sources are only converted partly, possibly because of this

Re: [gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue

taja: gdal-dev@lists.osgeo.org; Scott Aihe: Re: [gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue Thanks, The primary key on the source tables is on required columns - it is a natural key based on core data so cannot be excluded to have a meaningful dataset exported. Also

Re: [gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue

Change target.gpkg to target.sqlite ...and in the -sql use something like "select cast(primary_key as integer) as key, [all your other columns] from table" Thanks, The primary key on the source tables is on required columns - it is a natural key based on core data so cannot be excluded to

Re: [gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue

Thanks, The primary key on the source tables is on required columns - it is a natural key based on core data so cannot be excluded to have a meaningful dataset exported. Also, invoking it your way provides a data only export - just the output of a select statement. I'm trying to create

Re: [gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue

You could build your column list without that specific column and pass it to ogr2ogr (bash): ogr2ogr -sql "select $(psql -d mydb --pset='footer=off' -F',' -qAc "select * from table where false" |sed 's/columnToOmit,//g') from table" target.gpkg PG:dbname=mydb On 8/15/23 17:12, Brent Wood

[gdal-dev] ogr2ogr postgis to sqlite composite primary key issueissue

Hi, I'm looking to use ogr2ogr to replicate a Postgis schema into a Spatialite database. (Make a snapshot copy of institutional tables on a field laptop with possible no internet access) I have a script to identify the relevant tables and invoke ogr2ogr to copy each table. Some of the source

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

Thanks Even! On Mon, Jul 3, 2023, 3:11 AM Even Rouault wrote: > Hi Patrick, > > blocking error fixed per https://github.com/OSGeo/gdal/pull/8040 > > and the one related to esriFieldTypeDate in > https://github.com/OSGeo/gdal/pull/8039 > > Even > > Le 28/06/2023 à 17:18, Patrick Young a écrit :

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

Hi Patrick, blocking error fixed per https://github.com/OSGeo/gdal/pull/8040 and the one related to esriFieldTypeDate in https://github.com/OSGeo/gdal/pull/8039 Even Le 28/06/2023 à 17:18, Patrick Young a écrit : Hi, I'm playing around with the gdb dataset

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

There may be a way to address the underlying issue, I don't know what it is. In lieu of that, here's a work around in 2 steps in 3.7. Create a gpkg: ogr2ogr -sql "select * from roadbed where source_id = 13350003311" tmp.gpkg /vsizip/NYC_DoITT_Planimetric_OpenData.gdb.zip Then do it with

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

Here's the CREATE TABLE from 3.4: OGR2SQLITE: sqlite3_declare_vtab(CREATE TABLE "roadbed"("SOURCE_ID" FLOAT,"FEATURE_CODE" INTEGER_INT16,"SUB_FEATURE_CODE" INTEGER,"STATUS" VARCHAR(16),"SHAPE_Length" FLOAT,"SHAPE_Area" FLOAT,OGR_STYLE VARCHAR HIDDEN,"SHAPE" BLOB_MULTIPOLYGON_XY_2263,

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

The table exists, the SQL emitted to create the virtual table is the issue from the looks of it. The command works as is in gdal 3.4 FWIW On Wed, Jun 28, 2023, 9:29 AM Scott wrote: > The last line says: > no such table: roadbed > > See if it exists in the gdb: > > ogrinfo dataset.gdb > >

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

The last line says: no such table: roadbed See if it exists in the gdb: ogrinfo dataset.gdb On 6/28/23 08:18, Patrick Young wrote: Hi, I'm playing around with the gdb dataset https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d

[gdal-dev] ogr2ogr sqlite dialect and fgb format

Hi, I'm playing around with the gdb dataset https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d and getting errors starting with gdal 3.5 when I try to use the sqlite dialect: ogr2ogr -f GeoJSONSeq -sql "select shape from roadbed where source_id = 13350003311" -dialect

[gdal-dev] ogr2ogr: is there something like the inverse of clipsrc

Hi, I have a complex OpenFileGDB file, which has inside a lot of relations and then a lot of non-geometric tables. I need to spatially extract/clip 1/3 of this. So I thought I'd make a copy and to use an external layer not as clip (that maintains inside), but as an eraser. This way I am sure that

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

t; >> >> >> -Jukka Rahkonen- >> >> >> >> *Lähettäjä:* gdal-dev *Puolesta *Moises >> Calzado via gdal-dev >> *Lähetetty:* perjantai 5. toukokuuta 2023 12.32 >> *Vastaanottaja:* gdal-dev@lists.osgeo.org >> *Aihe:* Re: [gdal-dev

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

gt; my mind is a workaround as a geodata format. Maybe JSON could handle your > newlines by the same. > > > > -Jukka Rahkonen- > > > > *Lähettäjä:* gdal-dev *Puolesta *Moises > Calzado via gdal-dev > *Lähetetty:* perjantai 5. toukokuuta 2023 12.32 > *Vastaanottaja:

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Lähetetty: perjantai 5. toukokuuta 2023 12.32 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns Hi Even! I've just created the two issues: - https://github.com/OSGeo/gdal/issues/7699 - https://github.com/OSGeo/gdal

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi Even! I've just created the two issues: - https://github.com/OSGeo/gdal/issues/7699 - https://github.com/OSGeo/gdal/issues/7700 Robert, as I explained before, we need the `/vsistdout/` driver as we're processing the file in streaming mode, so we can't save the result to the storage.

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Moises, please fild 2 issues in the github issue tracker: - one about /vsistdout/ where .csvt and .prj content shouldn't be emitted - one about decoupling the layer GEOMETRY_NAME creation option with CREATE_CSVT=YES Even Le 04/05/2023 à 13:58, Moises Calzado via gdal-dev a écrit : Hi

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi. Here is an image of your dataframe.shp converted to CSV displayed in QGIS: https://i.imgur.com/6f6VFNf.png The dot CSV file in Excel and Calc https://i.imgur.com/twmfIPp.png On Thu, May 4, 2023 at 6:07 AM Robert Hewlett wrote: > Hi, > > Traced back through the thread and found your

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi, Traced back through the thread and found your sample data set and was able to convert it to a CSV file that both Calc, Excel and QGIS loaded without issue. The command I used the three files ogr2ogr -f CSV df.csv dataframe.shp -lco CREATE_CSVT=YES -lco GEOMETRY=AS_WKT -lco GEOMETRY_NAME=geom

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi Robert! I think that we're losing a bit the main issue that we reported, as in fact the problem is related with line breaks in the output generated while using /vsistdout and the CREATE_CSVT=YES option. Even pointed out that avoiding that flag it works as expected, but when it's used the

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi, I just tested with : GDAL 3.6.4, released 2023/04/17 Using the ogr2ogr as follows: ogr2ogr -f CSV poi_out.csv poi.shp -lco CREATE_CSVT=YES I get three files but no geometry ogr2ogr -f CSV poi_out.csv poi.shp -lco CREATE_CSVT=YES -lco GEOMETRY=AS_WKT I get three file with the geometry as WKT

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi, Not to start a controversy but it feels like the standard hints at three files. Did the standard change? If it is three files which works for me in QGIS and geopandas i.e. data lands where it is suppose to, then more layer creations options are needed to handle the SRID/CRS

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi Robert, Yes, we're getting one with all the info! El mié, 3 may 2023 a las 18:14, Robert Hewlett () escribió: > Just to clarify, instead of getting three files you are getting one with > all the info: types, projection, data? > > https://giswiki.hsr.ch/GeoCSV > > On Wed, May 3, 2023 at 8:57 

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Just to clarify, instead of getting three files you are getting one with all the info: types, projection, data? https://giswiki.hsr.ch/GeoCSV On Wed, May 3, 2023 at 8:57 AM Moises Calzado via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > We're also specifying the GEOM_POSSIBLE_NAMES, so it

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

We're also specifying the GEOM_POSSIBLE_NAMES, so it would be great if with that option we could use the GEOMETRY_NAME without using the CREATE_CSVT=YES option. Regarding emitting the .prj and .csvt in /vsistdout mode, that's why I'm saying that there is an issue while generating the resultant

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

The .CSVT and .PRJ help to make a proper geocsv dataset. Helps with QGIS And geopandas. The column name that I use in the CSV is usually geom and WKT shows up in the CSVT file which seems to be a one line file that hints at the data types in the CSV file. I hope that makes sense. CSVT Integer,

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Le 03/05/2023 à 14:22, Moises Calzado via gdal-dev a écrit : Hi Even, Thanks so much for taking a look into that one! I have one doubt regarding the CSVT content, as we're not really using it, but it's required when using the GEOMETRY_NAME layer creation option, as can be checked in the CSV

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hi Even, Thanks so much for taking a look into that one! I have one doubt regarding the CSVT content, as we're not really using it, but it's required when using the GEOMETRY_NAME layer creation option, as can be checked in the CSV driver documentation: >- > >GEOMETRY_NAME=name

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Moises, as far as I can see with your example, the CSV driver behaves "properly" in reading and writing of field values with line breaks. It follows the "Fields with embedded line breaks must be quoted" rule of https://en.wikipedia.org/wiki/Comma-separated_values $ ogr2ogr out.csv

[gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

Hello! We're trying to convert a Shapefile into a CSV using ogr2ogr and we're having some issues while dealing with some columns that contain line breaks inside their values. If we have a line with the following string, ogr2ogr detects that the line break is a new line and it returns two lines.

Re: [gdal-dev] OGR2OGR for ungeoreferenced shapefile to geolocated with 3 points

Cf the -gcp, -order and -tps options of https://gdal.org/programs/ogr2ogr.html. And -a_srs to assign a CRS. Le 21/02/2023 à 16:15, Vijay Kurhade a écrit : Hello, I have a ungeoreferenced shapefile; 3 geo-cordinates for the area on which I want to have this file geo-referenced. Any examples

[gdal-dev] OGR2OGR for ungeoreferenced shapefile to geolocated with 3 points

Hello, I have a ungeoreferenced shapefile; 3 geo-cordinates for the area on which I want to have this file geo-referenced. Any examples or guidance on how I can use the OGR2OGR library to get this file geo-referenced with those 3 points. If required will share the shapefile, 3 points. Thanks

Re: [gdal-dev] Ogr2ogr: when both -update and -append is needed?

Lng time ago you need to add both, but nowadays -append automatically enables -update. Just clarified it in https://github.com/OSGeo/gdal/commit/480456769abc11daae065bd112cb7361aa618636 Le 12/12/2022 à 21:25, Rahkonen Jukka a écrit : Hi, I saw this question

[gdal-dev] Ogr2ogr: when both -update and -append is needed?

Hi, I saw this question https://gis.stackexchange.com/questions/447359/what-is-the-effect-of-using-update-when-merging-geojsons-with-ogr2ogr and started to think when the ogr2ogr option -update is needed even when the -append option is used. At least for some drivers -append alone seems to be

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

are not allowed to install software or even run portable versions of applications. Greetings from EMOTET and Co.  Best regards Michael Von: gdal-dev Im Auftrag von Andrea Giudiceandrea via gdal-dev Gesendet: Freitag, 21. Oktober 2022 12:12 An: gdal-dev@lists.osgeo.org Betreff: Re: [gdal-dev] ogr2ogr

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

Il 21/10/2022 11:48, Reetz, Michael (NLPV) ha scritto: Since the GDAL/ogr MS SQL Server driver is obviously missing on my computer, I will see if I can get a complete installation of the current GDAL package. This would be very helpful in many situations, especially in scripts that perform

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

Betreff: Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server ACHTUNG!! Diese E-Mail erreicht Sie von einem Absender außerhalb der niedersächsischen Landesverwaltungs-Infrastruktur mit TLS-Verschlüsselung. Bitte klicken Sie auf keine Links oder öffnen Sie keine E-Mail-Anhänge, falls Sie den

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

Hi Michael, the GDAL/OGR library needs the "ogr_MSSQLSpatial.dll" file to properly activate the MSSQLSpatial GDAL/OGR driver ("MSSQL:"). You can check if the MSSQLSpatial GDAL/OGR driver is supported by your ogr2ogr tool using the following command in the OSGeo4W Shell: ogr2ogr --formats |

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

shipped installed with QGIS. So, the nice setup frontend is missing. Best regards Michael Von: Pedro Venâncio Gesendet: Donnerstag, 20. Oktober 2022 16:31 An: Reetz, Michael (NLPV) Cc: gdal-dev@lists.osgeo.org Betreff: Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server ACHTUNG!! Diese E

Re: [gdal-dev] ogr2ogr doesn't connect to MS SQL Server

Michael, Check if you have gdal-mss and msodbcsql libs installed, like here: https://github.com/PedroVenancio/export_to_sql_server/issues/2#issuecomment-1240675079 Best regards, Pedro Reetz, Michael (NLPV) escreveu no dia quinta, 20/10/2022 à(s) 15:13: > Dear members of the GDAL-DEV group,

[gdal-dev] ogr2ogr doesn't connect to MS SQL Server

Dear members of the GDAL-DEV group, we are using MS SQL Server 2019 as basis for Geoserver and GIS software. There are a lot of tables that have attributes of the types geometry and/or geography. I can load these tables into QGIS (3.22.8) projects without any problems. Now I'm asked to add an

Re: [gdal-dev] ogr2ogr preserve fid behavior

; > -Jukka Rahkonen- > > > > *Lähettäjä:* gdal-dev *Puolesta *Alexandre > Gacon > *Lähetetty:* torstai 13. lokakuuta 2022 11.13 > *Vastaanottaja:* gdal-dev > *Aihe:* [gdal-dev] ogr2ogr preserve fid behavior > > > > Hello, > > > > I am trying to upload

Re: [gdal-dev] ogr2ogr preserve fid behavior

Gacon Lähetetty: torstai 13. lokakuuta 2022 11.13 Vastaanottaja: gdal-dev Aihe: [gdal-dev] ogr2ogr preserve fid behavior Hello, I am trying to upload some data from SHP to Postgres. I would like to use a given column of my SHP as primary key in replacement of the id generated by ogr2ogr but I

[gdal-dev] ogr2ogr preserve fid behavior

Hello, I am trying to upload some data from SHP to Postgres. I would like to use a given column of my SHP as primary key in replacement of the id generated by ogr2ogr but I don't manage to have the result I want. I used the options -preserve_fid and --config PGSQL_OGR_FID id which gave the

[gdal-dev] ogr2ogr: -clipsrc but select-only without clipping?

Hi, I would like to use ogr2ogr for selecting all the features from the source layer that intersect with the reference layer but without computing the intersections. So fundamentally what the -spat option does but with more complicated geometries than BBOX. The -clipsrc option finds the right

Re: [gdal-dev] ogr2ogr error: ERROR 1: Maximum number of characters allowed reached.

Moises, there's a limit for the maximum size of a line to 1 MB that was added some time ago (post 3.2.2). I've increased it to 10 million bytes in https://github.com/OSGeo/gdal/pull/6403 and also added an open option to make it configurable. Even Le 20/09/2022 à 18:06, Moises Calzado via

[gdal-dev] ogr2ogr error: ERROR 1: Maximum number of characters allowed reached.

Hello everyone, We're facing some issues working with ogr2ogr in the 3.5.1 version, as we've found a dataset that provokes the following error during the execution of the following command: ogr2ogr -f CSV -skipfailures -makevalid /vsistdout/ > CSV:munic_s_anonymized.csv -simplify 0.1 -dim XY

Re: [gdal-dev] Ogr2ogr taking too much time to process a MapInfo TAB file

I confirm that the structure of the TAB dataset uses 512 bytes data blocks organized in a tree structure, so reading from the file implies lots of random access over the whole file even if you read the features sequentially since a single feature is stored in multiple data blocks of various

Re: [gdal-dev] Ogr2ogr taking too much time to process a MapInfo TAB file

Moises, I've not reviewed in depth the MITAB driver, but reading from a .tab file may require random access, and it is thus not surprising that reading from a compressed file may exhibit poor performance. You might try to set the VSI_CACHE config option / env variable to YES, but no

[gdal-dev] Ogr2ogr taking too much time to process a MapInfo TAB file

Hi everyone! We're using ogr2ogr to convert MapInfo TAB files into CSV format using the following command: ogr2ogr -f CSV -skipfailures -makevalid /vsistdout/ /vsizip/onLDU.zip -oo > AUTODETECT_TYPE=YES -lco CREATE_CSVT=YES > test_2.csv The file weights ≈200 MB and the process is taking too

Re: [gdal-dev] Ogr2ogr issue with big integers

Moises, the reason is that autodetection by default only considers the content of the first megabyte of the file, and in the first megabyte, only integer values that fit on 32 bit integers are present, hence this data type is selected. But later in the file, there are values that require a

[gdal-dev] Ogr2ogr issue with big integers

test.csv Hello everyone! I'm dealing with some issues with ogr2ogr trying to convert my data to a CSV file guessing the data types. I'm using the following command: ogr2ogr -f CSV -skipfailures -makevalid

Re: [gdal-dev] Ogr2ogr conversion to geojson problem..... any suggestion?

Frando Alexis, when using ogr2ogr, there are 3 components involved: - the driver that reads the input file - the driver that writes the output file - the ogr2ogr specific logic So when analyzing an issue, it is important to try to localize where it is. Here the issue comes from the input

[gdal-dev] Ogr2ogr conversion to geojson problem..... any suggestion?

Hi dev team! I am writing to inquire about conversion to geojson problem, I will show you in a little and simple example that shows my big problem: We are using the ogr2ogr executables commands like this : ogr2ogr -f GeoJSON C:\tmp\nombres.json C:\tmp\TUNJA1R_ejemplo.xtf testModel.table As

[gdal-dev] ogr2ogr -clipsrc datasource postgis?

Hi all, I'm trying to import a layer from a filegeodatabase into postgis and clip it to a separate postgis layer. Is that possible? I can successfully clip using the clipsrc argument with a bounding box I've read that it will also work for a shapefile, but I'd prefer to use a postgis layer as the

Re: [gdal-dev] ogr2ogr Postgres upload performance

ogr2ogr command for each layer. -Jukka Rahkonen- *Lähettäjä:* gdal-dev *Puolesta *Alexandre Gacon *Lähetetty:* maanantai 23. toukokuuta 2022 17.39 *Vastaanottaja:* Jan Heckman *Kopio:* gdal-dev *Aihe:* Re: [gdal-dev] ogr2ogr Postgres upload performance I will try

Re: [gdal-dev] ogr2ogr Postgres upload performance

er. > > > > -Jukka Rahkonen- > > > > *Lähettäjä:* gdal-dev *Puolesta *Alexandre > Gacon > *Lähetetty:* maanantai 23. toukokuuta 2022 17.39 > *Vastaanottaja:* Jan Heckman > *Kopio:* gdal-dev > *Aihe:* Re: [gdal-dev] ogr2ogr Postgres upload performance > > > > I w

Re: [gdal-dev] ogr2ogr Postgres upload performance

Vastaanottaja: Jan Heckman Kopio: gdal-dev Aihe: Re: [gdal-dev] ogr2ogr Postgres upload performance I will try this way. To turn of SI creation : SPATIAL_INDEX=NONE Le lun. 23 mai 2022 à 15:15, Jan Heckman mailto:jan.heck...@gmail.com>> a écrit : Perhaps the spatial index update (in

Re: [gdal-dev] ogr2ogr Postgres upload performance

I meant : "I used the "PG_USE_COPY *yes*" option without any other refinements" Med venlig hilsen / Kind regards Bo Victor Thomsen Den 24-05-2022 kl. 08:11 skrev Alexandre Gacon: As far as I remember, I used the "PG_USE_COPY no" option without any other refinements. But I didn't

Re: [gdal-dev] ogr2ogr Postgres upload performance

Hello, The PG_USE_COPY parameter doesn't seem to work when appending several files in the same table (replace the old data or something like that). Postponing the creation of the spatial index does not seem to improve things either. I didn't try increasing the number of rows per transaction but

Re: [gdal-dev] ogr2ogr Postgres upload performance

Hi, Thank you for the input. I will give a try to different combination and share my results. Alexandre Le lun. 23 mai 2022 à 19:43, Bo Victor Thomsen a écrit : > Hi Alexandre - > > I made a DOS script ages ago to try the different use cases. It's inserted > as text below. It doesn't test

Re: [gdal-dev] ogr2ogr Postgres upload performance

Hi Alexandre - I made a DOS script ages ago to try the different use cases. It's inserted as text below. It doesn't test every setup combination, only the most pertinent. The base gpkg layer is all the buildings i Denmark, ca. 5.8 million objects. For my layer I had the following results:

Re: [gdal-dev] ogr2ogr Postgres upload performance

I will try this way. To turn of SI creation : *SPATIAL_INDEX*=NONE Le lun. 23 mai 2022 à 15:15, Jan Heckman a écrit : > Perhaps the spatial index update (in de DB) slows the insertion. > It might be more efficient to not create the SI in the first run, > then insert more rows and create the SI

Re: [gdal-dev] ogr2ogr Postgres upload performance

Perhaps the spatial index update (in de DB) slows the insertion. It might be more efficient to not create the SI in the first run, then insert more rows and create the SI after all is done. I don't know how to turn off SI creation in the command line. On Mon, May 23, 2022 at 3:04 PM Alexandre

[gdal-dev] ogr2ogr Postgres upload performance

Hello, I am using ogr2ogr to upload data from several geopackages to a postgis database. Some tables contain thousands of rows (buildings for example). The import of the first file is quite fast (tables are created for the first file so PG_USE_COPY is used) but the following file is much slower

[gdal-dev] Ogr2ogr: disk I/O errors on external hard drive

Hello, I get a lot of disk I/O errors using exec commands like: "C:\OSGeo4W\bin\ogr2ogr -f gpkg E:\OSM\vektor_gpkg\Kontinente\europe-latest.osm.noAuthor_-44-46.gpkg E:\OSM\Kontinente\europe-latest.osm.noAuthor_-44-46.pbf multipolygons -select

Re: [gdal-dev] ogr2ogr and WFS 2

*Vastaanottaja:* gdal-dev@lists.osgeo.org *Aihe:* [gdal-dev] ogr2ogr and WFS 2 Dear all I seem to have problems understanding how ogr2ogr interacts with WFS and hope someone would be able to clarify. I have this WFS: https://stationsregister.miljodatasamverkan.se/geoserver/stationsregistret/wfs

Re: [gdal-dev] ogr2ogr and WFS 2

- *Lähettäjä:* gdal-dev *Puolesta *Hernán De Angelis *Lähetetty:* maanantai 29. marraskuuta 2021 12.43 *Vastaanottaja:* gdal-dev@lists.osgeo.org *Aihe:* [gdal-dev] ogr2ogr and WFS 2 Dear all I seem to have problems understanding how ogr2ogr interacts with WFS and hope someone would be able

Re: [gdal-dev] ogr2ogr and WFS 2

OGR_WFS_PAGING_ALLOWED OFF. I would try that. -Jukka Rahkonen- Lähettäjä: gdal-dev Puolesta Hernán De Angelis Lähetetty: maanantai 29. marraskuuta 2021 12.43 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] ogr2ogr and WFS 2 Dear all I seem to have problems understanding how ogr2ogr interacts

Re: [gdal-dev] ogr2ogr and WFS 2

Thank you for the clarification, Even. Now I understand how this works. I also think the pull request will help to make ogr2ogr behave more as is intuitively expected. Have a nice day! Hernán Den 2021-11-29 kl. 13:11, skrev Even Rouault: Hernán, The URL you specify in the WFS: connection

Re: [gdal-dev] ogr2ogr and WFS 2

Hernán, The URL you specify in the WFS: connection string isn't in general the one of a GetFeature request, but only the endpoint of the service. When you specify a GetFeature request, some heuristics trigger, but in WFS 2, the COUNT parameter is interpreted as the page size, but as this

[gdal-dev] ogr2ogr and WFS 2

Dear all I seem to have problems understanding how ogr2ogr interacts with WFS and hope someone would be able to clarify. I have this WFS: https://stationsregister.miljodatasamverkan.se/geoserver/stationsregistret/wfs? The capabilities document states that it supports WFS 1.0.0, 1.1.0 and

Re: [gdal-dev] ogr2ogr, perform queries to obtain values from the database

that before and my view is therefore certainly biased. -Jukka Rahkonen- Lähettäjä: gdal-dev Puolesta Andrés Nadal Lähetetty: torstai 18. marraskuuta 2021 16.50 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] ogr2ogr, perform queries to obtain values from the database Hello. Excuse me

[gdal-dev] ogr2ogr, perform queries to obtain values from the database

Hello. Excuse me my English, it's not my native language. I need to convert the names (string) of classes, in some fields of a Shapefile, to the respective numerical values of the database. In some cases, the fields have up to 3 different classes, for different rows. CLASS A = 1 CLASS B = 2

Re: [gdal-dev] ogr2ogr does not convert OSM polygons

different database schemas so they are not very comparable. -Jukka Rahkonen- Lähettäjä: gdal-dev Puolesta Clay, Bruce Lähetetty: tiistai 2. marraskuuta 2021 16.22 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] ogr2ogr does not convert OSM polygons When I convert a OSM pbf file to post

[gdal-dev] ogr2ogr does not convert OSM polygons

When I convert a OSM pbf file to postgres using the following script it does not create a multipolygon table ogr2ogr -f PostgreSQL PG:"dbname='open_street_map' host='db-host' port='5432' user='postgres' password='pwd'" -lco schema=country country_pbf --config OSM_MAX_TMPFILE_SIZE 1024 When I

Re: [gdal-dev] ogr2ogr changes field length

Hi Jukka, GDAL has the internal default of 80 characters. GDAL users do not really need other options because fields are automatically expanded when longer strings are appended afterwards but if the shapefile is used in some other software it does happen that it is impossible to insert long

Re: [gdal-dev] ogr2ogr changes field length

: matteo Lähetetty: perjantai 8. lokakuuta 2021 12.29 Vastaanottaja: Rahkonen Jukka (MML) ; Even Rouault ; gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] ogr2ogr changes field length Hi Jukka, works perfectly, thanks! so no default options to impose the field length but it is alterable after

Re: [gdal-dev] ogr2ogr changes field length

Hi Jukka, works perfectly, thanks! so no default options to impose the field length but it is alterable after Cheers! Matteo ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr2ogr changes field length

). -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: matteo Lähetetty: perjantai 8. lokakuuta 2021 11.20 Vastaanottaja: Rahkonen Jukka (MML) ; Even Rouault ; gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] ogr2ogr changes field length Hi Jukka, > The strings fields are created by defau

Re: [gdal-dev] ogr2ogr changes field length

Hi Jukka, The strings fields are created by default with width 80. If there are longer strings in the data the width is automatically extended up till 255 characters that is the maximum that dBase format supports. If strings are shorter than 80 charaters it is possible to use RESIZE and

Re: [gdal-dev] ogr2ogr changes field length

some concrete problem with field widths that you want to resolve? -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: gdal-dev Puolesta matteo Lähetetty: perjantai 8. lokakuuta 2021 9.11 Vastaanottaja: Even Rouault ; gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] ogr2ogr changes field

Re: [gdal-dev] ogr2ogr changes field length

Hi Even, If you add -lco RESIZE=YES fields will be resized to their minimum size. See https://gdal.org/drivers/vector/shapefile.html#layer-creation-options yep I see. I also read * String fields without an assigned width are treated as 80 characters does it mean that if a field is set as

Re: [gdal-dev] ogr2ogr changes field length

If you add -lco RESIZE=YES fields will be resized to their minimum size. See https://gdal.org/drivers/vector/shapefile.html#layer-creation-options Le 07/10/2021 à 17:40, matteo a écrit : Even, still a weird issue with the command. I'm using this command ogr2ogr -f "ESRI Shapefile"

Re: [gdal-dev] ogr2ogr changes field length

Even, still a weird issue with the command. I'm using this command ogr2ogr -f "ESRI Shapefile" /home/matteo/myfolder "PG:dbname='mydb' host=myhost port=5432 user='myuser' password='mypws' sslmode=disable schemas='dati'" --config PG_USE_COPY YES --config PG_LIST_ALL_TABLES YES -progress' so

Re: [gdal-dev] ogr2ogr changes field length

Hi Even, thanks for the hint :) it si just an additional step Cheers! Matteo ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr2ogr changes field length

Matteo, When overwriting, a fresh new file/table is creating, without keeping anything from the existing version. You could probably just blank your target with ogrinfo target.shp -sql "DELETE FROM target" -dialect SQLITE and then ogr2ogr -append target.shp PG:   -nln target Even Le

[gdal-dev] ogr2ogr changes field length

Hi list, I've a script that uses ogr2ogr to copy, actually overwrite, tables from PG to shapfiles. The PG fields are all text (without a specified length) and the target shapefile that will be overwritten have all text field length equals to 254. When overwriting the length is changes to

Re: [gdal-dev] ogr2ogr from gpx to postgis - track_fid not maintained for track_points

On Sat, Jan 2, 2021 at 7:59 AM Michael Saupe wrote: > Hi Evan, > > thank you for the explanation. That makes sense. However, in my opinion > ogr2ogr behavior is not really consistent in this case: The column ogc_fid > , which is in fact the track id, is incremented on each gpx file import ( >

Re: [gdal-dev] ogr2ogr from gpx to postgis - track_fid not maintained for track_points

Hi, For my mind the behavior is consistent. FID (in this case named as ogc_fid) is a unique feature ID and tracks table contains tracks so it is natural that FID is increasing when more tracks are added - and it must increase or otherwise the FID is not unique. The track_points table contains

Re: [gdal-dev] ogr2ogr from gpx to postgis - track_fid not maintained for track_points

I am referring to ogc_fid of 'tracks' table. Am 02.01.2021 um 15:59 schrieb Michael Saupe: > Hi Evan, > > thank you for the explanation. That makes sense. However, in my opinion > ogr2ogr behavior is not really consistent in this case: The column ogc_fid , > which is in fact the track id, is

  1   2   3   4   5   6   7   8   9   >