OK, I’ll try it. Thanks for theses informations. Matthieu
> Le 13 oct. 2017 à 12:43, Even Rouault <[email protected]> a écrit : > > On vendredi 13 octobre 2017 10:06:56 CEST Matthieu Lefort wrote: > > Hi list, > > I am using ogr2ogr to load big geojson files (about 1 Go each) into > > postgresql database. On my MacBook, (GDAL 1.11.3, released 2015/09/16), > > although it lasts about 20 minutes for each file, this works fine. I tried > > to do the same thing on a VPS (Debian Stretch 9.2, 8 Go RAM) but the > > process consumes all RAM in 20 seconds before stopping (out of memory). > > Install was made with apt-get install gdal-bin. (GDAL 2.1.2, released > > 2016/10/24) > > > > Is there a way to control memory usage or to avoid at least using more > > memory than the available one ? > > Matthieu, > > no, this is a known issue ( see https://trac.osgeo.org/gdal/ticket/6540 > <https://trac.osgeo.org/gdal/ticket/6540> ) which would require substantial > changes in the geojson driver to be adressed. It needs to ingest the whole > file in memory currently, parse it completely, and store the resulting > features in RAM. So typically, you need 10 to 20 times more RAM than the file > size (I wouldn't expect the memory usage to be substantially different > between GDAL 1.11 and 2.1, but there has been indeed a small change to use > the MEM driver underneath) > > So for now, your best chance is to try splitting the features[] array into > smaller files. > > https://www.google.fr/search?q=splitting+geojson+files > <https://www.google.fr/search?q=splitting+geojson+files> points to various > hints on how to split geojson files. > > Even > > > -- > Spatialys - Geospatial professional services > http://www.spatialys.com <http://www.spatialys.com/>
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
