> > Hi There, > > I'm working with a set of fairly large geojsonl files, one for each state > and territory in the United States. I'm using ogr2ogr to ingest this data > into PostGIS, generally without issue, to speed up an expensive geospatial > join. > > However, one of my files, specifically the Commonwealth of Pennsylvania, > is about 6.5 GB and has been throwing an odd error that I can't seem to > find much information on. "ERROR 6: Too large feature". Below is the > command and output from running ogrinfo with the debug flag set to on, > which returns the same error. > > I'm just hoping for some guidance in terms of what if anything I can do to > sidestep the issue. > > So far, I've taken a look at what seems to be the GDAL source code that > generates this error > <https://github.com/OSGeo/gdal/blob/030ff40cf8340273bcc797e90c938cc32d14a34f/ogr/ogrsf_frmts/geojson/ogrgeojsonseqdriver.cpp#L410>, > but it's unclear to me if it's possible to adjust the size constraint that > is specified in the condition that throws the error or if the constraint is > arbitrary or indicative of a larger problem with the file. I've already > tried using the -gt flag with various sizes as well, but none seem to > resolve the issue. I ran into an issue previously that required me to set > OGR_GEOJSON_MAX_OBJ_SIZE=2048MB but that doesn't seem to have any effect > here for some reason. Perhaps that only effects the GeoJSON driver and not > the GeoJSONSeq driver? For what it's worth this is running on a node with > ~350GB of RAM, so generally memory shouldn't be an issue. > > ogrinfo --version > GDAL 3.5.0, released 2022/05/10 > ------------------------------------------------------ > > ogrinfo --build > > PAM_ENABLED=YES > OGR_ENABLED=YES > GEOS_ENABLED=YES > GEOS_VERSION=3.10.2-CAPI-1.16.0 > PROJ_BUILD_VERSION=9.0.0 > PROJ_RUNTIME_VERSION=9.0.0 > ------------------------------------------------------ > > ogrinfo crs4326_fema_42_2022-07-22.geojsonl --debug ON > GeoJSONSeq: First pass: 0.06 % > GeoJSONSeq: First pass: 0.12 % > GeoJSONSeq: First pass: 0.18 % > ... > GeoJSONSeq: First pass: 92.08 % > GeoJSONSeq: First pass: 92.14 % > GeoJSONSeq: First pass: 92.20 % > ERROR 6: Too large feature > GDAL: GDALOpen(crs4326_fema_42_2022-07-22.geojsonl, this=0x564966fae8b0) > succeeds as GeoJSONSeq. > INFO: Open of `crs4326_fema_42_2022-07-22.geojsonl' > using driver `GeoJSONSeq' successful. > OGR: GetLayerCount() = 1 > > 1: crs4326_fema_42_2022-07-22 (Multi Polygon) > GDAL: GDALClose(crs4326_fema_42_2022-07-22.geojsonl, this=0x564966fae8b0) > GDAL: In GDALDestroy - unloading GDAL shared library. > > Thank you for any help you're able to offer, it's much appreciated. > > TL > > Theo Linnemann <https://www.theolinnemann.com/> >
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev