Thank you.
These building outlines don't produce fields that are extremely wide
(lots of characters on a single line). But I tend to store my geoj's
with each point on a single line, to prevent excessively wide lines.
It makes a BIG difference for QGIS with the geoj's I work with.
You might try that just to see if it makes a difference for you.
So you could replace "],[" with "],\n[" (\n being a newline). The file
might get even bigger, but QGIS is much happier with the newlines than
with the wide rows.
Mike
On 7/29/21 9:26 AM, Newcomb, Doug wrote:
https://github.com/microsoft/USBuildingFootprints
<https://github.com/microsoft/USBuildingFootprints>
<https://github.com/microsoft/USBuildingFootprints>
GitHub - microsoft/USBuildingFootprints: Computer generated building
footprints for the United States
<https://github.com/microsoft/USBuildingFootprints>
Introduction. Microsoft Maps is releasing country wide open building
footprints datasets in United States. This dataset contains
129,591,852 computer generated building footprints derived using our
computer vision algorithms on satellite imagery.
github.com
Texas is 2.38 GB , but you get a variety of other sizes
------------------------------------------------------------------------
*From:* gdal-dev <[email protected]> on behalf of Mike
Flannigan <[email protected]>
*Sent:* Thursday, July 29, 2021 9:49 AM
*To:* [email protected] <[email protected]>
*Subject:* [EXTERNAL] Re: [gdal-dev] Large GeoJSONs and aborting file
opening
This email has been received from outside of DOI - Use caution before
clicking on links, opening attachments, or responding.
I would like to hear more about large GeoJSON files.
How large are they?
My GeoJSON files contain linear features only. My
largest one is 50.2 MB with 1,230,000 newlines in it.
Next biggest one is 12 MB with 280,000 newlines. These
and about 140 other geojsons are open in the same project
and I have no problems. In fact I converted from
SHP to geojson 2 years ago because I used to have problems
with SHP linear files.
I use QGIS 3.16.8 on Linux Mint.
Mike
On 7/28/21 2:36 PM, [email protected] wrote:
> Date: Wed, 28 Jul 2021 12:22:12 -0700
> From: Simon Eves<[email protected]>
> To:[email protected]
> Subject: [gdal-dev] Large GeoJSONs and aborting file opening
> Message-ID:
> <cajf0ktrsasksospv8tba+itb+tql_ui5y4n05wgldw_3gur...@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear All,
>
> I am aware that some improvements were made in the 2.3 timeframe with
> regards to dealing with large GeoJSON files, although even in 3.2, it's
> still very slow and memory hungry.
>
> Our system allows for aborting imports, but this only works reliably
once
> it has actually got to the stage of reading features from the file. With
> the GeoJSON, it just sits in the GDALOpenEx call for ages.
>
> My question, therefore, is whether it might be practical to run the
> GDALOpenEx in a separate thread with a future to return the resulting
> handle, such that it could be monitored and killed if necessary?
>
> Mainly I would be concerned that killing the thread might trash some
global
> GDAL state that might then not be recoverable, or that the open
relies on
> some TLS for the process thread and therefore might not work properly.
>
> We're going to try it anyway, but opinions welcomed, thanks!
>
> Simon
_______________________________________________
gdal-dev mailing list
[email protected]
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0
<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0>
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
