Scott and list, With respect I think this conversation is moving off the point, which was related to a 5gb shapefile
https://en.m.wikipedia.org/wiki/Shapefile Which is beyond the design limits of the Shapefile standards and perhaps therefore the source of the problem that the OP has. Notably not a limitation of QGIS On Thu, Jul 28, 2022, 10:45 Madry, Scott via Qgis-user < [email protected]> wrote: > Dear colleagues, > > As we move more into the world of big data, IOE, data mining, and data > analytics, dealing with larger and larger data sets will become required, > and, eventually, the norm. > > GRASS GIS has made very good progress in this regard, optimizing the code > to run on massively parallel computer architectures, with very good effect: > > https://grasswiki.osgeo.org/wiki/Supercomputing > > Perhaps it is time for QGIS to consider this in its future development > plans? We will certainly be needing this in the future. > > And thanks to all the team who keep the QGIS train running, I am > constantly amazed at the energy, effort, and availability of people to the > broader community. > > Scott > > Scott Madry, Ph.D. > Research Associate Professor of Archaeology > The University of North Carolina at Chapel Hill > > Tel 1-919-448-4493 > Email: [email protected] <[email protected]> > https://scottmadry.web.unc.edu > Skype: scott madry > > > > On Jul 28, 2022, at 10:14 AM, jhubbslist--- via Qgis-user < > [email protected]> wrote: > > Past a point, trying to run a GIS app with very large data structures just > doesn't work well. If you really need all the information you're carrying > around for your analysis or whatever, you may need to reach for different > tools or use the tools you have differently. > > I assume it's not so much the >5GiB of disk space that's the issue and > that you've maxxed out the CPU, graphics, and disk I/O rate as much as is > practical so mostly it's a matter of how long it takes maps etc. to paint > onscreen. It may help you to move the heavy-lift onto PostgreSQL/PostGIS > where you can make use of spatial indexing. Or, you can craft your > operations the way you want them in QGIS but do the actual work with e.g. > GDAL calls in Python. I spoke with someone a couple weeks ago whose > particular GIS process worked better in GRASS than in QGIS, so that's > something you might look into as well. > > On 7/28/22 7:52 AM, krishna Ayyala via Qgis-user wrote: > > dbf file itself is 5.1GB. Rest all other files are less than 500MB. It is > the number of records which is huge. It has about 117,2100 points. > > On Thu, Jul 28, 2022 at 12:02 AM Bernd Vogelgesang via Qgis-user < > [email protected]> wrote: > >> how big is the dbf file of that shape? Maybe you can also drop some >> attributes. >> >> Am 28.07.22 um 03:46 schrieb krishna Ayyala via Qgis-user: >> > Hello, >> > I have a shapefile of 5GB in size. Is it possible to convert this >> > shapefile to a smaller size file? It can be any format, not >> > necessarily a shapefile. But, preferably a vector format. I tried to >> > convert it into tiles, but that didn't work as it was losing the >> > resolution. I am looking to convert this 5GB size file to about 500MB. >> > >> > Regards. >> > >> > _______________________________________________ >> > Qgis-user mailing list >> > [email protected] >> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user >> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user >> _______________________________________________ >> Qgis-user mailing list >> [email protected] >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user >> > > _______________________________________________ > Qgis-user mailing [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > > > _______________________________________________ > Qgis-user mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > > > _______________________________________________ > Qgis-user mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user >
_______________________________________________ Qgis-user mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-user Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
