In the case if FileGDB, which driver you use to read the FileGDB makes a big difference.
On Thu, Jan 20, 2022 at 8:59 AM Jon Morris <[email protected]> wrote: > Hi Sean, > > > > That seems to be the best option. I was testing on Shapefile initially, > but after I posted, I tried a memory layer and got quite different results. > Ultimately, our output format is determined by clients (it's usually > Shapefile or File GDB), so we'll just have to let them know there are > performance implications with some formats. > > > > Thanks, > > > > Jon > > > e: *[email protected]* <[email protected]> > t: *+44 (0)1756 799919* <+44%20(0)1756%20799919> > *www.jbarisk.com* <http://www.jbarisk.com/> > [image: Facebook] <https://www.facebook.com/TheFloodPeople> > [image: LinkedIn] <https://www.linkedin.com/company/jba-risk-management/> > [image: Twitter] <https://twitter.com/JBARisk> > [image: YouTube] > <https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw> > > > All JBA Risk Management's email messages contain confidential information > and are intended only for the individual(s) named. If you are not the named > addressee you should not disseminate, distribute or copy this e-mail. > Please notify the sender immediately by email if you have received this > email by mistake and delete this email from your system. > JBA Risk Management Limited is registered in England, company number > 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North > Yorkshire, BD23 3FD, England. > > *From:* gdal-dev <[email protected]> *On Behalf Of *Sean > Gillies > *Sent:* 20 January 2022 15:44 > *To:* [email protected] > *Subject:* Re: [gdal-dev] Faster alternative to GetFeatureCount? > > > > *CAUTION:* This email originated from outside of JBA and *contains one or > more links and one or more attachments*. DO NOT click links or open > attachments unless you recognise the sender's email address and are > absolutely certain that the content is safe. > See the Phishing page on IMS on SP for more information about how to spot > and report suspicious messages. > > *WARNING: This email contains one or more attachments with links within > the attachment(s)*. Please take extra care when handling the attachments. > > Hi Jon, > > > > The performance of GetFeatureCount, filters, and GetNextFeature depends a > lot on the data format. If your data is in Postgres, for example, these > operations are super fast because the database keeps the number of rows > (features) in memory and has highly optimized views of selected rows. If > your data is in a CSV file, GDAL has to read every line to count the number > of features and when getting filtered features has to read all the lines in > between the ones you want. See > https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/csv/ogrcsvlayer.cpp#L1667 > . > > > > Can you switch to a more optimized format? > > > > On Thu, Jan 20, 2022 at 2:02 AM Jon Morris <[email protected]> wrote: > > I'm writing applications using the GDAL Python bindings and when I profile > for performance, GetFeatureCount frequently comes out near the top. I'm > often using it to check whether a spatial or attribute filter has returned > any features and don't need the full count. When the layer contains > millions of features, there would be a big performance improvement if we > could exit the count early. > > > > Is there a better way of doing this? I've tried using GetNextFeature > instead, but there must be quite a lot of overhead in that function as it > is much slower. All I need to know is if the layer has has 0, 1 or >1 > features, I don't need the actual count. Can anyone suggest the fastest way > of doing this in Python? I'm using GDAL 3.3.1 at the moment but could > upgrade if there is new functionality that would help. > > > > Thanks, > > > > Jon > > > > *Jon Morris* > > *Software Developer* > > > > > > *e: * > > [email protected] > > *t: * > > +44 (0)1756 799919 <+44%20(0)1756%20799919> > > www.jbarisk.com > > [image: Facebook] <https://www.facebook.com/TheFloodPeople> > > [image: LinkedIn] <https://www.linkedin.com/company/jba-risk-management/> > > [image: Twitter] <https://twitter.com/JBARisk> > > [image: YouTube] > <https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw> > > > > > > > > All JBA Risk Management's email messages contain confidential information > and are intended only for the individual(s) named. If you are not the named > addressee you should not disseminate, distribute or copy this e-mail. > Please notify the sender immediately by email if you have received this > email by mistake and delete this email from your system. > JBA Risk Management Limited is registered in England, company number > 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North > Yorkshire, BD23 3FD, England. > > > > -- > > Sean Gillies > _______________________________________________ > gdal-dev mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/gdal-dev >
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
