Thanks for the graphs Duncan! My guess is that at the low tile sizes, the individual round-trip time would be short but the volume of disk i/o is what's coming into play. The opposite is true for large tile sizes where the disk i/o volume is decreased but it takes longer for each round-trip.
Having said all that, I'm not willing to suggest that there is one range appropriate for every user or deployment. Assuming you ran your scripts on a typical desktop/laptop, I'd agree that the 200 range is an adequate starting point. If on the other hand you're deploying with more hardware (lots of RAM [128GB+], large number of disk spindles (DAS, FC SANs), SSDs or RAM drives), I would expect that curve to look very different. -bborie I really need to build a performance test suite... On Thu, Sep 19, 2013 at 4:55 PM, Duncan Golicher <[email protected]>wrote: > Here is a simple speed test using R system.time to record the result after > reloading the same raster with different tile sizes. > > https://dl.dropboxusercontent.com/u/2703650/SpeedTest.html > > Not had any time to anotate or explain the code but the test should be > easily replicable (at least under Ubuntu). I provide a link to the data > (which was used in the example on the geostat course). The code only runs > without modification on Linux as I use system to send commands pasted > toegether in R to the shell. Also needs unix odbc setting up. > > Note that the point on raster overlay can be beaten easily for speed by > the extract function in the R raster package. However the polygon overlays > are now very fast and compare well with any alternative way of getting the > result. Using PLR to run R functions within PostGIS is great if you want > medians, quartiles etc or any other derived property. > > Duncan > > > > > On Thu, Sep 19, 2013 at 4:32 PM, Duncan Golicher <[email protected]>wrote: > >> Out of interest I quickly checked whether the conclusions still hold for >> PostGIS2.2. >> The changes made in ST_Clip, and some other functions including st_value >> seem to have altered not just the absolute timing (much faster) but also >> the relative timing of operations as a function of tile size. >> Point on raster overlays are now slower when tile size is small (<50 >> pixels), whereas previously there was an almost linear increase with tile >> size. Bborie may be able to explain why this change has occurred. I will >> try to add an update to the weblog at some point in order to clarify the >> sitation. It is GOOD NEWS as apparently there is now a single optimum tile >> size for both polygon and point overlays and this does seem to lie at >> around 200 - 300 pixels using the same example as the weblog, although I >> have not run enough tests to confirm this. I'll try to find time to confirm >> this. >> >> Duncan >> >> >> >> >> >> On Thu, Sep 19, 2013 at 10:36 AM, Pierre Racine < >> [email protected]> wrote: >> >>> You can also have a look at this article from Duncan Golicher if you are >>> doing raster/vector analysis: >>> >>> >>> http://duncanjg.wordpress.com/2012/10/30/tile-size-for-raster-vector-overlays-in-postgis/ >>> >>> > -----Original Message----- >>> > From: [email protected] [mailto:postgis-users- >>> > [email protected]] On Behalf Of Stephen Crawford >>> > Sent: Friday, September 13, 2013 12:34 PM >>> > To: [email protected] >>> > Subject: Re: [postgis-users] tile size >>> > >>> > OK, thanks. I will give that a try. >>> > >>> > >>> > >>> > On 9/13/2013 12:31 PM, Adam Eskreis wrote: >>> > >>> > >>> > The most common tile size that I've seen in production is 256x256 >>> > >>> > >>> > On Fri, Sep 13, 2013 at 10:33 AM, Bborie Park >>> > <[email protected]> wrote: >>> > >>> > >>> > Steve, >>> > >>> > There really isn't. What I do recommend is that if your >>> raster >>> > data is not going to change over time (and you don't need to replicate >>> the >>> > database), load them as out-db rasters. That way, you can easily >>> change tile >>> > size within the database with ST_Tile. >>> > >>> > >>> > >>> > -bborie >>> > >>> > >>> > On Fri, Sep 13, 2013 at 6:06 AM, Stephen Crawford >>> > <[email protected]> wrote: >>> > >>> > >>> > Hi All, >>> > >>> > Is there a rule of thumb for determining the >>> best tile >>> > size when tiling a raster? >>> > >>> > Thanks, >>> > Steve >>> > >>> > >>> > -- >>> > Stephen Crawford >>> > Center for Environmental Informatics >>> > The Pennsylvania State University >>> > <tel:814.865.9905> >>> > >>> > >>> > >>> > _______________________________________________ >>> > postgis-users mailing list >>> > [email protected] >>> > http://lists.osgeo.org/cgi- >>> > bin/mailman/listinfo/postgis-users >>> > >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > postgis-users mailing list >>> > [email protected] >>> > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis- >>> > users >>> > >>> > >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > postgis-users mailing list >>> > [email protected] >>> > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users >>> > >>> > >>> > -- >>> > Stephen Crawford >>> > Center for Environmental Informatics >>> > The Pennsylvania State University >>> > [email protected] >>> > 814.865.9905 >>> _______________________________________________ >>> postgis-users mailing list >>> [email protected] >>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users >>> >> >> >> >> -- >> Dr Duncan Golicher >> Investigador Titular, >> El Colegio de la Frontera Sur, Chiapas,Mexico >> Mexico tel +52 1 967 137 94 20 >> Skype name duncangolicher >> >> Publications: http://www.mendeley.com/profiles/duncan-golicher >> >> Senior lecturer, Bournemouth University, UK >> Centre for Conservation Ecology & Environmental Change >> School of Applied Sciences >> Christchurch House rm C218a >> Bournemouth University >> Fern Barrow >> Poole (Dorset) BH12 5BB UK >> Tel. +44 (0)1202 961682 >> >> For list of publications see Researcher ID: >> http://www.researcherid.com/rid/B-4240-2009 >> >> [email protected] >> [email protected] >> >> Researcher ID: >> http://www.researcherid.com/rid/B-4240-2009 >> > > > > -- > Dr Duncan Golicher > Investigador Titular, > El Colegio de la Frontera Sur, Chiapas,Mexico > Mexico tel +52 1 967 137 94 20 > Skype name duncangolicher > > Publications: http://www.mendeley.com/profiles/duncan-golicher > > Senior lecturer, Bournemouth University, UK > Centre for Conservation Ecology & Environmental Change > School of Applied Sciences > Christchurch House rm C218a > Bournemouth University > Fern Barrow > Poole (Dorset) BH12 5BB UK > Tel. +44 (0)1202 961682 > > For list of publications see Researcher ID: > http://www.researcherid.com/rid/B-4240-2009 > > [email protected] > [email protected] > > Researcher ID: > http://www.researcherid.com/rid/B-4240-2009 > > _______________________________________________ > postgis-users mailing list > [email protected] > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users >
_______________________________________________ postgis-users mailing list [email protected] http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
