If I create overviews for the images, do the Geoserver layers associated with it need to be recreated?
Max On 17 March 2016 at 14:21, Rahkonen Jukka (MML) <[email protected]> wrote: > Hi, > > > > If you want to keep all 60000 layers which is not optimal as Andrea just > wrote, the most obvious thing to do is to create overviews for the images. > > gdaladdo –r average 2 4 8 16 32 64 image.tif --config COMPRESS_OVERVIEW > DEFLATE > > > > It is worth making a test before reprojecting images physically into > EPSG:3857. On-the-fly reprojecting is not necessarily so slow. > > I saw that you have used the BIGTIFF=YES option. It is not needed for your > small images and it probably makes reading the images a little bit slower > but I guess that the difference is marginal. Tile size 256x256 could be a > bit faster than 512x512 but that’t also marginal. > > > > > > -Jukka Rahkonen- > > > > basiliosz wrote: > > > > As 60000 layers. They are not contiguous, are completely separate and are > viewed strictly one at a time per user. > > The scenario: user selects a city and a couple parameters, the JS code > figures out what layer to request, then submits the request for that layer > to Geoserver. Eventually the layer is added to the map. > > Max > > On 17 Mar 2016 13:44, "Andrea Aime" <[email protected]> wrote: > > Hi, > > just to be sure, how have you published them? > > As a single image mosaic or as 60000 separate layers? > > > > Cheers > > Andrea > > > > > > On Thu, Mar 17, 2016 at 12:56 PM, Max <[email protected]> wrote: > > Thanks - because of the origin of the data the Geotiffs are not > adjacent, there's a little group for each of about 500 cities, but > strictly one of them at a time is shown, so I'm not sure how much > that's going to help. If there's something computationally intensive > going on at the server's end, that's going to be a drag in any case. > > Going through the Geoserver on Steroids presentation now, thanks! > > Max > > > On 17 March 2016 at 11:16, Ian Turton <[email protected]> wrote: >> Those sound like small tiffs so you would be better combining them >> together >> to avoid opening too many files at a time. Have a look at gdalbuildvrt to >> make a virtual raster catalogue that you can then convert into a tiled >> compressed geotiff that is in your main output projection. >> >> Have a read through the annual "GeoServer on Steroids" presentation >> (http://www.slideshare.net/geosolutions/geoserver-on-steroids-foss4g-2015 >> ) >> for more ideas.# >> >> Ian >> >> On 17 March 2016 at 09:51, Max <[email protected]> wrote: >>> >>> I have a large number - more than 60 thousand - of relatively small >>> Geotiffs, usually from 2 to 12 Mb. I have a web client that uses >>> Leaflet to view them, but things are quite slow even inside our own >>> network. I have a hunch that both Geoserver and these Geotiffs are not >>> configured in the best way possible. >>> >>> The tiffs were generated from a bunch of EPSG:3035 Ascii grid files >>> using this command: >>> >>> for f in *.rsl; do gdal_translate -a_srs EPSG:3035 -co "TILED=YES" >>> -co "BLOCKXSIZE=512" -co "BLOCKYSIZE=512" -co "COMPRESS=DEFLATE" -co >>> "ZLEVEL=9" -co "BIGTIFF=YES" $f $f.tif; done >>> >>> Then I wrote a Python script that created a store and a layer for each >>> of the tiffs. >>> >>> Here is the output from gdalinfo for a typical one: >>> >>> Driver: GTiff/GeoTIFF >>> Files: se502c_R9_C1_3_T_2531_sec.rsl.tif >>> Size is 3396, 2271 >>> Coordinate System is: >>> PROJCS["ETRS89 / LAEA Europe", >>> GEOGCS["ETRS89", >>> DATUM["European_Terrestrial_Reference_System_1989", >>> SPHEROID["GRS 1980",6378137,298.2572221010002, >>> AUTHORITY["EPSG","7019"]], >>> TOWGS84[0,0,0,0,0,0,0], >>> AUTHORITY["EPSG","6258"]], >>> PRIMEM["Greenwich",0], >>> UNIT["degree",0.0174532925199433], >>> AUTHORITY["EPSG","4258"]], >>> PROJECTION["Lambert_Azimuthal_Equal_Area"], >>> PARAMETER["latitude_of_center",52], >>> PARAMETER["longitude_of_center",10], >>> PARAMETER["false_easting",4321000], >>> PARAMETER["false_northing",3210000], >>> >>> As you can see they are EPSG:3035. They should be internally tiled. In >>> the Coordinate Reference Systems for Geoserver, it says that both >>> Native SRS and Declared SRS are EPSG:3035, and that the handling >>> should be to reproject native to declared. >>> >>> My web client overlays these Geotiffs on a standard OpenStreetMap >>> layer in Web Mercator. All the geotiffs we have tried appear >>> correctly, so I guess reprojection is still happening at some stage. >>> >>> My gut feeling is that I might gain some speed by reprojecting the >>> original Geotiffs to Web Mercator, or at least changing the declared >>> SRS to Web Mercator in Geoserver. Not keen on it, it's going to take >>> days. What other properties of the Geotiffs could I tinker with? >>> >>> Would caching with GeoWebCache help, given the file size? >>> >>> Bonus question - if a Geotiff is changed, by reprojecting or retiling >>> or what have you, does the Geoserver layer associated with it get >>> invalidated? It took days to create them all. >>> >>> I'd really appreciate your feeback. >>> >>> Thanks in advance, >>> Max >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Transform Data into Opportunity. >>> Accelerate data analysis in your applications with >>> Intel Data Analytics Acceleration Library. >>> Click to learn more. >>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 >>> _______________________________________________ >>> Geoserver-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/geoserver-users >> >> >> >> >> -- >> Ian Turton > > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 > _______________________________________________ > Geoserver-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > > > > > -- > > == > > GeoServer Professional Services from the experts! Visit > > http://goo.gl/it488V for more information. > > == > > > > Ing. Andrea Aime > > @geowolf > > Technical Lead > > > > GeoSolutions S.A.S. > Via di Montramito 3/A > 55054 Massarosa (LU) > > phone: +39 0584 962313 > > fax: +39 0584 1660272 > > mob: +39 339 8844549 > > > > http://www.geo-solutions.it > > http://twitter.com/geosolutions_it > > > > AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 > > Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i > file/s allegato/i sono da considerarsi strettamente riservate. Il loro > utilizzo è consentito esclusivamente al destinatario del messaggio, per le > finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio > senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia > via e-mail e di procedere alla distruzione del messaggio stesso, > cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo > anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per > finalità diverse, costituisce comportamento contrario ai principi dettati > dal D.Lgs. 196/2003. > > > > The information in this message and/or attachments, is intended solely for > the attention and use of the named addressee(s) and may be confidential or > proprietary in nature or covered by the provisions of privacy act > (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection > Code).Any use not in accord with its purpose, any disclosure, reproduction, > copying, distribution, or either dissemination, either whole or partial, is > strictly forbidden except previous formal approval of the named > addressee(s). If you are not the intended recipient, please contact > immediately the sender by telephone, fax or e-mail and delete the > information in this message that has been received in error. The sender does > not give any warranty or accept liability as the content, accuracy or > completeness of sent messages and accepts no responsibility for changes > made after they were sent or for other risks which arise as a result of > e-mail transmission, viruses, etc. > > > > ------------------------------------------------------- ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
