Hi,
If you want to keep all 60000 layers which is not optimal as Andrea just wrote,
the most obvious thing to do is to create overviews for the images.
gdaladdo –r average 2 4 8 16 32 64 image.tif --config COMPRESS_OVERVIEW DEFLATE
It is worth making a test before reprojecting images physically into EPSG:3857.
On-the-fly reprojecting is not necessarily so slow.
I saw that you have used the BIGTIFF=YES option. It is not needed for your
small images and it probably makes reading the images a little bit slower but I
guess that the difference is marginal. Tile size 256x256 could be a bit faster
than 512x512 but that’t also marginal.
-Jukka Rahkonen-
basiliosz wrote:
As 60000 layers. They are not contiguous, are completely separate and are
viewed strictly one at a time per user.
The scenario: user selects a city and a couple parameters, the JS code figures
out what layer to request, then submits the request for that layer to
Geoserver. Eventually the layer is added to the map.
Max
On 17 Mar 2016 13:44, "Andrea Aime"
<[email protected]<mailto:[email protected]>> wrote:
Hi,
just to be sure, how have you published them?
As a single image mosaic or as 60000 separate layers?
Cheers
Andrea
On Thu, Mar 17, 2016 at 12:56 PM, Max
<[email protected]<mailto:[email protected]>> wrote:
Thanks - because of the origin of the data the Geotiffs are not
adjacent, there's a little group for each of about 500 cities, but
strictly one of them at a time is shown, so I'm not sure how much
that's going to help. If there's something computationally intensive
going on at the server's end, that's going to be a drag in any case.
Going through the Geoserver on Steroids presentation now, thanks!
Max
On 17 March 2016 at 11:16, Ian Turton
<[email protected]<mailto:[email protected]>> wrote:
> Those sound like small tiffs so you would be better combining them together
> to avoid opening too many files at a time. Have a look at gdalbuildvrt to
> make a virtual raster catalogue that you can then convert into a tiled
> compressed geotiff that is in your main output projection.
>
> Have a read through the annual "GeoServer on Steroids" presentation
> (http://www.slideshare.net/geosolutions/geoserver-on-steroids-foss4g-2015 )
> for more ideas.#
>
> Ian
>
> On 17 March 2016 at 09:51, Max
> <[email protected]<mailto:[email protected]>> wrote:
>>
>> I have a large number - more than 60 thousand - of relatively small
>> Geotiffs, usually from 2 to 12 Mb. I have a web client that uses
>> Leaflet to view them, but things are quite slow even inside our own
>> network. I have a hunch that both Geoserver and these Geotiffs are not
>> configured in the best way possible.
>>
>> The tiffs were generated from a bunch of EPSG:3035 Ascii grid files
>> using this command:
>>
>> for f in *.rsl; do gdal_translate -a_srs EPSG:3035 -co "TILED=YES"
>> -co "BLOCKXSIZE=512" -co "BLOCKYSIZE=512" -co "COMPRESS=DEFLATE" -co
>> "ZLEVEL=9" -co "BIGTIFF=YES" $f $f.tif; done
>>
>> Then I wrote a Python script that created a store and a layer for each
>> of the tiffs.
>>
>> Here is the output from gdalinfo for a typical one:
>>
>> Driver: GTiff/GeoTIFF
>> Files: se502c_R9_C1_3_T_2531_sec.rsl.tif
>> Size is 3396, 2271
>> Coordinate System is:
>> PROJCS["ETRS89 / LAEA Europe",
>> GEOGCS["ETRS89",
>> DATUM["European_Terrestrial_Reference_System_1989",
>> SPHEROID["GRS 1980",6378137,298.2572221010002,
>> AUTHORITY["EPSG","7019"]],
>> TOWGS84[0,0,0,0,0,0,0],
>> AUTHORITY["EPSG","6258"]],
>> PRIMEM["Greenwich",0],
>> UNIT["degree",0.0174532925199433],
>> AUTHORITY["EPSG","4258"]],
>> PROJECTION["Lambert_Azimuthal_Equal_Area"],
>> PARAMETER["latitude_of_center",52],
>> PARAMETER["longitude_of_center",10],
>> PARAMETER["false_easting",4321000],
>> PARAMETER["false_northing",3210000],
>>
>> As you can see they are EPSG:3035. They should be internally tiled. In
>> the Coordinate Reference Systems for Geoserver, it says that both
>> Native SRS and Declared SRS are EPSG:3035, and that the handling
>> should be to reproject native to declared.
>>
>> My web client overlays these Geotiffs on a standard OpenStreetMap
>> layer in Web Mercator. All the geotiffs we have tried appear
>> correctly, so I guess reprojection is still happening at some stage.
>>
>> My gut feeling is that I might gain some speed by reprojecting the
>> original Geotiffs to Web Mercator, or at least changing the declared
>> SRS to Web Mercator in Geoserver. Not keen on it, it's going to take
>> days. What other properties of the Geotiffs could I tinker with?
>>
>> Would caching with GeoWebCache help, given the file size?
>>
>> Bonus question - if a Geotiff is changed, by reprojecting or retiling
>> or what have you, does the Geoserver layer associated with it get
>> invalidated? It took days to create them all.
>>
>> I'd really appreciate your feeback.
>>
>> Thanks in advance,
>> Max
>>
>>
>> ------------------------------------------------------------------------------
>> Transform Data into Opportunity.
>> Accelerate data analysis in your applications with
>> Intel Data Analytics Acceleration Library.
>> Click to learn more.
>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>> _______________________________________________
>> Geoserver-users mailing list
>> [email protected]<mailto:[email protected]>
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
>
> --
> Ian Turton
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Geoserver-users mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313<tel:%2B39%200584%20962313>
fax: +39 0584 1660272<tel:%2B39%200584%201660272>
mob: +39 339 8844549<tel:%2B39%20%C2%A0339%208844549>
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
-------------------------------------------------------
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
