Hi Robert, I also did some tests with adjacent raster data (topographic maps, right now ~20GB) using the pyramid-plugin and also the mosaic store. I can only confirm what has been said so far: The mosaic store gives much better performance (A real performance test is still to be done, but smaller tests were made). What I did during gdal_translate is to use tiff compression (LZW, I have no idea if JPEG will be better, does someone has experience?) and in a second step I build overviews with gdaladdo. This enhances the performance significantly (I did also test without overviews).
@Andrea You mentioned a WMS-shootout using a 100GB mosaic file. Are the results documented somewhere? Thanks and cheers Torsten -------------------------------------------------------------------------- T-Systems International GmbH Torsten Drey Spezialist Geo-Consulting GeoServices/GIS Vorgebirgsstraße 49 D-53119 Bonn Fon: +49 (0)228 9841 3690 Fax: +49 (0)228 9841 3990 E-Mail: [email protected] Internet: http://www.t-systems.com T-Systems International GmbH Aufsichtsrat: René Obermann (Vorsitzender) Geschäftsführung: Reinhard Clemens (Vorsitzender), Dr. Ferri Abolhassan, Olaf Heyden, Georg Pepping, Klaus Werner Handelsregister: Amtsgericht Frankfurt am Main HRB 55933 Sitz der Gesellschaft: Frankfurt am Main WEEE-Reg.-Nr. DE87523644 Notice: This transmittal and/or attachments may be privileged or confidential. If you are not the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you. -----Ursprüngliche Nachricht----- Von: [email protected] [mailto:[email protected]] Gesendet: Mittwoch, 20. April 2011 16:31 An: [email protected] Betreff: Geoserver-users Digest, Vol 59, Issue 78 Send Geoserver-users mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/geoserver-users or, via email, send a message with subject or body 'help' to [email protected] You can reach the person managing the list at [email protected] When replying, please edit your Subject line so it is more specific than "Re: Contents of Geoserver-users digest..." Today's Topics: 1. Re: Publishing 1300 orthophotos with geoserver (Robert Buckley) 2. Re: Publishing 1300 orthophotos with geoserver (Rahkonen Jukka) ---------------------------------------------------------------------- Message: 1 Date: Wed, 20 Apr 2011 15:18:35 +0100 (BST) From: Robert Buckley <[email protected]> Subject: Re: [Geoserver-users] Publishing 1300 orthophotos with geoserver To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" @ Simone My trial run (for 10 photos) will be this.. 1. Desktop: fwtools > Gdal_merge - make one photo from the 10 2. Desktop: fwtools > gdal_translate - Transform from ETRS89 to WGS84 3. Upload file(s) to my ubuntu server 4. Configure and serve in geoserver Any tips welcome. Thanks, Rob ________________________________ Von: Simone Giannecchini <[email protected]> An: Robert Buckley <[email protected]> CC: [email protected] Gesendet: Mittwoch, den 20. April 2011, 15:41:23 Uhr Betreff: Re: [Geoserver-users] Publishing 1300 orthophotos with geoserver Ciao Robert, hold on a sec. When you say uploading, you mean configuring GeoServer to serve data or upload to a remote server and then configure? Regards, Simone Giannecchini ------------------------------------------------------- Ing. Simone Giannecchini GeoSolutions S.A.S. Founder Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 mob: +39 333 8128928 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/simonegiannecchini http://twitter.com/simogeo ------------------------------------------------------- On Wed, Apr 20, 2011 at 3:36 PM, Robert Buckley <[email protected]> wrote: > Ok, > > I am just not used to uploading such quantities of data and didn?t > know if that was something I could avoid or not. If that is normal.. then so > be it! > > It?s a lot of work but if geoserver can serve it quickly and > efficiently then it?ll be worth it. > > I?ll start with a few tests with maybe 10 or 20 photos and see how it > all works. > > thanks for the help, > > Robert > > > > ________________________________ > Von: Andrea Aime <[email protected]> > An: Robert Buckley <[email protected]> > CC: [email protected] > Gesendet: Mittwoch, den 20. April 2011, 15:11:06 Uhr > Betreff: Re: [Geoserver-users] Publishing 1300 orthophotos with > geoserver > > On Wed, Apr 20, 2011 at 9:28 AM, Robert Buckley > <[email protected]> > wrote: >> >> Hi, >> >> Thanks for the reply, >> >> The orthophotos make up a mosaic of my regional planning area . (see >> Attatchment) Nothing more than a tiled surface. >> > > Ah ok, then build a single mosaic out of them. In the WMS shootout we > had good result with exactly one mosaic whose size was 100GB. We did > not try out the pyramid > >> >> So you are suggesting that I upload 1300 separate files into geoserver? >> 1300 x 76 mb? (100gb) That alone would take about 2 weeks > > Hmmm... regardless of the form chosen for the data you have to get it > on the server no? > What makes you think building a pyramid will make it significantly smaller? > In any case, uploading 100GB is indeed a lot, if you play with such > data amounts better send a disk with the data on it to the ISP or go > there directly with it? > Cheers > Andrea > -- > ------------------------------------------------------- > Ing. Andrea Aime > GeoSolutions S.A.S. > Tech lead > > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > > phone: +39 0584 962313 > fax: +39 0584 962313 > mob: +39 333 8128928 > > http://www.geo-solutions.it > http://geo-solutions.blogspot.com/ > http://www.youtube.com/user/GeoSolutionsIT > http://www.linkedin.com/in/andreaaime > http://twitter.com/geowolf > > ------------------------------------------------------- > > ---------------------------------------------------------------------- > -------- Benefiting from Server Virtualization: Beyond Initial > Workload Consolidation -- Increasing the use of server virtualization > is a top priority.Virtualization can reduce costs, simplify > management, and improve application availability and disaster > protection. Learn more about boosting the value of server > virtualization. http://p.sf.net/sfu/vmware-sfdev2dev > _______________________________________________ > Geoserver-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ Message: 2 Date: Wed, 20 Apr 2011 17:31:19 +0300 From: "Rahkonen Jukka" <[email protected]> Subject: Re: [Geoserver-users] Publishing 1300 orthophotos with geoserver To: "Robert Buckley" <[email protected]>, <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" Hi, If you want to save disk space you might want to experiment with jpeg compressed tiff files. Read "About JPEG compression of RGB images" from http://gdal.org/frmt_gtiff.html Jpeg compression saves easily 90 percent of the space needed. Naturally if you will need lossless quality for some use it will not be an option. And I do not know if Geoserver even likes such tiff files because I am running WMS from Mapserver but you were asking for tips. -Jukka Rahkonen- ________________________________ Robert Buckley wrote: @ Simone My trial run (for 10 photos) will be this.. 1. Desktop: fwtools > Gdal_merge - make one photo from the 10 2. Desktop: fwtools > gdal_translate - Transform from ETRS89 to WGS84 3. Upload file(s) to my ubuntu server 4. Configure and serve in geoserver Any tips welcome. Thanks, Rob ________________________________ Von: Simone Giannecchini <[email protected]> An: Robert Buckley <[email protected]> CC: [email protected] Gesendet: Mittwoch, den 20. April 2011, 15:41:23 Uhr Betreff: Re: [Geoserver-users] Publishing 1300 orthophotos with geoserver Ciao Robert, hold on a sec. When you say uploading, you mean configuring GeoServer to serve data or upload to a remote server and then configure? Regards, Simone Giannecchini ------------------------------------------------------- Ing. Simone Giannecchini GeoSolutions S.A.S. Founder Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 mob: +39 333 8128928 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/simonegiannecchini http://twitter.com/simogeo ------------------------------------------------------- On Wed, Apr 20, 2011 at 3:36 PM, Robert Buckley <[email protected]> wrote: > Ok, > > I am just not used to uploading such quantities of data and didn?t know if > that was something I could avoid or not. If that is normal.. then so be it! > > It?s a lot of work but if geoserver can serve it quickly and efficiently > then it?ll be worth it. > > I?ll start with a few tests with maybe 10 or 20 photos and see how it all > works. > > thanks for the help, > > Robert > > > > ________________________________ > Von: Andrea Aime <[email protected]> > An: Robert Buckley <[email protected]> > CC: [email protected] > Gesendet: Mittwoch, den 20. April 2011, 15:11:06 Uhr > Betreff: Re: [Geoserver-users] Publishing 1300 orthophotos with geoserver > > On Wed, Apr 20, 2011 at 9:28 AM, Robert Buckley <[email protected]> > wrote: >> >> Hi, >> >> Thanks for the reply, >> >> The orthophotos make up a mosaic of my regional planning area . (see >> Attatchment) Nothing more than a tiled surface. >> > > Ah ok, then build a single mosaic out of them. In the WMS shootout we had > good result with exactly > one mosaic whose size was 100GB. We did not try out the pyramid > >> >> So you are suggesting that I upload 1300 separate files into geoserver? >> 1300 x 76 mb? (100gb) That alone would take about 2 weeks > > Hmmm... regardless of the form chosen for the data you have to get it on the > server no? > What makes you think building a pyramid will make it significantly smaller? > In any case, uploading 100GB is indeed a lot, if you play with such data > amounts better > send a disk with the data on it to the ISP or go there directly with it? > Cheers > Andrea > -- > ------------------------------------------------------- > Ing. Andrea Aime > GeoSolutions S.A.S. > Tech lead > > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > > phone: +39 0584 962313 > fax: +39 0584 962313 > mob: +39 333 8128928 > > http://www.geo-solutions.it > http://geo-solutions.blogspot.com/ > http://www.youtube.com/user/GeoSolutionsIT > http://www.linkedin.com/in/andreaaime > http://twitter.com/geowolf > > ------------------------------------------------------- > > ------------------------------------------------------------------------------ > Benefiting from Server Virtualization: Beyond Initial Workload > Consolidation -- Increasing the use of server virtualization is a top > priority.Virtualization can reduce costs, simplify management, and improve > application availability and disaster protection. Learn more about boosting > the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev > _______________________________________________ > Geoserver-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ ------------------------------------------------------------------------------ Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev ------------------------------ _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users End of Geoserver-users Digest, Vol 59, Issue 78 *********************************************** ------------------------------------------------------------------------------ Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
