Re: [Geotools-gt2-users] [Geotools-devel] WCS Image mosaicking problem and image utils question
Thanks a lot for these valuable tips :-) I haven't tried yet as I have been busy by other stuff - porting my code to v2.3x - but I give you some feedback soon. I am really looking forward experiencing real performance with your coverage tools! I have no doubts about it ;-) Olivier 2006/11/15, Simone Giannecchini [EMAIL PROTECTED]: On 11/15/06, Olivier Doucet [EMAIL PROTECTED] wrote: Ciao Simone, 2006/11/14, Simone Giannecchini [EMAIL PROTECTED]: Ciao Olivier, you need to specify a wild card, something like -s /usr/simone/work/data/ -w *.tiff -name imm Thanks, it works now! Any doubts? :-) Really, which versin of geotools are you playing with? I hope 2.3.x... if you want the mosaic builder to be able to find the correct files you want to index. Aside from this, if you have one single tiff which is not too big (up to 300-400 mb) I recommend you use the OverviewsEmbedder which just embeds overviews inside the same file. I tried the mosaic and pyramid utils on one small file (230 Mb) but I'll eventually use several files of around 1.2 Gb each, so it sounds like the pyramid is a better idea. As I understand it, the overview isn't divided into tiles, is that correct? Which reader should be used to display it? The OverviewsEmbedder just embeds overviews inside tiff and geotiff, You need to use the WorldImageReader or the GeoTiffReader to read this single files afterwards. Anyway, if you wish to build a pyramid starting with a few very big files I suggest these steps: 1Build a mosaic for the input files. 2Use the CoverageTiler and tile the mosaic you just created. This way you will split the mosaic into smaller tiles that will perform much faster than a couple of huge coverages. 3Rebuild the mosaic for the first level 4Build the successive levels. Smaller tiles (1-1.5 mb) are very fast inside the GeoServer since we use multiple threads (actually I think it is about time to make the number of threads configurable) to load them. Also concerning the PyramidBuilder options, I have a few questions : -t (--tiled_dimension) Is there a recommended/default value? How to estimate the right dimension? There is no magic number. More or less make tiles as big as needed to have each one of them around 1mb. To estimate the size more or less do as follows num_bands*tile_width*tile_height*size_of_data_type/8 For an rgb byte tile of 512x512 you get 3*512*512=768k which is not bad. I usually use 800*800 though (I dunno why :-) ) Can this value be different from one level of the pyramid to another? Sure thing. It only has a meaning internal to each level. It basically controls the number of tiles you get. There is only one thing that must not change between two levels, the envelope covered. -c (--cache_size) Is it just a parameter to optimize the use of the resources when building the pyramid or does this affect the final product of the PyramidBuilder? Could you give an example of value here. This element controls the TileCache size used for building the pyramid. The bigger it is the faster the tool will perform because it will cache successive tiles in memory. Use at least 64 mb, I usually use 400 mb which big raster. Finally, I'd need more details about the properties file to provide at the last step of pyramid building. Your example raises two questions: # #Mon Aug 21 22:23:27 CEST 2006 #name of the coverage Name=ikonos #different resolution levels available in each mosaic Levels=1.2218682749859724E-5,9.220132503102996E-6 2.4428817977683634E-5,1.844026500620314E-5 4.8840552865873626E-5,3.686350299024973E-5 9.781791400307775E-5,7.372700598049946E-5 1.956358280061555E-4,1.4786360643866836E-4 3.901787184256844E-4,2.9572721287731037E-4 #where all the levels reside LevelsDirs=0 2 4 8 16 32 #number of levels availaible LevelsNum=5 #envelope for this pyramid Envelope2D=13.398228477973406 ,43.591366397808976 13.537912459169803,43.67121274528585 1- How do you get the different resolution levels in the 'Levels' section? You can get the resolution levels out of the single levels' properties file. 2- LevelsDirs=0 2 4 8 16 32 suggests that we have a 6 level pyramid, however LevelsNum=5... Is it level 0 that doesn't count? That should be an error! I will check asap. The level number should match the number of availaible dirs. Thank you for your help. Olivier No problem man, thx for your feedback and keep me posted on how things proceed. Any suggestions on how to improve the tools is highly appreciated (donations are even more appreciated :-) ) Simone. Simone. On 11/14/06, Olivier Doucet [EMAIL PROTECTED] wrote: Hi Simone, I am also trying to use the image mosaic pyramid tools, but without success so far... If I understand correctly, the build a pyramid out of a single image.tif
Re: [Geotools-gt2-users] [Geotools-devel] WCS Image mosaicking problem and image utils question
Ciao Simone,2006/11/14, Simone Giannecchini [EMAIL PROTECTED]:
Ciao Olivier,you need to specify a wild card, something like -s /usr/simone/work/data/-w *.tiff -name immThanks, it works now!
if you want the mosaic builder to be able to find the correct filesyou want to index.Aside from this, if you have one single tiff which is not too big (upto 300-400 mb) I recommend you use the OverviewsEmbedder which just
embeds overviews inside the same file.I tried the mosaic and pyramid utils on one small file (230 Mb) but I'll eventually use several files of around 1.2 Gb each, so it sounds like the pyramid is a better idea.
As I understand it, the overview isn't divided into tiles, is that correct? Which reader should be used to display it?Also concerning the PyramidBuilder options, I have a few questions :-t (--tiled_dimension)
Is there a recommended/default value? How to estimate the right dimension? Can this value be different from one level of the pyramid to another?-c (--cache_size)Is it just a parameter to optimize the use of the resources when building the pyramid or does this affect the final product of the PyramidBuilder?
Could you give an example of value here.Finally, I'd need more details about the properties file to provide at the last step of pyramid building.Your example raises two questions: #
#Mon Aug 21 22:23:27 CEST 2006 #name of the coverage Name=ikonos #different resolution levels available in each mosaic Levels=1.2218682749859724E-5,9.220132503102996E-6
2.4428817977683634E-5,1.844026500620314E-54.8840552865873626E-5,3.686350299024973E-59.781791400307775E-5,7.372700598049946E-51.956358280061555E-4,1.4786360643866836E-43.901787184256844E-4,2.9572721287731037E-4
#where all the levels reside LevelsDirs=0 2 4 8 16 32 #number of levels availaible LevelsNum=5 #envelope for this pyramid Envelope2D=13.398228477973406
,43.59136639780897613.537912459169803,43.671212745285851- How do you get the different resolution levels in the 'Levels' section?2- LevelsDirs=0 2 4 8 16 32 suggests that we have a 6 level pyramid, however LevelsNum=5... Is it level 0 that doesn't count?
Thank you for your help.OlivierSimone.On 11/14/06, Olivier Doucet
[EMAIL PROTECTED] wrote: Hi Simone, I am also trying to use the image mosaic pyramid tools, but without success so far... If I understand correctly, the build a pyramid out of a single
image.tif file, I first have to use the MosaicIndexBuilder. The folder I give as a parameter contains only 3 files: - image01.tif - image01.tfw - image01.prj A strange thing is,
at line 238 in MosaicIndexBuilder.java The index of the iteration for (int i = 0; i numFiles; i++) { is used line 336 for testing whether it is the first time if (i == 0) { we
have to treat an image. So what if the folder contains non image files and and one of those is iterated first? Then globEnvelope is null when accessed in the else section. Are there any examples on how to use these utils? You refer to geotools
examples but I couldn't find them... Thank you. Olivier 2006/10/20, Simone Giannecchini [EMAIL PROTECTED]
: Ciao Sandor, sorry for not being able to get to you earleri, but I am constantly 3 days behind schedule with emails. Read below. On 10/18/06, Sándor Farkas
[EMAIL PROTECTED] wrote: Hi Simone, Im Sándor from Hungary, I have some question about imaging and WCS image
plugins. First I asked Alessio and He gave me your address as You are the imaging expert. I would like to publicate orthophoto GEOTIFF-s with WCS and I ask you to
help me find the best way for quality/performance optimum. I have 112 pcs (orthophoto) GEOTIFF images , about 107 Mb of size each. I made them tiled whith
C:\Progra~1\FWTools1.0.5\bin\tiffcp.exe -w 256 -l 256 65-324-111.tif 65-324-111_t.tif ...
k.and used c:\Progra~1\FWTools1.0.5\bin\gdaladdo.exe -r average d:\65-322- 431_t.tif 2 4 8 16 32 64 ...
k. then I used c:\Progra~1\FWTools1.0.5\bin\gdaltindex.exe Budaors_tiled.shp *.tif
To create the mosaic the best thing to do is using the MosaicIndexBuilder tool you can find at the link below. Otherwise you would have to manually create the accompanying property file. Iwould
lik to remarck once again that all the images must have the same resolution. This constraint could be relaxes but right now I have time allocated for this taks (unless someone wants it badly and pays for it
:-) ). After then, when I made the Image mosaicking coverage and watched it in UDIG, I saw that all images are malformed which are at the boundary, but
seems good in the middle, where they have adjacent images. I attached some pictures to the mail. What could I do with this?Well, I fixed a bug in the ImageMosaic a couple of days ago, which
version of geotools,geoserver are you runing?. For testing I inserted a single image (same as in the mosaicking, with the same
Re: [Geotools-gt2-users] [Geotools-devel] WCS Image mosaicking problem and image utils question
Ciao Olivier,
you need to specify a wild card, something like
-s /usr/simone/work/data/ -w *.tiff -name imm
if you want the mosaic builder to be able to find the correct files
you want to index.
Aside from this, if you have one single tiff which is not too big (up
to 300-400 mb) I recommend you use the OverviewsEmbedder which just
embeds overviews inside the same file.
Simone.
On 11/14/06, Olivier Doucet [EMAIL PROTECTED] wrote:
Hi Simone,
I am also trying to use the image mosaic pyramid tools, but without
success so far...
If I understand correctly, the build a pyramid out of a single image.tif
file, I first have to use the MosaicIndexBuilder.
The folder I give as a parameter contains only 3 files:
- image01.tif
- image01.tfw
- image01.prj
A strange thing is,
at line 238 in MosaicIndexBuilder.java
The index of the iteration for (int i = 0; i numFiles; i++) {
is used line 336 for testing whether it is the first time if (i == 0) { we
have to treat an image.
So what if the folder contains non image files and and one of those is
iterated first?
Then globEnvelope is null when accessed in the else section.
Are there any examples on how to use these utils? You refer to geotools
examples but I couldn't find them...
Thank you.
Olivier
2006/10/20, Simone Giannecchini [EMAIL PROTECTED]:
Ciao Sandor,
sorry for not being able to get to you earleri, but I am constantly 3
days behind schedule with emails.
Read below.
On 10/18/06, Sándor Farkas [EMAIL PROTECTED] wrote:
Hi Simone,
Im Sándor from Hungary, I have some question about imaging and WCS image
plugins.
First I asked Alessio and He gave me your address as You are the imaging
expert.
I would like to publicate orthophoto GEOTIFF-s with WCS and I ask you to
help me find the best way for quality/performance optimum.
I have 112 pcs (orthophoto) GEOTIFF images , about 107 Mb of size each.
I
made them tiled whith
C:\Progra~1\FWTools1.0.5\bin\tiffcp.exe -w 256 -l 256
65-324-111.tif
65-324-111_t.tif
...
k.
and used
c:\Progra~1\FWTools1.0.5\bin\gdaladdo.exe -r average
d:\65-322-
431_t.tif 2 4 8 16 32 64
...
k.
then I used
c:\Progra~1\FWTools1.0.5\bin\gdaltindex.exe
Budaors_tiled.shp *.tif
To create the mosaic the best thing to do is using the
MosaicIndexBuilder tool you can find at the link below. Otherwise you
would have to manually create the accompanying property file. Iwould
lik to remarck once again that all the images must have the same
resolution. This constraint could be relaxes but right now I have time
allocated for this taks (unless someone wants it badly and pays for it
:-) ).
After then, when I made the Image mosaicking coverage and watched it in
UDIG, I saw that all images are malformed which are at the boundary, but
seems good in the middle, where they have adjacent images. I attached
some
pictures to the mail.
What could I do with this?
Well, I fixed a bug in the ImageMosaic a couple of days ago, which
version of geotools,geoserver are you runing?.
For testing I inserted a single image (same as in the mosaicking, with
the
same projection) and it looks good.
Then definitely the you are running into the bug I fixed lately.
Furthermore I realized when I used Image Mosaicking, I didn't see
gdaladdo-d
levels.
If you attach the properties files the ImageMosaic plugin uses it to
exploit embedded overviews.
I know I should find time to write a guide for using the tools, but
for the moment I have no spare time to spend on this.
I would like to try WCS Image pyramidal plugin.
What is the difference between pyramidal images and gdaladdo -d images?
The ImagePyramid plugin is thought for being used with multiple mosaic
at different levels of resolutions.
What file should I give in WCS Image pyramidal plugin (shapefile like
mosaicking plugin or it supports only one file)?
Please, have a look at the example data provided with geotools. Anyway
the informations you need to provide are roughly explained in the
javadoc of the Imagepyramidreader, I am reporting them here for
simplicity:
This reader is repsonsible for providing access to a pyramid of
mosaics of georeferenced coverages that are read directly through
imageio readers, like tiff, pngs, etc...
Specifically this plugin relies on the image mosaic plugin to handle
each single level of resolutions avaible, hence all the magic is done
inside the mosaic plugin.
For information on how to build a mosaic, please refere to the
ImageMosaicReader documentation.
If you are looking for information on how to create a pyramid, here
you go. The pyramid itself does no magic. All the magic is performed
by the single mosaic readers that are polled depending on the requeste
resolution levels. Therefore the first
