Driver: GTiff/GeoTIFF
Files: p_731_a.tif
Size is 4145, 3189
Coordinate System is:
PROJCS["WGS 84 / UTM zone 54S",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",141],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",10000000],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["Easting",EAST],
AXIS["Northing",NORTH],
AUTHORITY["EPSG","32754"]]
Origin = (271914.159999999974389,6123214.030000000260770)
Pixel Size = (1.000000000000000,-1.000000000000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=PACKBITS
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 271914.160, 6123214.030) (138d30' 1.82"E, 35d 0'28.82"S)
Lower Left ( 271914.160, 6120025.030) (138d29'58.67"E, 35d 2'12.24"S)
Upper Right ( 276059.160, 6123214.030) (138d32'45.20"E, 35d 0'32.15"S)
Lower Right ( 276059.160, 6120025.030) (138d32'42.11"E, 35d 2'15.58"S)
Center ( 273986.660, 6121619.530) (138d31'21.95"E, 35d 1'22.21"S)
Band 1 Block=416x320 Type=Byte, ColorInterp=Cyan
Band 2 Block=416x320 Type=Byte, ColorInterp=Magenta
Band 3 Block=416x320 Type=Byte, ColorInterp=Yellow
On Wednesday, September 12, 2018 at 4:48:14 PM UTC+2, Rémi wrote:
>
> The color space is l-alpha-beta but I don't think it's related to this
> strange behavior.
>
> I suspect that your images are in a non-streamable format. Meaning, you
> can't read them block by block, instead you are forced to read the entire
> file. Can you paste a gdalinfo of 1 of your file (feel free to modify
> coordinates and stuff) it's just to check the image file format.
> If I'm right, you can try to convert all your files in GeoTiff format
> (using otbcli_Convert with an output file extension equal to ".tif", then
> run the Mosaic application over the fresh .tif files and it will work fine
>
> Rémi
>
> Le mercredi 12 septembre 2018 16:32:43 UTC+2, Gus a écrit :
>>
>> Trying to reduce the RAM value seems to have no effect on memory
>> footprint. If I set it to 1, it grows a lot at first, then the process
>> seems to progress very slowly. If I set it to 10, it gows a lot at first,
>> then it does progress increasingly ocuppying memory. It seems to be
>> independent of your colour space transform, whatever method you choose,
>> band per band or (I guess you use cieLab?). I'm testing the no
>> harmonization options that you told (that I had already tried in windows).
>> It's writing the tiff and seems to be ok but, although it probably will
>> finish, it's taking a (while slower) progressively increasing amount of ram.
>> I tested ram parameter in otbcli_HaralickTextureExtraction and it seems
>> to correctly make use of chunks within parameter.
>> Just in case, I'm using imagery of this kind, 8bit unsigned integer where
>> I use 0 as no value (it worked ok for small sets).
>>
>>
>> On Wednesday, September 12, 2018 at 2:07:45 PM UTC+2, Rémi wrote:
>>>
>>> From this log, it seems like the problem occurs during statistics
>>> computation.
>>>
>>> To be sure of this, can you please check that the following commands
>>> work fine:
>>>
>>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -out
>>> /media/gus/Gus/seagrass/processed_data/r2_simple_noharmo.tif uint8 -ram
>>> 128
>>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -comp.feather
>>> large -out /media/gus/Gus/seagrass/processed_data/r2_feather_noharmo.tif
>>> uint8 -tmpdir /media/gus/Gus/seagrass/processed_data/temp -ram 128
>>>
>>> And can you please tell me if these commands are successful or not:
>>>
>>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -harmo.method
>>> band -harmo.cost musig -out
>>> /media/gus/Gus/seagrass/processed_data/r2_harmo-band.tif uint8 -ram 128
>>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -harmo.method
>>> rgb -harmo.cost musig -out
>>> /media/gus/Gus/seagrass/processed_data/r2_harmo-rgb.tif uint8 -ram 128
>>>
>>> Now, you can try to set a very small RAM value and tell me what's
>>> happening. Set -ram 1 for instance...
>>>
>>> Le mercredi 12 septembre 2018 11:42:19 UTC+2, Gus a écrit :
>>>>
>>>> I tried again last night. OTB 6.6.0 packaged linux version.
>>>> Command line: ./otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/*
>>>> -comp.feather large -harmo.method rgb -harmo.cost musig -out
>>>> /media/gus/Gus/seagrass/processed_data/r2.tif uint8 -tmpdir
>>>> /media/gus/Gus/seagrass/processed_data/temp -ram 128 -progress 1
>>>>
>>>> Last lines of output:
>>>> 2018-09-12 03:29:30 (INFO): Computing statistics in a decorrelated
>>>> colors space suitable for true-colors
>>>> 2018-09-12 03:29:44 (INFO): Estimated memory for full processing:
>>>> 3.15661e+06MB (avail.: 128 MB), optimal image partitioning: 24662 blocks
>>>> 2018-09-12 03:29:44 (INFO): Estimation will be performed in 18639
>>>> blocks of 832x320 pixels
>>>>
>>>> Anyway, after start, used RAM begins to slowly increase until it
>>>> occupied my swap space and froze the computer during the night.
>>>>
>>>> [image: Screenshot at 2018-09-12 03-30-10.png]
>>>>
>>>>
>>>>
>>>> On Sunday, September 9, 2018 at 5:23:19 PM UTC+2, Rémi wrote:
>>>>>
>>>>> Hello Gus,
>>>>>
>>>>> To investigate a bit I would like to know if you can copy/paste:
>>>>> -your command line
>>>>> -the full log
>>>>> (In a text file, or on pastebin)
>>>>>
>>>>> What version of OTB, and what OS do you have?
>>>>> Thanks,
>>>>>
>>>>> Rémi
>>>>>
>>>>> Le samedi 8 septembre 2018 18:07:56 UTC+2, Gus a écrit :
>>>>>>
>>>>>> Thanks, Rémi
>>>>>>
>>>>>> Your method works fairly well, the results are really good and
>>>>>> visually pleasant. In fact, I was able to use you app (from gui) for
>>>>>> small
>>>>>> subsets of my data, which is a set of aerial pictures of the sea surface
>>>>>> from which the sunglint has been removed.
>>>>>>
>>>>>> [image: 2dc4d9d7-25da-4254-9cb5-31536801f461.jpg][image:
>>>>>> 54ed0802-e492-438f-9f4d-8dea3593a2b6.jpg]
>>>>>>
>>>>>> But, when I feed your program with the complete set (about 2500
>>>>>> pictures, 30 Mb each approximately, 8 bit colour) it fails. Last thing I
>>>>>> can check is that the temporary distance maps have been correctly
>>>>>> written,
>>>>>> and that's the last console output. while if Iskip the large feathering
>>>>>> step, it seems to work ok, as in next image using slim blending mode,
>>>>>> but
>>>>>> as for large blending, it fails...
>>>>>>
>>>>>> [image: sea_1PNG.PNG]
>>>>>>
>>>>>> On Saturday, September 8, 2018 at 10:57:53 AM UTC+2, Rémi wrote:
>>>>>>>
>>>>>>> Hello Gus,
>>>>>>>
>>>>>>> The RAM parameter deals only with the memory footprint during mosaic
>>>>>>> compositing and statistics computation, both streamable pipelines (no
>>>>>>> limitation on images size).
>>>>>>> However, the distance map calculation (which is a temporary file,
>>>>>>> then used as an input of the streamed mosaicing compositing pipeline,
>>>>>>> just
>>>>>>> for feathering) is not a streamable process. Meaning that, for each
>>>>>>> input
>>>>>>> image to mosaic, the distance map will theoretically require the full
>>>>>>> image
>>>>>>> in memory to be processed. That's why there is the distancemap.sr
>>>>>>> (sampling ratio) parameter. This enables the subsampling of the input
>>>>>>> image, to compute the distance map with a smaller footprint. See
>>>>>>> README.md,
>>>>>>> section "Performance tuning: Distance map images sampling ratio".
>>>>>>> This is a current limitation of the application, I will add this
>>>>>>> explicitly in the application description.
>>>>>>>
>>>>>>> Your problem could also come from the fact that your input images
>>>>>>> have not the same projection reference (and leading to a near infinite
>>>>>>> image, e.g. if you mix degrees and meters units, or with wrong
>>>>>>> coordinate
>>>>>>> reference system), check this.
>>>>>>>
>>>>>>> Hope that helps,
>>>>>>>
>>>>>>> Rémi
>>>>>>>
>>>>>>> Le vendredi 7 septembre 2018 01:44:23 UTC+2, Gus a écrit :
>>>>>>>>
>>>>>>>>
>>>>>>>> Hello everyone.
>>>>>>>>
>>>>>>>> I don't clearly understand the process followed by the remote
>>>>>>>> module "mosaic". The memory occupied during some phase after distance
>>>>>>>> maps
>>>>>>>> calculation for large blending with lab color space harmonization
>>>>>>>> grows a
>>>>>>>> lot, so I obtain a huge page file and my computer reboots. I checked
>>>>>>>> the
>>>>>>>> paper and peeked into the code a bit, but I'm not familiar with the
>>>>>>>> libraries. Is there any way of estimating the ram needed for the
>>>>>>>> process?
>>>>>>>> Is there a way of overcoming this issue? (probably I will process it
>>>>>>>> in
>>>>>>>> chunks anyway but that would simplify things quite a lot).
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> Gus
>>>>>>>>
>>>>>>>
--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
