Driver: GTiff/GeoTIFF Files: p_731_a.tif Size is 4145, 3189 Coordinate System is: PROJCS["WGS 84 / UTM zone 54S", GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["WGS 84",6378137,298.257223563, AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]], PRIMEM["Greenwich",0, AUTHORITY["EPSG","8901"]], UNIT["degree",0.0174532925199433, AUTHORITY["EPSG","9122"]], AUTHORITY["EPSG","4326"]], PROJECTION["Transverse_Mercator"], PARAMETER["latitude_of_origin",0], PARAMETER["central_meridian",141], PARAMETER["scale_factor",0.9996], PARAMETER["false_easting",500000], PARAMETER["false_northing",10000000], UNIT["metre",1, AUTHORITY["EPSG","9001"]], AXIS["Easting",EAST], AXIS["Northing",NORTH], AUTHORITY["EPSG","32754"]] Origin = (271914.159999999974389,6123214.030000000260770) Pixel Size = (1.000000000000000,-1.000000000000000) Metadata: AREA_OR_POINT=Area Image Structure Metadata: COMPRESSION=PACKBITS INTERLEAVE=PIXEL Corner Coordinates: Upper Left ( 271914.160, 6123214.030) (138d30' 1.82"E, 35d 0'28.82"S) Lower Left ( 271914.160, 6120025.030) (138d29'58.67"E, 35d 2'12.24"S) Upper Right ( 276059.160, 6123214.030) (138d32'45.20"E, 35d 0'32.15"S) Lower Right ( 276059.160, 6120025.030) (138d32'42.11"E, 35d 2'15.58"S) Center ( 273986.660, 6121619.530) (138d31'21.95"E, 35d 1'22.21"S) Band 1 Block=416x320 Type=Byte, ColorInterp=Cyan Band 2 Block=416x320 Type=Byte, ColorInterp=Magenta Band 3 Block=416x320 Type=Byte, ColorInterp=Yellow
On Wednesday, September 12, 2018 at 4:48:14 PM UTC+2, Rémi wrote: > > The color space is l-alpha-beta but I don't think it's related to this > strange behavior. > > I suspect that your images are in a non-streamable format. Meaning, you > can't read them block by block, instead you are forced to read the entire > file. Can you paste a gdalinfo of 1 of your file (feel free to modify > coordinates and stuff) it's just to check the image file format. > If I'm right, you can try to convert all your files in GeoTiff format > (using otbcli_Convert with an output file extension equal to ".tif", then > run the Mosaic application over the fresh .tif files and it will work fine > > Rémi > > Le mercredi 12 septembre 2018 16:32:43 UTC+2, Gus a écrit : >> >> Trying to reduce the RAM value seems to have no effect on memory >> footprint. If I set it to 1, it grows a lot at first, then the process >> seems to progress very slowly. If I set it to 10, it gows a lot at first, >> then it does progress increasingly ocuppying memory. It seems to be >> independent of your colour space transform, whatever method you choose, >> band per band or (I guess you use cieLab?). I'm testing the no >> harmonization options that you told (that I had already tried in windows). >> It's writing the tiff and seems to be ok but, although it probably will >> finish, it's taking a (while slower) progressively increasing amount of ram. >> I tested ram parameter in otbcli_HaralickTextureExtraction and it seems >> to correctly make use of chunks within parameter. >> Just in case, I'm using imagery of this kind, 8bit unsigned integer where >> I use 0 as no value (it worked ok for small sets). >> >> >> On Wednesday, September 12, 2018 at 2:07:45 PM UTC+2, Rémi wrote: >>> >>> From this log, it seems like the problem occurs during statistics >>> computation. >>> >>> To be sure of this, can you please check that the following commands >>> work fine: >>> >>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -out >>> /media/gus/Gus/seagrass/processed_data/r2_simple_noharmo.tif uint8 -ram >>> 128 >>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -comp.feather >>> large -out /media/gus/Gus/seagrass/processed_data/r2_feather_noharmo.tif >>> uint8 -tmpdir /media/gus/Gus/seagrass/processed_data/temp -ram 128 >>> >>> And can you please tell me if these commands are successful or not: >>> >>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -harmo.method >>> band -harmo.cost musig -out >>> /media/gus/Gus/seagrass/processed_data/r2_harmo-band.tif uint8 -ram 128 >>> - otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* -harmo.method >>> rgb -harmo.cost musig -out >>> /media/gus/Gus/seagrass/processed_data/r2_harmo-rgb.tif uint8 -ram 128 >>> >>> Now, you can try to set a very small RAM value and tell me what's >>> happening. Set -ram 1 for instance... >>> >>> Le mercredi 12 septembre 2018 11:42:19 UTC+2, Gus a écrit : >>>> >>>> I tried again last night. OTB 6.6.0 packaged linux version. >>>> Command line: ./otbcli_Mosaic -il /media/gus/Gus/seagrass/3_band/* >>>> -comp.feather large -harmo.method rgb -harmo.cost musig -out >>>> /media/gus/Gus/seagrass/processed_data/r2.tif uint8 -tmpdir >>>> /media/gus/Gus/seagrass/processed_data/temp -ram 128 -progress 1 >>>> >>>> Last lines of output: >>>> 2018-09-12 03:29:30 (INFO): Computing statistics in a decorrelated >>>> colors space suitable for true-colors >>>> 2018-09-12 03:29:44 (INFO): Estimated memory for full processing: >>>> 3.15661e+06MB (avail.: 128 MB), optimal image partitioning: 24662 blocks >>>> 2018-09-12 03:29:44 (INFO): Estimation will be performed in 18639 >>>> blocks of 832x320 pixels >>>> >>>> Anyway, after start, used RAM begins to slowly increase until it >>>> occupied my swap space and froze the computer during the night. >>>> >>>> [image: Screenshot at 2018-09-12 03-30-10.png] >>>> >>>> >>>> >>>> On Sunday, September 9, 2018 at 5:23:19 PM UTC+2, Rémi wrote: >>>>> >>>>> Hello Gus, >>>>> >>>>> To investigate a bit I would like to know if you can copy/paste: >>>>> -your command line >>>>> -the full log >>>>> (In a text file, or on pastebin) >>>>> >>>>> What version of OTB, and what OS do you have? >>>>> Thanks, >>>>> >>>>> Rémi >>>>> >>>>> Le samedi 8 septembre 2018 18:07:56 UTC+2, Gus a écrit : >>>>>> >>>>>> Thanks, Rémi >>>>>> >>>>>> Your method works fairly well, the results are really good and >>>>>> visually pleasant. In fact, I was able to use you app (from gui) for >>>>>> small >>>>>> subsets of my data, which is a set of aerial pictures of the sea surface >>>>>> from which the sunglint has been removed. >>>>>> >>>>>> [image: 2dc4d9d7-25da-4254-9cb5-31536801f461.jpg][image: >>>>>> 54ed0802-e492-438f-9f4d-8dea3593a2b6.jpg] >>>>>> >>>>>> But, when I feed your program with the complete set (about 2500 >>>>>> pictures, 30 Mb each approximately, 8 bit colour) it fails. Last thing I >>>>>> can check is that the temporary distance maps have been correctly >>>>>> written, >>>>>> and that's the last console output. while if Iskip the large feathering >>>>>> step, it seems to work ok, as in next image using slim blending mode, >>>>>> but >>>>>> as for large blending, it fails... >>>>>> >>>>>> [image: sea_1PNG.PNG] >>>>>> >>>>>> On Saturday, September 8, 2018 at 10:57:53 AM UTC+2, Rémi wrote: >>>>>>> >>>>>>> Hello Gus, >>>>>>> >>>>>>> The RAM parameter deals only with the memory footprint during mosaic >>>>>>> compositing and statistics computation, both streamable pipelines (no >>>>>>> limitation on images size). >>>>>>> However, the distance map calculation (which is a temporary file, >>>>>>> then used as an input of the streamed mosaicing compositing pipeline, >>>>>>> just >>>>>>> for feathering) is not a streamable process. Meaning that, for each >>>>>>> input >>>>>>> image to mosaic, the distance map will theoretically require the full >>>>>>> image >>>>>>> in memory to be processed. That's why there is the distancemap.sr >>>>>>> (sampling ratio) parameter. This enables the subsampling of the input >>>>>>> image, to compute the distance map with a smaller footprint. See >>>>>>> README.md, >>>>>>> section "Performance tuning: Distance map images sampling ratio". >>>>>>> This is a current limitation of the application, I will add this >>>>>>> explicitly in the application description. >>>>>>> >>>>>>> Your problem could also come from the fact that your input images >>>>>>> have not the same projection reference (and leading to a near infinite >>>>>>> image, e.g. if you mix degrees and meters units, or with wrong >>>>>>> coordinate >>>>>>> reference system), check this. >>>>>>> >>>>>>> Hope that helps, >>>>>>> >>>>>>> Rémi >>>>>>> >>>>>>> Le vendredi 7 septembre 2018 01:44:23 UTC+2, Gus a écrit : >>>>>>>> >>>>>>>> >>>>>>>> Hello everyone. >>>>>>>> >>>>>>>> I don't clearly understand the process followed by the remote >>>>>>>> module "mosaic". The memory occupied during some phase after distance >>>>>>>> maps >>>>>>>> calculation for large blending with lab color space harmonization >>>>>>>> grows a >>>>>>>> lot, so I obtain a huge page file and my computer reboots. I checked >>>>>>>> the >>>>>>>> paper and peeked into the code a bit, but I'm not familiar with the >>>>>>>> libraries. Is there any way of estimating the ram needed for the >>>>>>>> process? >>>>>>>> Is there a way of overcoming this issue? (probably I will process it >>>>>>>> in >>>>>>>> chunks anyway but that would simplify things quite a lot). >>>>>>>> >>>>>>>> Cheers >>>>>>>> >>>>>>>> Gus >>>>>>>> >>>>>>> -- -- Check the OTB FAQ at http://www.orfeo-toolbox.org/FAQ.html You received this message because you are subscribed to the Google Groups "otb-users" group. To post to this group, send email to otb-users@googlegroups.com To unsubscribe from this group, send email to otb-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/otb-users?hl=en --- You received this message because you are subscribed to the Google Groups "otb-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to otb-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.