On Wed, 20 Aug 2025 at 20:31, Even Rouault <even.roua...@spatialys.com> wrote:
> Javier, > > - which GDAL version it is ? > $ gdalinfo --version GDAL 3.11.3 "Eganville", released 2025/07/12 (I installed conda and qgis today in that machine) > - the "corrupted double-linked list" is definitely a proof of either a > bug, or a mis-configuration. This is a message we see sometimes when GDAL > accidentally links against 2 different libproj. Can you check "ldd > /path/to/libgdal.so" to see if there is not 2 libproj appearing ? > $ ldd anaconda3/envs/qgis/lib/libgdal.so | grep proj libproj.so.25 => /home/jshaw/anaconda3/envs/qgis/lib/./libproj.so.25 (0x00007f2af442e000) > - can you reproduce the issue with a fully blank file with the same > characterists. For example try "gdal_create -if web_mercator.tif > web_mercator_blank.tif -co SPARSE_OK=YES -co TILED=YES", and then run the > gdal_translate to COG on that web_mercator_blank.tif > It says it takes 2 hours... but it was only 30 min The final file was 1.8 GB > - does running in single threaded mode with GDAL_NUM_THREADS=1 make a > difference regarding the error messages? > I started right after the first email a session without defining GDAL_NUM_THREADS. It said 12 hours... let's see tomorrow. (after connecting to check) So far it goes to 40%, much more than before that crashed about 13% (and only 3 hours left) BTW, htop shows 16 cores and 64 GB of RAM. Even > Le 20/08/2025 à 19:17, Javier Jimenez Shaw via gdal-dev a écrit : > > Hi > > I am warping a file to EPSG:3857 and later generating a COG with JPEG > compression. > The input tif file has also JPEG compression. (I learned that I need the - > dstalpha to keep it transparent) > I am doing it in Ubuntu 22.04 using conda. > > First I tried this, using LZW as output of the warp > > GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdalwarp -t_srs EPSG:3857 -ct > "+proj=pipeline +step +inv +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 > +x_0=400000 +y_0=-100000 +ellps=airy +step +proj=push +v_3 +step +proj=cart > +ellps=airy +step +proj=helmert +x=446.448 +y=-125.157 +z=542.06 +rx=0.15 > +ry=0.247 +rz=0.842 +s=-20.489 +convention=position_vector +step +inv > +proj=cart +ellps=WGS84 +step +proj=pop +v_3 +step +proj=webmerc +lat_0=0 > +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84" -co COMPRESS=LZW -co BIGTIFF=YES -co > TILED=YES -dstalpha orthomosaic.tiff web_mercator.tif > > Creating output file that is 677881P x 303664L. > Processing orthomosaic.tiff [1/1] : > 0...10...20...30...40...50...60...70...80...90...100 - done in 00:27:04. > > > then I try to generate the COG > GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdal_translate > web_mercator.tif web_mercator_cog.tif -co COMPRESS=JPEG -co BIGTIFF=YES -of > COG > Input file size is 677881, 303664 > 0ERROR 1: LZWDecode:Not enough data at scanline 0 (short 40448 bytes) > 0. - estimated remaining > time: 00:14:34 > > and failed > > As it is an LZW problem, I tried with DEFLATE: > > GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdalwarp -t_srs EPSG:3857 -ct > "+proj=pipeline +step +inv +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 > +x_0=400000 +y_0=-100000 +ellps=airy +step +proj=push +v_3 +step +proj=cart > +ellps=airy +step +proj=helmert +x=446.448 +y=-125.157 +z=542.06 +rx=0.15 > +ry=0.247 +rz=0.842 +s=-20.489 +convention=position_vector +step +inv > +proj=cart +ellps=WGS84 +step +proj=pop +v_3 +step +proj=webmerc +lat_0=0 > +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84" -co COMPRESS=DEFLATE -co BIGTIFF=YES > -co TILED=YES -dstalpha orthomosaic.tiff web_mercator_d.tif > > Creating output file that is 677881P x 303664L. > Processing orthomosaic.tiff [1/1] : > 0...10...20...30...40...50...60...70...80...90...100 - done in 00:27:34. > > > And then the same > GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdal_translate > web_mercator_d.tif web_mercator_cog.tif -co COMPRESS=JPEG -co BIGTIFF=YES > -of COG > Input file size is 677881, 303664 > 0.. - estimated remaining > time: 02:29:49ERROR 1: ZIPDecode:Decoding error at scanline 0 > 0...10... - estimated remaining > time: 00:42:12 > > and failed :( > > Trying again I got this output at a similar point (but nothing about > ZIPDecode:Decoding > error at scanline 0). > corrupted double-linked list > Aborted (core dumped) > > What can it be? Is there any workaround? > The file is big in pixels, but a big part is transparent. > web_mercator_d.tif is "only" 14GB > > Thank you. > > _______________________________________________ > gdal-dev mailing > listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev > > -- http://www.spatialys.com > My software is free, but my time generally not. > >
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
