Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer
Hi, Large MaxConnections value leads to more concurrent requests and it may cause problems if the WMS server is very busy. Reducing MaxConnections may help if you are the only user of that WMS but if there are thousands of other users as well the service may still run at 100% and drop request either because they reach the server side timeout or because load balancer denies requests. If the overload is temporary then retry after a while could help but I do not believe that GDAL WMS driver can be configured that way https://gdal.org/drivers/raster/wms.html. Special tile cache programs like GeoWebCache are better prepared for failing requests https://www.geowebcache.org/docs/current/production/index.html#seed-failure-tolerance The GDAL WMS configuration options are documented only by example https://gdal.org/drivers/raster/wms.html. I wonder where I could find information about the meaning of options that I can't guess by the name, like "CleanTimeout". -Jukka Rahkonen- Lähettäjä: Roman Breitfuss-Schiffer Lähetetty: perjantai 19. marraskuuta 2021 10.18 Vastaanottaja: Rahkonen Jukka (MML) ; gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer Dear Jukka! Thanks for your answer! I think I'm gonna follow that road you suggested. We have been thinking a lot and I was just wondering if there could be some GDAL configuration which could cause that behaviour? For example the value for in the WMS XML file? Or if more applications are accessing the same WMS XML at the same time which is theoretically possible in our case. Best regards Roman Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML): Hi, It seems that the WMS server is closing the door and there is nothing else to do on the GDAL side except to try again later. Contact the WMS service maintainer and report your troubles. They may be able to provide you a more reliable service especially if you are ready to pay for it. -Jukka Rahkonen- Lähettäjä: gdal-dev <mailto:gdal-dev-boun...@lists.osgeo.org> Puolesta Roman Breitfuss-Schiffer Lähetetty: torstai 18. marraskuuta 2021 11.59 Vastaanottaja: gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Aihe: [gdal-dev] GDALWMS - error: TCP connection reset by peer Dear all! We are running a Python application an a linux distribution (RedHat) which creates clips in MBTILES format from a WMS. We are using the Python-API and are basically doing something like this: src_ds = gdal.Warp( tmp_file, ds_in, format='GTiff', outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]], dstSRS=output_srs, ) ds_out = gdal.Translate( out_filepath, src_ds, format='MBTILES', creationOptions=creation_options, ) The application throughs an error from time to time since a few days. We couldn't figure out a pattern yet and therefore we are also not able to reproduce the error. The error is the following: Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602. URL: TCP connection reset by peer HTTP status code: 0, error: TCP connection reset by peer. Add the HTTP status code to to ignore this error (see http://www.gdal.org/frmt_wms.html). Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock failed at X offset 2764, Y offset 601 Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, Y block offset 601 The exact same data gets processed successfully in one run and not successfully in another run. We didn't change the code nor the Python or GDAL version. Hence, we are kind of puzzled as the error comes and goes seemingly at random. Does anyone of you have any hints? Thanks & best regards! Roman ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) -- ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) Veronikagasse 38/20, 1170 Wien +43 664 5547678 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer
Dear Jukka! Thanks for your answer! I think I'm gonna follow that road you suggested. We have been thinking a lot and I was just wondering if there could be some GDAL configuration which could cause that behaviour? For example the value for in the WMS XML file? Or if more applications are accessing the same WMS XML at the same time which is theoretically possible in our case. Best regards Roman Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML): Hi, It seems that the WMS server is closing the door and there is nothing else to do on the GDAL side except to try again later. Contact the WMS service maintainer and report your troubles. They may be able to provide you a more reliable service especially if you are ready to pay for it. -Jukka Rahkonen- *Lähettäjä:* gdal-dev *Puolesta *Roman Breitfuss-Schiffer *Lähetetty:* torstai 18. marraskuuta 2021 11.59 *Vastaanottaja:* gdal-dev@lists.osgeo.org *Aihe:* [gdal-dev] GDALWMS - error: TCP connection reset by peer Dear all! We are running a Python application an a linux distribution (RedHat) which creates clips in MBTILES format from a WMS. We are using the Python-API and are basically doing something like this: src_ds = gdal.Warp( tmp_file, ds_in, format='GTiff', outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]], dstSRS=output_srs, ) ds_out = gdal.Translate( out_filepath, src_ds, format='MBTILES', creationOptions=creation_options, ) The application throughs an error from time to time since a few days. We couldn't figure out a pattern yet and therefore we are also not able to reproduce the error. The error is the following: Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602. URL: TCP connection reset by peer HTTP status code: 0, error: TCP connection reset by peer. Add the HTTP status code to to ignore this error (see http://www.gdal.org/frmt_wms.html). Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock failed at X offset 2764, Y offset 601 Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, Y block offset 601 The exact same data gets processed successfully in one run and not successfully in another run. We didn't change the code nor the Python or GDAL version. Hence, we are kind of puzzled as the error comes and goes seemingly at random. Does anyone of you have any hints? Thanks & best regards! Roman ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) -- ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) Veronikagasse 38/20, 1170 Wien +43 664 5547678 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer
Hi, It seems that the WMS server is closing the door and there is nothing else to do on the GDAL side except to try again later. Contact the WMS service maintainer and report your troubles. They may be able to provide you a more reliable service especially if you are ready to pay for it. -Jukka Rahkonen- Lähettäjä: gdal-dev Puolesta Roman Breitfuss-Schiffer Lähetetty: torstai 18. marraskuuta 2021 11.59 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] GDALWMS - error: TCP connection reset by peer Dear all! We are running a Python application an a linux distribution (RedHat) which creates clips in MBTILES format from a WMS. We are using the Python-API and are basically doing something like this: src_ds = gdal.Warp( tmp_file, ds_in, format='GTiff', outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]], dstSRS=output_srs, ) ds_out = gdal.Translate( out_filepath, src_ds, format='MBTILES', creationOptions=creation_options, ) The application throughs an error from time to time since a few days. We couldn't figure out a pattern yet and therefore we are also not able to reproduce the error. The error is the following: Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602. URL: TCP connection reset by peer HTTP status code: 0, error: TCP connection reset by peer. Add the HTTP status code to to ignore this error (see http://www.gdal.org/frmt_wms.html). Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock failed at X offset 2764, Y offset 601 Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, Y block offset 601 The exact same data gets processed successfully in one run and not successfully in another run. We didn't change the code nor the Python or GDAL version. Hence, we are kind of puzzled as the error comes and goes seemingly at random. Does anyone of you have any hints? Thanks & best regards! Roman ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] GDALWMS - error: TCP connection reset by peer
Dear all! We are running a Python application an a linux distribution (RedHat) which creates clips in MBTILES format from a WMS. We are using the Python-API and are basically doing something like this: src_ds = gdal.Warp( tmp_file, ds_in, format='GTiff', outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]], dstSRS=output_srs, ) ds_out = gdal.Translate( out_filepath, src_ds, format='MBTILES', creationOptions=creation_options, ) The application throughs an error from time to time since a few days. We couldn't figure out a pattern yet and therefore we are also not able to reproduce the error. The error is the following: Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602. URL: TCP connection reset by peer HTTP status code: 0, error: TCP connection reset by peer. Add the HTTP status code to to ignore this error (see http://www.gdal.org/frmt_wms.html). Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock failed at X offset 2764, Y offset 601 Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, Y block offset 601 The exact same data gets processed successfully in one run and not successfully in another run. We didn't change the code nor the Python or GDAL version. Hence, we are kind of puzzled as the error comes and goes seemingly at random. Does anyone of you have any hints? Thanks & best regards! Roman ___ Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
