Re: [Geoserver-users] COG plugin error: Unable to read header for url
Oh you suggested s3://dev/cog.tif so I needed to change url when creating
the coverage store, I didn't realize my bad.
Okay it fully works now :)
Thank you
po 15. 5. 2023 o 12:27 Marek Kadlečík
napísal(a):
> Hello,
> sorry for the late reply, the email fell into spam :/
> Yeah, we just figured out on GitHub that this helps (
> https://github.com/geosolutions-it/imageio-ext/issues/223). Currently I'm
> struggling with the REST API http request payload. When I create a Coverage
> Store through GUI, it works fine, I can create Layers further. However,
> when I create a Coverage Store through the REST API and then try to create
> Coverage, I'm getting an error:
>
> software.amazon.awssdk.services.s3.model.S3Exception: The specified bucket
> is not valid. (Service: S3, Status Code: 400, Request ID: 175F488EDA855F9A).
>
> I'm calling POST
> /rest/workspaces/{workspace_name}/coveragestores/{store_name}/coverages?recalculate=nativebbox
> with payload
> {layer_name}{_type + "_" + layer_name}
> {projection}
>
> Should I add some additional fields to the payload ?
>
> Marek
>
___
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see this:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] COG plugin error: Unable to read header for url
Hello,
sorry for the late reply, the email fell into spam :/
Yeah, we just figured out on GitHub that this helps (
https://github.com/geosolutions-it/imageio-ext/issues/223). Currently I'm
struggling with the REST API http request payload. When I create a Coverage
Store through GUI, it works fine, I can create Layers further. However,
when I create a Coverage Store through the REST API and then try to create
Coverage, I'm getting an error:
software.amazon.awssdk.services.s3.model.S3Exception: The specified bucket
is not valid. (Service: S3, Status Code: 400, Request ID: 175F488EDA855F9A).
I'm calling POST
/rest/workspaces/{workspace_name}/coveragestores/{store_name}/coverages?recalculate=nativebbox
with payload
{layer_name}{_type + "_" + layer_name}
{projection}
Should I add some additional fields to the payload ?
Marek
___
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see this:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] COG plugin error: Unable to read header for url
If you are trying to access private bucket , as it seems from the error picture. Probably you should set url to s3://dev/cog.tif and set environment properties before Geoserver startup. Check this one https://sourceforge.net/p/geoserver/mailman/message/37795912/ Good luck, Goran uto, 25. tra 2023. u 15:33 Marek Kadlečík napisao je: > Hello Andrea, > I think the problem is somewhere else. I found this issue on Atlassian a > few days ago: > https://osgeo-org.atlassian.net/browse/GEOS-10951?search_id=3078624a-bb7c-4187-a594-9a5720088e37 > I'm using the latest available dockerized version of MinIO. Maybe > something is wrong with just the object storage setup. Or maybe I'm putting > the wrong URL in Store GUI. Something like cog:// > http://172.17.0.2:9000/dev/cog.tiff should work with MinIO available at > 172.17.0.2:9000, bucket dev and object cog.tiff, should't it ? Shall I > also set up a region for my MinIO docker instance ? > > Have a nice day, > > Marek > > po 24. 4. 2023 o 19:00 Andrea Aime > napísal(a): > >> Hi Marek, >> the system property is currently the only way to go. >> >> Best regards >> Andrea >> >> On Sun, Apr 16, 2023 at 8:00 PM Marek Kadlečík >> wrote: >> >>> Hello folks, >>> By investigating the stuff, I assume the error is caused by http range >>> request rejection from the side of MinIO. As documentation of COG says ( >>> https://docs.geoserver.org/latest/en/user/community/cog/cog.html), that >>> the plugin is based on these range requests, I found an github issue for >>> minio (https://github.com/minio/minio/issues/5285) which states that >>> multiple http ranges are not supported. However this did not discourage me >>> from trying and I came up with an idea of increasing the >>> it.geosolutions.cog.default.header.length system property. From COG >>> plugin doc: >>> >>> *By default, when accessing a COG, an initial chunk of 16 KB is read in >>> attempt to parse the header so that the reader will have the offset and >>> length of the available tiles. When dealing with files hosting many tiles, >>> it is possible that the whole header won’t fit in the initial chunk. In >>> this case additional reads (chunks of the same size) will be progressively >>> made to complete loading the header. >>> A it.geosolutions.cog.default.header.length system property can be >>> configured to set the length (in bytes) of the reading chunk. Tuning this >>> so that the header is read with few extra requests can help improve >>> performance. A value too large can cause memory consumption issues and will >>> reduce efficiency, as un-necessary data will be read.* >>> >>> The only thing is that I don't know which is the easiest way to change >>> the system property, since I'm running geoserver in docker container. What >>> do you recommend? >>> >>> Have a nice rest of the day, >>> >>> Marek >>> >>> so 15. 4. 2023 o 16:00 Marek Kadlečík >>> napísal(a): >>> Hello folks, I'm playing with the COG plugin and bumping into an error Unable to read header for (see picture). What does this mean? I translated tif file beforehand using gdal_translate nejakeId_B02.tif nejakeId_B02_translated.tif -of COG -co COMPRESS=LZW As for S3 store I'm using local instance of minio object storage. I found this piece of email communication https://sourceforge.net/p/geoserver/mailman/message/37227114/, where the problem seems to be the minio refusing the range requests. I hope the convention :// still applies also in minio. Do you guys know where the problem could be? Would help me a lot. Best regards, Marek ___ >>> Geoserver-users mailing list >>> >>> Please make sure you read the following two resources before posting to >>> this list: >>> - Earning your support instead of buying it, but Ian Turton: >>> http://www.ianturton.com/talks/foss4g.html#/ >>> - The GeoServer user list posting guidelines: >>> http://geoserver.org/comm/userlist-guidelines.html >>> >>> If you want to request a feature or an improvement, also see this: >>> https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer >>> >>> >>> Geoserver-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/geoserver-users >>> >> >> >> -- >> >> Regards, >> >> Andrea Aime >> >> == >> GeoServer Professional Services from the experts! >> >> Visit http://bit.ly/gs-services-us for more information. >> == >> >> Ing. Andrea Aime >> @geowolf >> Technical Lead >> >> GeoSolutions Group >> phone: +39 0584 962313 >> >> fax: +39 0584 1660272 >> >> mob: +39 339 8844549 >> >> https://www.geosolutionsgroup.com/ >> >> http://twitter.com/geosolutions_it >> >> --- >> >> Con riferimento alla normativa sul trattamento dei dati personali (Reg. >> UE 2016/679 - Regolamento generale sulla protezione dei dati
Re: [Geoserver-users] COG plugin error: Unable to read header for url
Hello Andrea, I think the problem is somewhere else. I found this issue on Atlassian a few days ago: https://osgeo-org.atlassian.net/browse/GEOS-10951?search_id=3078624a-bb7c-4187-a594-9a5720088e37 I'm using the latest available dockerized version of MinIO. Maybe something is wrong with just the object storage setup. Or maybe I'm putting the wrong URL in Store GUI. Something like cog://http://172.17.0.2:9000/dev/cog.tiff should work with MinIO available at 172.17.0.2:9000, bucket dev and object cog.tiff, should't it ? Shall I also set up a region for my MinIO docker instance ? Have a nice day, Marek po 24. 4. 2023 o 19:00 Andrea Aime napísal(a): > Hi Marek, > the system property is currently the only way to go. > > Best regards > Andrea > > On Sun, Apr 16, 2023 at 8:00 PM Marek Kadlečík > wrote: > >> Hello folks, >> By investigating the stuff, I assume the error is caused by http range >> request rejection from the side of MinIO. As documentation of COG says ( >> https://docs.geoserver.org/latest/en/user/community/cog/cog.html), that >> the plugin is based on these range requests, I found an github issue for >> minio (https://github.com/minio/minio/issues/5285) which states that >> multiple http ranges are not supported. However this did not discourage me >> from trying and I came up with an idea of increasing the >> it.geosolutions.cog.default.header.length system property. From COG >> plugin doc: >> >> *By default, when accessing a COG, an initial chunk of 16 KB is read in >> attempt to parse the header so that the reader will have the offset and >> length of the available tiles. When dealing with files hosting many tiles, >> it is possible that the whole header won’t fit in the initial chunk. In >> this case additional reads (chunks of the same size) will be progressively >> made to complete loading the header. >> A it.geosolutions.cog.default.header.length system property can be >> configured to set the length (in bytes) of the reading chunk. Tuning this >> so that the header is read with few extra requests can help improve >> performance. A value too large can cause memory consumption issues and will >> reduce efficiency, as un-necessary data will be read.* >> >> The only thing is that I don't know which is the easiest way to change >> the system property, since I'm running geoserver in docker container. What >> do you recommend? >> >> Have a nice rest of the day, >> >> Marek >> >> so 15. 4. 2023 o 16:00 Marek Kadlečík >> napísal(a): >> >>> Hello folks, >>> I'm playing with the COG plugin and bumping into an error Unable to read >>> header for (see picture). What does this mean? I translated tif file >>> beforehand using gdal_translate nejakeId_B02.tif >>> nejakeId_B02_translated.tif -of COG -co COMPRESS=LZW >>> As for S3 store I'm using local instance of minio object storage. I >>> found this piece of email communication >>> https://sourceforge.net/p/geoserver/mailman/message/37227114/, where >>> the problem seems to be the minio refusing the range requests. I hope the >>> convention :// still applies >>> also in minio. >>> >>> Do you guys know where the problem could be? Would help me a lot. >>> >>> Best regards, >>> >>> Marek >>> >>> >>> ___ >> Geoserver-users mailing list >> >> Please make sure you read the following two resources before posting to >> this list: >> - Earning your support instead of buying it, but Ian Turton: >> http://www.ianturton.com/talks/foss4g.html#/ >> - The GeoServer user list posting guidelines: >> http://geoserver.org/comm/userlist-guidelines.html >> >> If you want to request a feature or an improvement, also see this: >> https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer >> >> >> Geoserver-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geoserver-users >> > > > -- > > Regards, > > Andrea Aime > > == > GeoServer Professional Services from the experts! > > Visit http://bit.ly/gs-services-us for more information. > == > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions Group > phone: +39 0584 962313 > > fax: +39 0584 1660272 > > mob: +39 339 8844549 > > https://www.geosolutionsgroup.com/ > > http://twitter.com/geosolutions_it > > --- > > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE > 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si > precisa che ogni circostanza inerente alla presente email (il suo > contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è > riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il > messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra > operazione è illecita. Le sarei comunque grato se potesse darmene notizia. > > This email is intended only for the person or entity to which it is > addressed and may contain information
Re: [Geoserver-users] COG plugin error: Unable to read header for url
Hi Marek, the system property is currently the only way to go. Best regards Andrea On Sun, Apr 16, 2023 at 8:00 PM Marek Kadlečík wrote: > Hello folks, > By investigating the stuff, I assume the error is caused by http range > request rejection from the side of MinIO. As documentation of COG says ( > https://docs.geoserver.org/latest/en/user/community/cog/cog.html), that > the plugin is based on these range requests, I found an github issue for > minio (https://github.com/minio/minio/issues/5285) which states that > multiple http ranges are not supported. However this did not discourage me > from trying and I came up with an idea of increasing the > it.geosolutions.cog.default.header.length system property. From COG > plugin doc: > > *By default, when accessing a COG, an initial chunk of 16 KB is read in > attempt to parse the header so that the reader will have the offset and > length of the available tiles. When dealing with files hosting many tiles, > it is possible that the whole header won’t fit in the initial chunk. In > this case additional reads (chunks of the same size) will be progressively > made to complete loading the header. > A it.geosolutions.cog.default.header.length system property can be > configured to set the length (in bytes) of the reading chunk. Tuning this > so that the header is read with few extra requests can help improve > performance. A value too large can cause memory consumption issues and will > reduce efficiency, as un-necessary data will be read.* > > The only thing is that I don't know which is the easiest way to change the > system property, since I'm running geoserver in docker container. What do > you recommend? > > Have a nice rest of the day, > > Marek > > so 15. 4. 2023 o 16:00 Marek Kadlečík > napísal(a): > >> Hello folks, >> I'm playing with the COG plugin and bumping into an error Unable to read >> header for (see picture). What does this mean? I translated tif file >> beforehand using gdal_translate nejakeId_B02.tif >> nejakeId_B02_translated.tif -of COG -co COMPRESS=LZW >> As for S3 store I'm using local instance of minio object storage. I found >> this piece of email communication >> https://sourceforge.net/p/geoserver/mailman/message/37227114/, where the >> problem seems to be the minio refusing the range requests. I hope the >> convention :// still applies >> also in minio. >> >> Do you guys know where the problem could be? Would help me a lot. >> >> Best regards, >> >> Marek >> >> >> ___ > Geoserver-users mailing list > > Please make sure you read the following two resources before posting to > this list: > - Earning your support instead of buying it, but Ian Turton: > http://www.ianturton.com/talks/foss4g.html#/ > - The GeoServer user list posting guidelines: > http://geoserver.org/comm/userlist-guidelines.html > > If you want to request a feature or an improvement, also see this: > https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer > > > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > -- Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it --- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this:
Re: [Geoserver-users] COG plugin error: Unable to read header for url
Hello folks, By investigating the stuff, I assume the error is caused by http range request rejection from the side of MinIO. As documentation of COG says ( https://docs.geoserver.org/latest/en/user/community/cog/cog.html), that the plugin is based on these range requests, I found an github issue for minio (https://github.com/minio/minio/issues/5285) which states that multiple http ranges are not supported. However this did not discourage me from trying and I came up with an idea of increasing the it.geosolutions.cog.default.header.length system property. From COG plugin doc: *By default, when accessing a COG, an initial chunk of 16 KB is read in attempt to parse the header so that the reader will have the offset and length of the available tiles. When dealing with files hosting many tiles, it is possible that the whole header won’t fit in the initial chunk. In this case additional reads (chunks of the same size) will be progressively made to complete loading the header. A it.geosolutions.cog.default.header.length system property can be configured to set the length (in bytes) of the reading chunk. Tuning this so that the header is read with few extra requests can help improve performance. A value too large can cause memory consumption issues and will reduce efficiency, as un-necessary data will be read.* The only thing is that I don't know which is the easiest way to change the system property, since I'm running geoserver in docker container. What do you recommend? Have a nice rest of the day, Marek so 15. 4. 2023 o 16:00 Marek Kadlečík napísal(a): > Hello folks, > I'm playing with the COG plugin and bumping into an error Unable to read > header for (see picture). What does this mean? I translated tif file > beforehand using gdal_translate nejakeId_B02.tif > nejakeId_B02_translated.tif -of COG -co COMPRESS=LZW > As for S3 store I'm using local instance of minio object storage. I found > this piece of email communication > https://sourceforge.net/p/geoserver/mailman/message/37227114/, where the > problem seems to be the minio refusing the range requests. I hope the > convention :// still applies also > in minio. > > Do you guys know where the problem could be? Would help me a lot. > > Best regards, > > Marek > > > ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
