niltecedu commented on issue #398: URL: https://github.com/apache/arrow-rs-object-store/issues/398#issuecomment-3494517244
Sorry about the formatting, fixed it now. Let me come back with a reproducible steps and a test with azurite on #443, currently atleast in downstream libraries in object storage >= 0.12.3; parsing of certain endpoints is causing the GET Request URLs to drop the container name, that is what I observed based on datafusion and delta-rs. #399 and #403 are the only Azure changes I see between 0.12.2 and 0.12.3. Where based on the delta-rs debug logs I saw this behavior; > For "az://account/container/path" and some other endpoints > > The GET Operation ends up used with > > https://storage_acount_name/storage_acount_name?restype=container&comp=list&prefix=test%2Ftesting_delta_table%2F_delta_log%2 > > instead of > > https://storage_acount_name/test_container?restype=container&comp=list&prefix=test%2Ftesting_delta_table%2F_delta_log%2 > I cannot comment on S3 Buckets Parsing as I do not use them consistently but I do know that S3 and Blob do behave quite differently. For azure I have never seen container name being skipped out in any other format I have seen. From I have seen through the codebases parse(url) is called always when you want to build a connection/request. And that's where its stripping the container name off and creating invalid n; especially since the issue is isolated to az:// endpoints and the https:// ones which do have strip bucket function attached to them. For the test I would need to just call in the basic azure storage account requests to azurite with object store wherever it uses or might be using parse, add it to the tests and revert/modify this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
