daviskirk commented on issue #5263:
URL: https://github.com/apache/arrow-rs/issues/5263#issuecomment-1886949791
Thanks, the override of the endpoint URL works well, but the emulator gives
me a 404 any time I try to read it.
These are the requests that I do (this is the write from fsspec):
```
127.000.000.001.09090: {"name": "test.parquet"}
127.000.000.001.09090-127.000.000.001.34408: HTTP/1.1 200 OK
Content-Type: application/json
Location:
http://127.0.0.1:9090/upload/storage/v1/b/bla/o?uploadType=resumable&name=test.parquet&upload_id=386b51ebd4daca9eb150216d11ba7811
Date: Thu, 11 Jan 2024 11:26:43 GMT
Content-Length: 233
127.000.000.001.09090: PAR1...4...Polars.+...PAR1
HTTP/1.1 200 OK
Content-Type: application/json
Range: bytes=0-476
Date: Thu, 11 Jan 2024 11:26:43 GMT
Content-Length: 470
{"kind":"storage#object","name":"test.parquet","id":"bla/test.parquet","bucket":"bla","size":"477","contentType":"application/octet-stream","crc32c":"RrP1+Q==","acl":[{"bucket":"bla","entity":"projectOwner-test-project","object":"test.parquet","projectTeam":{},"role":"OWNER"}],"md5Hash":"8JNyEbW3APYXEj7Aine/3g==","etag":"\"8JNyEbW3APYXEj7Aine/3g==\"","timeCreated":"2024-01-11T11:26:43.876526Z","updated":"2024-01-11T11:26:43.876531Z","generation":"1704972403876533"}
```
Reading directly with fsspec works fine as well:
```
GET /download/storage/v1/b/bla/o/test.parquet?alt=media HTTP/1.1
Host: 127.0.0.1:9090
User-Agent: python-gcsfs/2023.10.0
Accept: */*
Accept-Encoding: gzip, deflate
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Content-Length: 477
Content-Type: application/octet-stream
Last-Modified: Thu, 11 Jan 2024 11:26:43 GMT
X-Goog-Generation: 1704972403876533
X-Goog-Stored-Content-Encoding: identity
Date: Thu, 11 Jan 2024 11:31:19 GMT
PAR1...4.>....PAR1
```
But the "read" using object_store/polars gives me a 404, (it uses the HEAD
as we want to do more using object_store/polars, whereas fsspec obviously just
downloads it directly), but I don't understand why this is failing either:
```
127.000.000.001.09090: HEAD /bla/test%2Eparquet HTTP/1.1
accept: */*
user-agent: object_store/0.8.0
host: 127.0.0.1:9090
127.000.000.001.09090-127.000.000.001.54214: HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Thu, 11 Jan 2024 11:26:47 GMT
Content-Length: 10
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
