Gabriel39 commented on PR #66595: URL: https://github.com/apache/doris/pull/66595#issuecomment-5489758052
I agree that STS is preferable when the caller controls the object-store identity and Doris is accessing the storage service directly, but I do not think STS is a substitute for this HTTP TVF use case. A presigned URL is often an opaque, time-limited capability issued by a third-party system. The consumer may have no ability to obtain STS credentials, and the same GET-only behavior can occur with GCS signed URLs, Azure SAS URLs, CDNs, and internal download gateways. Since HTTP TVF ultimately reads the resource with GET, requiring the same URL to also authorize HEAD during planning is an additional and currently undocumented restriction. A bounded ranged GET is therefore a reasonable compatibility mechanism. That said, I think the implementation needs to be tightened before merge: 1. BE currently returns immediately when HEAD is 200 but Content-Length is unavailable, so it does not perform the fallback described by the PR. 2. A 206 response with a missing, malformed, or unknown Content-Range total is silently accepted in BE. This can leave the file size at SIZE_MAX; consumers such as the whole-message JSON path may then attempt an enormous allocation. 3. Empty resources commonly answer Range: bytes=0-0 with 416 and Content-Range: bytes */0; both FE and BE should recognize that as a zero-length file. 4. The new expected fallback path logs the complete URL. For presigned URLs this can expose signature and credential query parameters, so the query and user-info must be redacted. 5. If FE has already obtained the size, BE should consume that planned metadata instead of repeating unnecessary probes where possible. My recommendation is to keep the GET-only URL compatibility goal, but simplify and harden the probe semantics rather than reject the use case in favor of STS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
