MonkeyCanCode commented on issue #3440: URL: https://github.com/apache/polaris/issues/3440#issuecomment-3802177450
@netapp-acheng @dimas-b so 2/3 fixes are now merged. To handle this specific issue, we will need to have some sort of reproducible. Currently this is happening for the setup described in this PR where spark when using assume role via client id/secret (and this client id/secret only have RO and assume role access and the target role has the write access) when dealing with non-AWS S3-compatible backend. This issue is not there as far as i know from my setup when using AWS S3. Now back to the problem, the issue appears to be the FileIO object returned via `io` function during insert. This is "problematic" for this specific setup as before insert can happen, `refresh` function will be call which will set the cached FileIO object to have only READ and LIST privileges. This is validated by implicitly setting WRITE access within `refresh` function as we don't have a reproducible locally. However, we can't put WRITE access here as it can break RO use case where only READ is allowed. Based on my understanding, Spark use the FileIO object returned to do `newOutputFile` which is what could caused the issue. To better resolve this last piece, we should get a reproducible to ensure the change if efficient and not changing the other behaviors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
