Hi, I guess you are looking at the doc in the wrong way. It is authorization, not authentication.
Well, when Spark accesses s3, it does require authentication. Yes, we can send different creds to each engine. However, these configurations are not generated and passed to engines by default. The possible ways to do this are: - pass them manually by client users in connection configurations[1] - inject these by implementing a custom conf advisor [2] Thanks Kent Yao [1] https://kyuubi.readthedocs.io/en/master/configuration/settings.html#via-jdbc-connection-url [2] https://kyuubi.readthedocs.io/en/master/extensions/server/configuration.html On 2024/03/05 10:59:55 Алексей Рябов wrote: > I am trying to understand the meaning of "storage-based" > authentication in kyuubi, which is, as documentation says > (https://kyuubi.readthedocs.io/en/v1.8.1-docs/security/authorization/spark/overview.html#storage-based-authorization), > enabled by default. > What i want to achieve is: > - user logs in to kyuubi > - kyuubi starts spark engine > - spark is configured to read/store data in apache iceberg format > - data is stored in s3 cluster (minio in my case) > - bucket policy is configured in minio to allow/deny access data > according to s3 credentials > > Following is the spark configuration example, related to Iceberg: > > kyuubi.engineEnv.AWS_REGION=us-east-1 > kyuubi.engineEnv.AWS_ACCESS_KEY_ID=minioadmin > kyuubi.engineEnv.AWS_SECRET_ACCESS_KEY=minioadmin > spark.executorEnv.AWS_REGION=us-east-1 > spark.executorEnv.AWS_ACCESS_KEY_ID=minioadmin > spark.executorEnv.AWS_SECRET_ACCESS_KEY=minioadmin > > The question is: can I start spark using different s3 credentials for > each user? In my case i want to change values of > kyuubi.engineEnv.AWS_ACCESS_KEY_ID and > kyuubi.engineEnv.AWS_SECRET_ACCESS_KEY according to username and > password logged in to kyuubi. > > Please advise. >
