I am trying to understand the meaning of "storage-based"
authentication in kyuubi, which is, as documentation says
(https://kyuubi.readthedocs.io/en/v1.8.1-docs/security/authorization/spark/overview.html#storage-based-authorization),
enabled by default.
What i want to achieve is:
- user logs in to kyuubi
- kyuubi starts spark engine
- spark is configured to read/store data in apache iceberg format
- data is stored in s3 cluster (minio in my case)
- bucket policy is configured in minio to allow/deny access data
according to s3 credentials
Following is the spark configuration example, related to Iceberg:
kyuubi.engineEnv.AWS_REGION=us-east-1
kyuubi.engineEnv.AWS_ACCESS_KEY_ID=minioadmin
kyuubi.engineEnv.AWS_SECRET_ACCESS_KEY=minioadmin
spark.executorEnv.AWS_REGION=us-east-1
spark.executorEnv.AWS_ACCESS_KEY_ID=minioadmin
spark.executorEnv.AWS_SECRET_ACCESS_KEY=minioadmin
The question is: can I start spark using different s3 credentials for
each user? In my case i want to change values of
kyuubi.engineEnv.AWS_ACCESS_KEY_ID and
kyuubi.engineEnv.AWS_SECRET_ACCESS_KEY according to username and
password logged in to kyuubi.
Please advise.
