jackye1995 commented on pull request #1844:
URL: https://github.com/apache/iceberg/pull/1844#issuecomment-742363762
> * CatalogProperties: I'm not sure what these are?
In the context of Spark, it is the `Map<String, String> argument` of spark
source. It is just generalized also for Flink (and any other use case that can
accept a map).
> Should a file io be created just for the json file and then a new file io
be created for the other part of the table after reading the table properties?
(manifest list/ manifest + data files?)
That is why FileIO is treated as a catalog property instead of a table
property, to avoid this complication.
> Where should the kms key id go? Is that a per table thing to write new
data files (table properties) or is it a user/system defined property which
should go in the arguments?
With the same reasoning as FileIO, if you put KMS key as a table property,
then you need to create a table to know the KMS key, which needs to write to S3
and cause a circular dependency.
I do agree there is a use case for one KMS key per table, that can be added
in the future if there is such a request, with the KMS key in catalog property
as default, and the one in table property as an override.
Please let me know if these answer your question.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
