jackye1995 commented on issue #1887:
URL: https://github.com/apache/iceberg/issues/1887#issuecomment-741077821
> My vote would be to start with X == 1mb.
> @jackye1995 is the Glue catalog small enough if included w/o dependencies?
Yeah that's actually a good way to go, basically mark all the AWS
dependencies as provided. In that case the added jar size will only be the
`iceberg-aws` itself which is 53KB.
So the user experience would be something like:
```
spark-sql --packages
org.apache.iceberg:iceberg-spark3-runtime:0.11.0,software.amazon.awssdk:bundle:2.15.40
\
--conf spark.sql.catalog.test=org.apache.iceberg.spark.SparkCatalog \
--conf
spark.sql.catalog.test.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \
--conf spark.sql.catalog.test.warehouse=s3://some-bucket
```
which looks good to me.
I will put up a PR for this after #1823 and #1844 are merged to change
dependencies to `compileOnly`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
