majinghe opened a new pull request, #14928: URL: https://github.com/apache/iceberg/pull/14928
As discussed in https://github.com/apache/iceberg/issues/14638, minio is under maintenance mode, so replacing the minio with RustFS. Testing works fine locally. Testing steps: - Generating spark default conf ``` spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions spark.sql.catalog.demo org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.demo.type rest spark.sql.catalog.demo.uri http://rest:8181 spark.sql.catalog.demo.io-impl org.apache.iceberg.aws.s3.S3FileIO spark.sql.catalog.demo.warehouse s3://warehouse/wh spark.sql.catalog.demo.s3.endpoint http://rustfs:9000 spark.sql.defaultCatalog demo spark.eventLog.enabled true spark.eventLog.dir /home/iceberg/spark-events spark.history.fs.logDirectory /home/iceberg/spark-events spark.sql.catalogImplementation in-memory spark.sql.catalog.demo.s3.path-style-access true ``` - Running container Running docker command to run all containers ``` docker compose up -d ``` - Insert data ``` docker exec -it spark-iceberg spark-sql Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 25/12/25 01:41:54 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 25/12/25 01:42:05 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041. Spark Web UI available at http://4dbf6384ac7a:4041 Spark master: local[*], Application Id: local-1766626926764 spark-sql ()> > CREATE NAMESPACE demo.nyc; Time taken: 4.89 seconds spark-sql ()> CREATE TABLE demo.nyc.taxis > ( > vendor_id bigint, > trip_id bigint, > trip_distance float, > fare_amount double, > store_and_fwd_flag string > ) > PARTITIONED BY (vendor_id); Time taken: 6.362 seconds spark-sql ()> INSERT INTO demo.nyc.taxis > VALUES (1, 1000371, 1.8, 15.32, 'N'), (2, 1000372, 2.5, 22.15, 'N'), (2, 1000373, 0.9, 9.01, 'N'), (1, 1000374, 8.4, 42.13, 'Y'); Time taken: 17.706 seconds spark-sql ()> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
