KodaiD commented on PR #16436:
URL: https://github.com/apache/iceberg/pull/16436#issuecomment-4494010139
Verified locally with the step updated:
```
$ docker exec -it spark-iceberg spark-sql \
--conf
spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
\
--conf
spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog \
--conf spark.sql.catalog.spark_catalog.type=hive \
--conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.local.type=hadoop \
--conf spark.sql.catalog.local.warehouse=/home/iceberg/warehouse \
--conf spark.sql.defaultCatalog=local
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
26/05/20 02:01:28 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
26/05/20 02:01:28 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
Spark Web UI available at http://4cdbfd11e5bc:4041/
Spark master: local[*], Application Id: local-1779242488876
spark-sql ()> CREATE DATABASE local.db;
Time taken: 0.471 seconds
spark-sql ()> CREATE TABLE local.db.sample (id int, name string);
Time taken: 0.285 seconds
spark-sql ()>
What's next:
Try Docker Debug for seamless, persistent debugging tools in any
container or image → docker debug spark-iceberg
Learn more at https://docs.docker.com/go/debug-cli/
$ ls warehouse/db/sample/metadata/
v1.metadata.json version-hint.text
```
```
$ docker exec -it spark-iceberg bash -c "cat << EOF >>
/opt/spark/conf/spark-defaults.conf
spark.sql.extensions
org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
spark.sql.catalog.spark_catalog
org.apache.iceberg.spark.SparkSessionCatalog
spark.sql.catalog.spark_catalog.type hive
spark.sql.catalog.local
org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.local.type hadoop
spark.sql.catalog.local.warehouse /home/iceberg/warehouse
spark.sql.defaultCatalog local
EOF"
What's next:
Try Docker Debug for seamless, persistent debugging tools in any
container or image → docker debug spark-iceberg
Learn more at https://docs.docker.com/go/debug-cli/
$ docker exec -it spark-iceberg spark-sql
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
26/05/20 02:09:55 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
26/05/20 02:09:55 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
Spark Web UI available at http://a68f389eb8e9:4041/
Spark master: local[*], Application Id: local-1779242995931
spark-sql ()> CREATE DATABASE local.db;
Time taken: 0.431 seconds
spark-sql ()> CREATE TABLE local.db.sample (id int, name string);
Time taken: 0.208 seconds
spark-sql ()>
What's next:
Try Docker Debug for seamless, persistent debugging tools in any
container or image → docker debug spark-iceberg
Learn more at https://docs.docker.com/go/debug-cli/
$ ls warehouse/db/sample/metadata/
v1.metadata.json version-hint.text
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
