AlexMercedCoder opened a new issue, #5372:
URL: https://github.com/apache/iceberg/issues/5372
To replicate my environment run the following using docker-compose (This is
a simple setup for quick demonstration, but running into an issue, this setup
used to work fine on older versions or spark/iceberg/nessie)
docker-compose.yml
```yaml
#### Nessie + Iceberg Playground Environment
services:
spark-iceberg:
image: alexmerced/nessie-sandbox-072722
ports:
- "8080:8080"
- "7077:7077"
- "8081:8081"
nessie:
image: projectnessie/nessie
ports:
- "19120:19120"
```
- run `docker-compose up`
- run ` docker-compose run spark-iceberg /bin/bash`
- then open up SparkSQL with the command below
```bash
spark-sql --packages
org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:0.13.0,org.projectnessie:nessie-spark-3.2-extensions:0.40.1
--conf
spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSpark32SessionExtensions"
--conf spark.sql.catalog.nessie.uri="http://nessie:19120/api/v1"; -- conf
spark.sql.catalog.nessie.ref=main -- conf
spark.sql.catalog.nessie.authentication.type=NONE --conf
spark.sql.catalog.nessie.catalog-impl=org.apache.iceberg.nessie.NessieCatalog
--conf spark.sql.catalog.nessie=org.apache.iceberg.spark.SparkCatalog --conf
spark.sql.catalog.nessie.warehouse=$PWD/warehouse
```
- I am able to create, insert and query a table just fine
- `LIST REFERENCES` works just fine
- but when I start using `CREATE BRANCH` I get this:
```
spark-sql> CREATE BRANCH IF NOT EXISTS hello IN nessie;
22/07/27 22:55:50 ERROR SparkSQLDriver: Failed in [CREATE BRANCH IF NOT
EXISTS hello IN nessie]
java.util.NoSuchElementException: spark.sql.catalog.nessie.ref
at org.apache.spark.SparkConf.$anonfun$get$1(SparkConf.scala:245)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.SparkConf.get(SparkConf.scala:245)
at
org.apache.spark.sql.execution.datasources.v2.NessieUtils$.getCurrentRefName(NessieUtils.scala:266)
at
org.apache.spark.sql.execution.datasources.v2.NessieUtils$.getCurrentRef(NessieUtils.scala:247)
at
org.apache.spark.sql.execution.datasources.v2.BaseCreateReferenceExec.runInternal(BaseCreateReferenceExec.scala:46)
at
org.apache.spark.sql.execution.datasources.v2.NessieExec.run(NessieExec.scala:34)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
