VasilyMelnik opened a new issue, #5515:
URL: https://github.com/apache/iceberg/issues/5515
### Query engine
Spark, Java API
### Question
Hi all.
I created iceberg table from spark-shell like this:
```
spark.sql(""" create table wrk.my_table
using iceberg
as select ...""")
```
I see my_table in Hive metastore and all SQL queries via Spark-shell work
fine.
But when i try to use Java API:
```
import org.apache.iceberg.hive.HiveCatalog
import org.apache.iceberg.catalog.TableIdentifier
import org.apache.iceberg.Table
import org.apache.iceberg.spark.actions.SparkActions
import org.apache.iceberg.expressions.Expressions
val catalog = new HiveCatalog();
catalog.setConf(spark.sparkContext.hadoopConfiguration);
val properties = new java.util.HashMap[String,String];
catalog.initialize("hive", properties);
val name = TableIdentifier.of("wrk", "my_table");
val table = catalog.loadTable(name)
SparkActions.get().rewriteDataFiles(table).filter(Expressions.alwaysTrue()).option("target-file-size-bytes",
(500 * 1024 * 1024).toString).option("min-input-files","2").execute()
```
i get an error:
> ERROR actions.BaseRewriteDataFilesSparkAction: Cannot complete rewrite,
partial-progress.enabled is not enabled and one of the file set groups failed
to be rewritten. This error occurred during the writing of new files, not
during the commit process. This indicates something is wrong that doesn't
involve conflicts with other Iceberg operations. Enabling
partial-progress.enabled may help in this case but the root cause should be
investigated. Cleaning up 0 groups which finished being written.
> java.lang.RuntimeException:
org.apache.spark.sql.catalyst.analysis.NoSuchTableException: Table
hive.wrk.my_table not found
Table object is valid: i can get location and currentSnapshot with it.
What is the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
