zhushanwei commented on issue #5404:
URL: https://github.com/apache/iceberg/issues/5404#issuecomment-1202309760
> The reason is Iceberg find some delete files associated with the data file
that you want to rewrite. Can you show me your case?
//writer data
Configuration conf = new Configuration();
Map<String, String> properties = new HashMap<>();
CatalogLoader catalogLoader = CatalogLoader.hive(Constants.CATALOG_NAME,
conf, properties);
TableIdentifier tableIdentifier =
TableIdentifier.of(Constants.DATABASE_NAME, Constants.TABLE_NAME);
TableLoader tableLoader = TableLoader.fromCatalog(catalogLoader,
tableIdentifier);
FlinkSink.forRowData(input)
.writeParallelism(parallelism)
.tableLoader(tableLoader)
.upsert(true)
.overwrite(false)
.append();
//rewriteDataFiles
StreamExecutionEnvironment env =
FlinkEnvironment.getEnvironment(parallelism);
tableLoader.open();
Table table = tableLoader.loadTable();
Actions.forTable(env, table)
.rewriteDataFiles()
.maxParallelism(parallelism)
.targetSizeInBytes(256 * 1024 * 1024)
.filter(Expressions.equal("news_postdate", newsPostdate))
.execute();
Is there any other way to solve this problem？Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
