mxm commented on PR #14358:
URL: https://github.com/apache/iceberg/pull/14358#issuecomment-3421072105
> Maybe we could add a new parameter to the `ManifestOutputFileFactory`
constructor, like:
>
> ```
> ManifestOutputFileFactory(
> Supplier<Table> tableSupplier,
> Map<String, String> props,
> String flinkJobId,
> String operatorUniqueId,
> int subTaskId,
> long attemptNumber,
> UUID uniqueId) {
> ```
>
> and if the UUID is provided then we can add it to the end of the filename,
like:
>
> ```
> private String generatePath(long checkpointId) {
> return FileFormat.AVRO.addExtension(
> String.format(
> Locale.ROOT,
> "%s-%s-%05d-%d-%d-%05d-%s",
> flinkJobId,
> operatorUniqueId,
> subTaskId,
> attemptNumber,
> checkpointId,
> fileCount.incrementAndGet(),
> uniqueId));
> }
> ```
>
> This would mean that the path generated by the normal sink is not change,
but we can change it for every dynamically generated file names.
Originally, I was a bit hesitant to change the file names, but we agreed on
this approach going forward in
https://github.com/apache/iceberg/pull/14358#discussion_r2439289983. I'll
update the PR as suggested by you above to retain the same file names for the
non-dynamic IcebergSink.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
