jerryzhujing opened a new issue, #4256:
URL: https://github.com/apache/amoro/issues/4256
Currently the iceberg is updated by kafka connect.
Optimizing job failed due to below error
Caused by: org.apache.iceberg.exceptions.ValidationException: Cannot commit,
found new position delete for replaced data file:
Is there any way to fix it?
I also have already added below table properties:
self-optimizing.filte = created_at < '2025-01-01T00:00:00Z'
self-optimizing.allow-partial-commit =true
'self-optimizing.allow-partial-commit' = 'true',
'self-optimizing.max-file-count' = '3000',
'self-optimizing.max-task-size-bytes' = '67108864',
'self-optimizing.minor.trigger.file-count' = '8',
'self-optimizing.minor.trigger.interval' = '900000',
'self-optimizing.major.trigger.duplicate-ratio' = '0.2',
'self-optimizing.target-size' = '134217728',
'self-optimizing.fragment-ratio' = '8',
'self-optimizing.min-target-size-ratio' = '0.75',
'self-optimizing.min-plan-interval' = '60000'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
