A-little-bit-of-data commented on issue #6251: URL: https://github.com/apache/paimon/issues/6251#issuecomment-4426693170
> i understand the Bug i can solve Use commit.parallelism = 1 WITH ( 'bucket' = '4', 'file.compression' = 'snappy', 'merge-engine' = 'deduplicate', 'changelog-producer' = 'input', 'sink.parallelism' = '4', 'commit.parallelism' = '1' ); Increase Hive lock timeout this is the mitigation, not fix hive.metastore.client.socket.timeout=600 hive.metastore.lock.numretries=10 hive.metastore.lock.retry.sleep.between.retries=10 > > For production on S3 + Hive: sink.parallelism > 1 commit.parallelism = 1 Or use FilesystemCatalog I searched the documentation and source code for version 1.4.1 for the `commit.parallelism` configuration, but couldn't find its source, although the documentation does mention commit-related aspects. I also tried FilesystemCatalog, but it had other issues, and given my preference for unified metadata management, I abandoned using FilesystemCatalog. <img width="2790" height="1227" alt="Image" src="https://github.com/user-attachments/assets/12293c0a-eeb7-4399-9440-20cb8fdc2adf"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
