walterddr opened a new pull request #8119:
URL: https://github.com/apache/pinot/pull/8119
Currently FileBasedSegmentWriter allows duplicate segment name generated
when running the file generation and upload process.
This was originally designed to avoid issues such as 2 instances of the
segment writer were running in the same machine and generates files
duplicately.
However this causes trouble.
1. for offline table, the duplicated segment name will be overwritten on
server even though a new file name with extra timestamp is generated in the
segment writer side.
2. for realtime table, this is simply broken.
This PR propose to:
1. overwrite the local file and logs a warning when it happens if user
configures it to overwrite segment.
2. throw exception when user doesn't config overwrite flag and a duplicate
segment name was created.
- also thrown meaningful error message indicating how to change the
segment name generator to avoid duplicates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
