zuston opened a new issue, #731: URL: https://github.com/apache/incubator-uniffle/issues/731
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues. ### Describe the feature I found some problems about BlockID in production env. 1. Due to too much iterations, some apps' (like XGBoost training using Spark) taskAttempt ID is large, which means the fixed length of 21 is not enough. 2. Due to above problem, I increase the TASK_ATTEMPT_ID_MAX_LENGTH to 23 and decrease the MAX_SEQUENCE_NO to 6. Pity, I found some apps will fail due to the exceeding the MAX_SEQUENCE_NO range. ### Under the hood of `BLOCKID` construction BlockID will be created by 3 parts, including partitionId, taskAttemptId and sequenceNumber. If using the fixed length for every part, it's hard to be adapt with different applications. So why not using the adaptive length of different parts of blockId? As we know, the partition number has been determinated before writing. That means we could adjust the rest 2 parts' length according to the partition number dynamically. BTW, another solution is to decrease the MAX partitionID length, the default value of 24 is too large. ### Motivation _No response_ ### Describe the solution _No response_ ### Additional context _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
