vinishjail97 commented on PR #20188: URL: https://github.com/apache/hudi/pull/20188#issuecomment-6000750925
@voonhous Thanks, you are right: `newBucketFileIdForNBCC` gives a bucket the same file id in every partition. I reworded the description and added `TestSparkNonBlockingConcurrencyControl#testRecordAndSecondaryIndexWithTheSameBucketInDifferentPartitions` in a14aef0f6ad3. It bulk inserts bucket 0 into par1 and par2, then upserts both in one commit with the global record index and the secondary index on. Without the fix the secondary index keeps the old value of the second partition. The bulk inserts are separate commits: a single NBCC bulk insert into two partitions fails with `Fixed handle create is only expected to be invoked once`, because `RDDSimpleBucketBulkInsertPartitioner` keys its tasks by file id prefix, which is the same in every partition under NBCC. That is the same root cause in the write path, so I'd prefer to handle it in a separate issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
