nsivabalan commented on issue #14460:
URL: https://github.com/apache/hudi/issues/14460#issuecomment-4795164331
This feature request appears to have been delivered. Bulk insert with bucket
index is supported for both Spark and Flink across multiple JIRAs that landed
under Epic HUDI-3039 (closed):
- HUDI-5690 — `simpleBucketPartitioner` for Spark bulk insert (#7834)
- HUDI-5994 — Bucket index bulk insert row writer (#8776, commit
9d58ee4b1f1f)
- HUDI-6144 — Consistent hashing bucket index bulk insert (#9137)
- HUDI-6381 — Bucket bulk insert for CoW (#8983)
- HUDI-6515 — Spark2 bucket bulk insert fix (#9163)
- HUDI-3728 / HUDI-3741 — Flink bucket bulk insert
The specific concern ("index key is String type and index config is empty")
is handled at `HoodieWriteConfig.getBucketIndexHashFieldWithDefault()`, which
falls back to the record-key field when `hoodie.bucket.index.hash.field` is not
set; this is what `BaseDatasetBulkInsertCommitActionExecutor` uses to construct
`BucketIndexBulkInsertPartitionerWithRows`. String-type index keys are
exercised in tests like `TestHoodieSimpleBucketIndex`.
Suggesting we close this issue (and the JIRA HUDI-2155) as resolved. If a
specific failure mode still reproduces on master, please attach a minimal repro
(table type, index config, schema of index key) and we can reopen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
