[jira] [Assigned] (HUDI-7585) Avoid reading log files for resolving schema for _hoodie_operation field

2024-05-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7585: Assignee: Danny Chen > Avoid reading log files for resolving schema for _hoodie_operation

[jira] [Assigned] (HUDI-6713) Redesign CDC workload to include partition column for partition pruning

2024-05-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6713: Assignee: Danny Chen (was: Vinoth Chandar) > Redesign CDC workload to include partition

[jira] [Updated] (HUDI-6778) Track schema in metadata table

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6778: - Status: In Progress (was: Open) > Track schema in metadata table >

[jira] [Commented] (HUDI-7234) Handle both inserts and updates in log blocks for partial updates

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844069#comment-17844069 ] Vinoth Chandar commented on HUDI-7234: -- this to be handled in 1.1.0 along with partial update

[jira] [Updated] (HUDI-7234) Handle both inserts and updates in log blocks for partial updates

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7234: - Status: Open (was: In Progress) > Handle both inserts and updates in log blocks for partial

[jira] [Updated] (HUDI-7234) Handle both inserts and updates in log blocks for partial updates

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7234: - Fix Version/s: 1.1.0 (was: 1.0.0) > Handle both inserts and updates in log

[jira] [Updated] (HUDI-7541) Ensure extensibility to new indexes - vectors, search and other formats (CLP, unstructured data)

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7541: - Fix Version/s: 1.1.0 (was: 1.0.0) > Ensure extensibility to new indexes -

[jira] [Commented] (HUDI-7541) Ensure extensibility to new indexes - vectors, search and other formats (CLP, unstructured data)

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844068#comment-17844068 ] Vinoth Chandar commented on HUDI-7541: -- Punting this to 1.1 > Ensure extensibility to new indexes -

[jira] [Closed] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar closed HUDI-7679. Resolution: Duplicate > Ensure extensibility to unstructured data, logs (CLP), vectors, other index

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Fix Version/s: (was: 1.1.0) > Consolidate the CDC Formats (changelog format, RFC-51) >

[jira] [Resolved] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar resolved HUDI-7679. -- > Ensure extensibility to unstructured data, logs (CLP), vectors, other index > types >

[jira] [Updated] (HUDI-7234) Handle both inserts and updates in log blocks for partial updates

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7234: - Status: In Progress (was: Open) > Handle both inserts and updates in log blocks for partial

[jira] [Updated] (HUDI-7541) Ensure extensibility to new indexes - vectors, search and other formats (CLP, unstructured data)

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7541: - Status: In Progress (was: Open) > Ensure extensibility to new indexes - vectors, search and

[jira] [Updated] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7679: - Status: In Progress (was: Open) > Ensure extensibility to unstructured data, logs (CLP),

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Fix Version/s: 1.1.0 > Consolidate the CDC Formats (changelog format, RFC-51) >

[jira] [Resolved] (HUDI-7540) Check for gaps on storing inserts on log files

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar resolved HUDI-7540. -- > Check for gaps on storing inserts on log files > -- >

[jira] [Updated] (HUDI-7234) Handle both inserts and updates in log blocks for partial updates

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7234: - Description: Inserts can be written to log blocks, e.g., Flink.  We need to handle such case for

[jira] [Closed] (HUDI-7540) Check for gaps on storing inserts on log files

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar closed HUDI-7540. Resolution: Invalid > Check for gaps on storing inserts on log files >

[jira] [Updated] (HUDI-7229) Enable partial updates for CDC work payload

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7229: - Description: OLTP workloads on upstream databases, often update/delete/insert different columns

[jira] [Commented] (HUDI-7229) Enable partial updates for CDC work payload

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843110#comment-17843110 ] Vinoth Chandar commented on HUDI-7229: -- Punting this to 1.1  # [1.1] Implement support on top of

[jira] [Updated] (HUDI-7229) Enable partial updates for CDC work payload

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7229: - Fix Version/s: 1.1.0 (was: 1.0.0) > Enable partial updates for CDC work

[jira] [Commented] (HUDI-7671) Make Hudi timeline backward compatible

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843109#comment-17843109 ] Vinoth Chandar commented on HUDI-7671: -- balaji - this may be a dupe.  > Make Hudi timeline backward

[jira] [Updated] (HUDI-7671) Make Hudi timeline backward compatible

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7671: - Epic Link: HUDI-6242 > Make Hudi timeline backward compatible >

[jira] [Assigned] (HUDI-7671) Make Hudi timeline backward compatible

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7671: Assignee: Balaji Varadarajan (was: Danny Chen) > Make Hudi timeline backward compatible >

[jira] [Updated] (HUDI-7678) Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads.

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7678: - Description: With the move towards making partial updates a first class citizen, that does not

[jira] [Updated] (HUDI-7678) Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads.

2024-05-02 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7678: - Description: With the move towards making partial updates a first class citizen, that does not

[jira] [Updated] (HUDI-7665) Rolling upgrade of 1.0

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7665: - Description: We need to update the table version due to the format changes in 1.0. | * Plan to

[jira] [Assigned] (HUDI-7665) Rolling upgrade of 1.0

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7665: Assignee: Balaji Varadarajan > Rolling upgrade of 1.0 > --- > >

[jira] [Updated] (HUDI-7665) Rolling upgrade of 1.0

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7665: - Summary: Rolling upgrade of 1.0 (was: Upgrade Table Version) > Rolling upgrade of 1.0 >

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Description: For sake of more consistency, we need to consolidate the the changelog mode

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Reviewers: Danny Chen, Ethan Guo > Consolidate the CDC Formats (changelog format, RFC-51) >

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Description: For sake of more consistency, we need to consolidate the the changelog mode

[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)

2024-05-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7538: - Description: For sake of more consistency, we need to consolidate the the changelog mode

[jira] [Updated] (HUDI-6712) Implement optimized keyed lookup on parquet files

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6712: - Status: Open (was: Patch Available) > Implement optimized keyed lookup on parquet files >

[jira] [Updated] (HUDI-6700) Archiving should be time based, not this min-max and not per instant. Lets treat it like a log (Phase 2)

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6700: - Status: Open (was: In Progress) > Archiving should be time based, not this min-max and not per

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842466#comment-17842466 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 9:26 PM: --- h2. [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842466#comment-17842466 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 9:01 PM: --- h2. [WIP]

[jira] [Updated] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Description: h4. We need to allow a writer w writing to file groups f1, f2, f3, concurrently

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 8:27 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 8:27 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 6:32 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 6:32 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 6:20 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 5:56 PM: --- h2.  [WIP]

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842466#comment-17842466 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 5:56 PM: --- h2. [WIP]

[jira] [Commented] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842466#comment-17842466 ] Vinoth Chandar commented on HUDI-1045: -- [WIP] Approach 2 : Introduce pointer data blocks into storage

[jira] [Commented] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842465#comment-17842465 ] Vinoth Chandar commented on HUDI-1045: -- h3.  [WIP] Approach 1 :  Redistribute records from the

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841372#comment-17841372 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 5:54 PM: --- At first it

[jira] [Comment Edited] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841372#comment-17841372 ] Vinoth Chandar edited comment on HUDI-1045 at 4/30/24 4:26 PM: --- At first it

[jira] [Updated] (HUDI-1045) Support updates during clustering

2024-04-30 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Description: We need to allow a writer w writing to file groups f1, f2, f3, concurrently while a

[jira] [Updated] (HUDI-1045) Support updates during clustering

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Status: In Progress (was: Open) > Support updates during clustering >

[jira] [Updated] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6495: - Sprint: Sprint 2023-04-26 > Finalize the RFC-61/Non-blocking Concurrency Control design >

[jira] [Updated] (HUDI-1045) Support updates during clustering

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Sprint: Sprint 2023-04-26 > Support updates during clustering > -

[jira] [Assigned] (HUDI-7610) Delete records are inconsistent depending on MOR/COW, Avro/Spark record merger, new filegroup reader enabled/disabled

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7610: Assignee: Ethan Guo (was: Vinoth Chandar) > Delete records are inconsistent depending on

[jira] [Assigned] (HUDI-7610) Delete records are inconsistent depending on MOR/COW, Avro/Spark record merger, new filegroup reader enabled/disabled

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7610: Assignee: Vinoth Chandar > Delete records are inconsistent depending on MOR/COW,

[jira] [Updated] (HUDI-7280) Add/Drop/Rename table properties hoodie.properties

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7280: - Sprint: Sprint 2023-04-26 > Add/Drop/Rename table properties hoodie.properties >

[jira] [Updated] (HUDI-7539) Use .compaction for compaction action consistently

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7539: - Sprint: Sprint 2023-04-26 > Use .compaction for compaction action consistently >

[jira] [Commented] (HUDI-7610) Delete records are inconsistent depending on MOR/COW, Avro/Spark record merger, new filegroup reader enabled/disabled

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841338#comment-17841338 ] Vinoth Chandar commented on HUDI-7610: -- [~guoyihua] to triage > Delete records are inconsistent

[jira] [Assigned] (HUDI-7280) Add/Drop/Rename table properties hoodie.properties

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7280: Assignee: Vinoth Chandar > Add/Drop/Rename table properties hoodie.properties >

[jira] [Deleted] (HUDI-7680) Decide on changing compaction completed action to be compaction vs .commit

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar deleted HUDI-7680: - > Decide on changing compaction completed action to be compaction vs .commit >

[jira] [Updated] (HUDI-1739) Standardize usage of replacecommit files across the code base

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1739: - Sprint: Sprint 2023-04-26 > Standardize usage of replacecommit files across the code base >

[jira] [Created] (HUDI-7680) Decide on changing compaction completed action to be compaction vs .commit

2024-04-26 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-7680: Summary: Decide on changing compaction completed action to be compaction vs .commit Key: HUDI-7680 URL: https://issues.apache.org/jira/browse/HUDI-7680 Project:

[jira] [Updated] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7679: - Sprint: Sprint 2023-04-26 > Ensure extensibility to unstructured data, logs (CLP), vectors, other

[jira] [Assigned] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7679: Assignee: Vinoth Chandar > Ensure extensibility to unstructured data, logs (CLP), vectors,

[jira] [Updated] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7679: - Fix Version/s: 1.0.0 > Ensure extensibility to unstructured data, logs (CLP), vectors, other

[jira] [Created] (HUDI-7679) Ensure extensibility to unstructured data, logs (CLP), vectors, other index types

2024-04-26 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-7679: Summary: Ensure extensibility to unstructured data, logs (CLP), vectors, other index types Key: HUDI-7679 URL: https://issues.apache.org/jira/browse/HUDI-7679

[jira] [Updated] (HUDI-7542) Ensure extensibility to time-travel writes

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7542: - Sprint: Sprint 2024-03-25 (was: Sprint 2024-03-25, Sprint 2023-04-26) > Ensure extensibility to

[jira] [Comment Edited] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841334#comment-17841334 ] Vinoth Chandar edited comment on HUDI-6495 at 4/26/24 6:22 PM: --- We still

[jira] [Commented] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841334#comment-17841334 ] Vinoth Chandar commented on HUDI-6495: -- We still need to complete the following  * How is NBCC

[jira] [Updated] (HUDI-7678) Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads.

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7678: - Sprint: Sprint 2023-04-26 > Finalize the Merger APIs and make a plan for moving over all existing

[jira] [Created] (HUDI-7678) Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads.

2024-04-26 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-7678: Summary: Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads. Key: HUDI-7678 URL: https://issues.apache.org/jira/browse/HUDI-7678

[jira] [Updated] (HUDI-7546) TLA+ Spec for Hudi CC

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7546: - Description: Aspects to model  * Time generation. * NBCC and OCC, together with table services

[jira] [Assigned] (HUDI-7547) Simplification of archival, savepoint, cleaning interplays

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7547: Assignee: Vinoth Chandar (was: Danny Chen) > Simplification of archival, savepoint,

[jira] [Updated] (HUDI-7546) TLA+ Spec for Hudi CC

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7546: - Sprint: Sprint 2023-04-26 > TLA+ Spec for Hudi CC > - > >

[jira] [Assigned] (HUDI-7546) TLA+ Spec for Hudi CC

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-7546: Assignee: Vinoth Chandar > TLA+ Spec for Hudi CC > - > >

[jira] [Updated] (HUDI-7677) Complete tech specs along with TLA+ for 1.x

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7677: - Description: * Need to cover table types (CoW as a MoR special case), all query types. * Cover

[jira] [Updated] (HUDI-7677) Complete tech specs along with TLA+ for 1.x

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7677: - Description: * Need to cover table types (CoW as a MoR special case), all query types. * Cover

[jira] [Updated] (HUDI-7677) Complete tech specs along with TLA+ for 1.x

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7677: - Sprint: Sprint 2023-04-26 > Complete tech specs along with TLA+ for 1.x >

[jira] [Created] (HUDI-7677) Complete tech specs along with TLA+ for 1.x

2024-04-26 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-7677: Summary: Complete tech specs along with TLA+ for 1.x Key: HUDI-7677 URL: https://issues.apache.org/jira/browse/HUDI-7677 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7652: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Add new MergeKey API to

[jira] [Updated] (HUDI-6791) Integrate FileGroupReader with NewHoodieParquetFileFormat for Spark CDC Query

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6791: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Integrate

[jira] [Updated] (HUDI-6787) Hive Integrate FileGroupReader with HoodieMergeOnReadSnapshotReader and RealtimeCompactedRecordReader for Hive

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6787: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Hive Integrate

[jira] [Updated] (HUDI-7543) Implement CDC query support (MoR/CoW) for Spark on FGReader

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7543: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Implement CDC query

[jira] [Updated] (HUDI-7633) Use try with resources for AutoCloseable

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7633: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Use try with resources

[jira] [Updated] (HUDI-7350) Introduce HoodieIOFactory to abstract the reader and writer implementation

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7350: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Introduce

[jira] [Updated] (HUDI-7544) Harden, Stress and Performance test the LSM timeline on cloud storage

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7544: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Harden, Stress and

[jira] [Updated] (HUDI-7639) Refactor HoodieFileIndex so that different indexes can be used via optimizer rules

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7639: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Refactor HoodieFileIndex

[jira] [Updated] (HUDI-6712) Implement optimized keyed lookup on parquet files

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6712: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Implement optimized

[jira] [Updated] (HUDI-7216) Support reading bloom filter block (BLOOM_CHUNK) in HFile reader

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7216: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Support reading bloom

[jira] [Updated] (HUDI-7157) Support filter pushdown for positional merging in Spark 3.5

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7157: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Support filter pushdown

[jira] [Updated] (HUDI-7668) Add APIs in StorageConfiguration

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7668: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Add APIs in

[jira] [Updated] (HUDI-7221) Move Hudi Option class from hudi-common to hudi-io module

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7221: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Move Hudi Option class

[jira] [Updated] (HUDI-7672) Fix the Hive server scratch dir for tests in hudi-utilities

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7672: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Fix the Hive server

[jira] [Updated] (HUDI-7669) Move config classes and utils to proper places

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7669: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Move config classes and

[jira] [Updated] (HUDI-7594) Create MOR record reader based on HoodieStorage abstraction for Trino

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7594: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Create MOR record reader

[jira] [Updated] (HUDI-6699) An indexed global timeline (phase2)

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6699: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > An indexed global

[jira] [Updated] (HUDI-7227) Enable completion time for File Group Reader

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7227: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Enable completion time

[jira] [Updated] (HUDI-7545) Concurrency control for LSM timeline management and writing.

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7545: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Concurrency control for

[jira] [Updated] (HUDI-7075) Fix validation of parquet column projection on HadoopFsRelation in TestParquetColumnProjection

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7075: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Fix validation of

[jira] [Updated] (HUDI-7065) Fix the new file group reader with COW in Spark integration

2024-04-26 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-7065: - Sprint: Sprint 2024-03-25, Sprint 2023-04-26 (was: Sprint 2024-03-25) > Fix the new file group

  1   2   3   4   5   6   7   8   9   10   >