[jira] [Updated] (HUDI-6616) Update delete block format, positions and other changes in Hudi Tech Spec

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6616: - Description: We change the format of delete blocks in HUDI-5760, by using Avro serde instead of K

[jira] [Commented] (HUDI-6616) Update delete block format, positions and other changes in Hudi Tech Spec

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762706#comment-17762706 ] Vinoth Chandar commented on HUDI-6616: -- Can we update all the changes.    * New del

[jira] [Updated] (HUDI-6616) Update delete block format, positions and other changes in Hudi Tech Spec

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6616: - Summary: Update delete block format, positions and other changes in Hudi Tech Spec (was: Update d

[jira] [Closed] (HUDI-5599) Make sure that non-string primary-keys are supported

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar closed HUDI-5599. Resolution: Won't Fix We have completed the analysis of various uuid schemes and the current string

[jira] [Closed] (HUDI-309) General Redesign of Archived Timeline for efficient scan and management

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar closed HUDI-309. --- Resolution: Fixed > General Redesign of Archived Timeline for efficient scan and management > -

[jira] [Updated] (HUDI-1739) Standardize usage of replacecommit files across the code base

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1739: - Fix Version/s: 1.0.0 > Standardize usage of replacecommit files across the code base > ---

[jira] [Commented] (HUDI-309) General Redesign of Archived Timeline for efficient scan and management

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762702#comment-17762702 ] Vinoth Chandar commented on HUDI-309: - I think we can close this, given 9209 has landed

[jira] [Resolved] (HUDI-309) General Redesign of Archived Timeline for efficient scan and management

2023-09-07 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar resolved HUDI-309. - > General Redesign of Archived Timeline for efficient scan and management > ---

[jira] [Updated] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2023-09-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6495: - Reviewers: Vinoth Chandar > Finalize the RFC-61/Non-blocking Concurrency Control design >

[jira] [Updated] (HUDI-6242) Format changes for Hudi 1.X release line

2023-09-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6242: - Description: This EPIC tracks changes to the Hudi storage format. h2. Proposals Format change is

[jira] [Updated] (HUDI-6242) Format changes for Hudi 1.X release line

2023-09-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6242: - Description: This EPIC tracks changes to the Hudi storage format. h2. Proposals Format change is a

[jira] [Updated] (HUDI-6784) Support custom logic for deletion

2023-08-29 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6784: - Fix Version/s: 1.0.0 > Support custom logic for deletion > - > >

[jira] [Updated] (HUDI-6712) Implement optimized keyed lookup on parquet files

2023-08-29 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6712: - Status: Patch Available (was: In Progress) > Implement optimized keyed lookup on parquet files >

[jira] [Commented] (HUDI-6711) Write a RFC for Multi Table Txns

2023-08-29 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759909#comment-17759909 ] Vinoth Chandar commented on HUDI-6711: -- Added myself as reviewer on the JIRA.   > W

[jira] [Updated] (HUDI-6711) Write a RFC for Multi Table Txns

2023-08-29 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6711: - Reviewers: Vinoth Chandar > Write a RFC for Multi Table Txns > >

[jira] [Comment Edited] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758833#comment-17758833 ] Vinoth Chandar edited comment on HUDI-1623 at 8/29/23 2:32 AM: -

[jira] [Comment Edited] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758833#comment-17758833 ] Vinoth Chandar edited comment on HUDI-1623 at 8/29/23 2:31 AM: -

[jira] [Updated] (HUDI-6712) Implement optimized keyed lookup on parquet files

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6712: - Reviewers: Vinoth Chandar > Implement optimized keyed lookup on parquet files > --

[jira] [Comment Edited] (HUDI-6712) Implement optimized keyed lookup on parquet files

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757157#comment-17757157 ] Vinoth Chandar edited comment on HUDI-6712 at 8/29/23 2:14 AM: -

[jira] [Updated] (HUDI-3217) RFC-46: Optimize Record Payload handling

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-3217: - Description: Currently Hudi is biased t/w assumption of particular payload representation (Avro),

[jira] [Assigned] (HUDI-3217) RFC-46: Optimize Record Payload handling

2023-08-28 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-3217: Assignee: Ethan Guo (was: Alexey Kudinkin) > RFC-46: Optimize Record Payload handling > --

[jira] [Comment Edited] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-24 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758834#comment-17758834 ] Vinoth Chandar edited comment on HUDI-1623 at 8/25/23 5:40 AM: -

[jira] [Commented] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-24 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758834#comment-17758834 ] Vinoth Chandar commented on HUDI-1623: -- On TrueTime, we add a new _TrueTimeGenerator_

[jira] [Commented] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-24 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758833#comment-17758833 ] Vinoth Chandar commented on HUDI-1623: -- On the naming of the active timeline instants

[jira] [Updated] (HUDI-6708) Support Record Index with the Async Indexer

2023-08-24 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6708: - Reviewers: Vinoth Chandar > Support Record Index with the Async Indexer >

[jira] [Updated] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-21 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1623: - Description: We suggest a new file naming for the *completed* metadata file: ${start_time}.${acti

[jira] [Commented] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-21 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757197#comment-17757197 ] Vinoth Chandar commented on HUDI-1623: -- Agree on points 2 and 3. I am going to invest

[jira] [Commented] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-21 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757196#comment-17757196 ] Vinoth Chandar commented on HUDI-1623: -- > we only add completion timestamp to the com

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Status: Patch Available (was: In Progress) > New LSM tree style archived timeline > -

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Reviewers: Vinoth Chandar > New LSM tree style archived timeline > ---

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Status: In Progress (was: Open) > New LSM tree style archived timeline >

[jira] [Updated] (HUDI-6242) Format changes for Hudi 1.X release line

2023-08-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6242: - Description: This EPIC tracks changes to the Hudi storage format. Format change is anything that c

[jira] [Updated] (HUDI-5672) Non-blocking multi writer support

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5672: - Status: Patch Available (was: In Progress) > Non-blocking multi writer support >

[jira] [Updated] (HUDI-5672) Archived Timeline as LSM Tree - Initial Impl

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5672: - Summary: Archived Timeline as LSM Tree - Initial Impl (was: Non-blocking multi writer support) >

[jira] [Updated] (HUDI-5672) Non-blocking multi writer support

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5672: - Status: In Progress (was: Reopened) > Non-blocking multi writer support > ---

[jira] [Updated] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1623: - Status: In Progress (was: Open) > Support start_commit_time & end_commit_times for serializable i

[jira] [Updated] (HUDI-5672) Non-blocking multi writer support

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5672: - Reviewers: Vinoth Chandar > Non-blocking multi writer support > -

[jira] [Updated] (HUDI-5672) Non-blocking multi writer support

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5672: - Epic Link: HUDI-6640 > Non-blocking multi writer support > - > >

[jira] [Reopened] (HUDI-5672) Non-blocking multi writer support

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reopened HUDI-5672: -- > Non-blocking multi writer support > - > > Key: HUD

[jira] [Assigned] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-2635: Assignee: Sagar Sumit (was: Vinoth Chandar) > Fix double locking issue with multi-writers

[jira] [Updated] (HUDI-5148) Write RFC for index function

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5148: - Status: Open (was: Patch Available) > Write RFC for index function >

[jira] [Updated] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2635: - Status: In Progress (was: Open) > Fix double locking issue with multi-writers with proper abstrac

[jira] [Updated] (HUDI-6707) Investigate any gaps to achieve longer range time-travel with new timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6707: - Status: In Progress (was: Open) > Investigate any gaps to achieve longer range time-travel with n

[jira] [Assigned] (HUDI-6709) Multi Table Transactions

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6709: Assignee: Sagar Sumit (was: Vinoth Chandar) > Multi Table Transactions > -

[jira] [Updated] (HUDI-6710) Initial investigation to unearth gaps and unearth more open items.

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6710: - Status: In Progress (was: Open) > Initial investigation to unearth gaps and unearth more open ite

[jira] [Updated] (HUDI-6496) Evaluate Parquet Rewriter performance for CoW/Compaction

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6496: - Status: Open (was: In Progress) > Evaluate Parquet Rewriter performance for CoW/Compaction > ---

[jira] [Created] (HUDI-6712) Implement optimized keyed lookup on parquet files

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6712: Summary: Implement optimized keyed lookup on parquet files Key: HUDI-6712 URL: https://issues.apache.org/jira/browse/HUDI-6712 Project: Apache Hudi Issue Typ

[jira] [Assigned] (HUDI-3057) Instants should be generated strictly under locks

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-3057: Assignee: (was: sivabalan narayanan) > Instants should be generated strictly under lock

[jira] [Updated] (HUDI-3057) Instants should be generated strictly under locks

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-3057: - Fix Version/s: 1.0.0 (was: 1.1.0) > Instants should be generated strictly u

[jira] [Commented] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755149#comment-17755149 ] Vinoth Chandar commented on HUDI-6495: -- What are the next steps here? Is the RFC upto

[jira] [Updated] (HUDI-5148) Write RFC for index function

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5148: - Reviewers: Vinoth Chandar > Write RFC for index function > > >

[jira] [Updated] (HUDI-5148) Write RFC for index function

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5148: - Fix Version/s: 1.0.0 > Write RFC for index function > > >

[jira] [Created] (HUDI-6711) Write a RFC for Multi Table Txns

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6711: Summary: Write a RFC for Multi Table Txns Key: HUDI-6711 URL: https://issues.apache.org/jira/browse/HUDI-6711 Project: Apache Hudi Issue Type: Task

[jira] [Created] (HUDI-6710) Initial investigation to unearth gaps and unearth more open items.

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6710: Summary: Initial investigation to unearth gaps and unearth more open items. Key: HUDI-6710 URL: https://issues.apache.org/jira/browse/HUDI-6710 Project: Apache Hudi

[jira] [Created] (HUDI-6709) Multi Table Transactions

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6709: Summary: Multi Table Transactions Key: HUDI-6709 URL: https://issues.apache.org/jira/browse/HUDI-6709 Project: Apache Hudi Issue Type: Epic Repor

[jira] [Updated] (HUDI-6242) Format changes for Hudi 1.X release line

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6242: - Description: This EPIC tracks changes to the Hudi storage format. Format change is anything that c

[jira] [Updated] (HUDI-4500) Support restore to savepoint with holes in the timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-4500: - Epic Link: HUDI-4677 (was: HUDI-3967) > Support restore to savepoint with holes in the timeline >

[jira] [Created] (HUDI-6707) Investigate any gaps to achieve longer range time-travel with new timeline

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6707: Summary: Investigate any gaps to achieve longer range time-travel with new timeline Key: HUDI-6707 URL: https://issues.apache.org/jira/browse/HUDI-6707 Project: Apach

[jira] [Updated] (HUDI-4677) Snapshot view management & Time Travel

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-4677: - Epic Name: hudi-snapshots-and-time-travel (was: snapshot-view) > Snapshot view management & Time

[jira] [Updated] (HUDI-4677) Snapshot view management & Time Travel

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-4677: - Summary: Snapshot view management & Time Travel (was: Snapshot view management) > Snapshot view

[jira] [Updated] (HUDI-6242) Format changes for Hudi 1.X release line

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6242: - Description: This EPIC tracks changes to the Hudi storage format. Format change is anything that c

[jira] [Updated] (HUDI-6626) Support more conprehensive compaction strategies for achived timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6626: - Epic Link: HUDI-6698 > Support more conprehensive compaction strategies for achived timeline > ---

[jira] [Created] (HUDI-6706) Put up a 1.0 tech specs doc, consolidating all storage changes

2023-08-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-6706: Summary: Put up a 1.0 tech specs doc, consolidating all storage changes Key: HUDI-6706 URL: https://issues.apache.org/jira/browse/HUDI-6706 Project: Apache Hudi

[jira] [Updated] (HUDI-309) General Redesign of Archived Timeline for efficient scan and management

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-309: Status: In Progress (was: Open) > General Redesign of Archived Timeline for efficient scan and manag

[jira] [Updated] (HUDI-6494) Explore how to thin HFile and provide C++/Rust reader/writers

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6494: - Status: Open (was: In Progress) > Explore how to thin HFile and provide C++/Rust reader/writers >

[jira] [Updated] (HUDI-6699) An indexed global timeline (phase2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6699: - Epic Link: HUDI-6698 > An indexed global timeline (phase2) > --- >

[jira] [Updated] (HUDI-6699) An indexed global timeline (phase2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6699: - Parent: (was: HUDI-6698) Issue Type: Bug (was: Sub-task) > An indexed global timeline

[jira] [Updated] (HUDI-6700) Archiving should be time based, not this min-max and not per instant. Lets treat it like a log (Phase 2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6700: - Fix Version/s: 1.0.0 > Archiving should be time based, not this min-max and not per instant. Lets

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Epic Link: HUDI-6698 > New LSM tree style archived timeline >

[jira] [Updated] (HUDI-6700) Archiving should be time based, not this min-max and not per instant. Lets treat it like a log (Phase 2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6700: - Epic Link: HUDI-6698 > Archiving should be time based, not this min-max and not per instant. Lets

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Parent: (was: HUDI-6698) Issue Type: Bug (was: Sub-task) > New LSM tree style archive

[jira] [Updated] (HUDI-6699) An indexed global timeline (phase2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6699: - Fix Version/s: 1.0.0 > An indexed global timeline (phase2) > --- >

[jira] [Updated] (HUDI-6700) Archiving should be time based, not this min-max and not per instant. Lets treat it like a log (Phase 2)

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6700: - Parent: (was: HUDI-6698) Issue Type: Bug (was: Sub-task) > Archiving should be time b

[jira] [Updated] (HUDI-6539) New LSM tree style archived timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6539: - Fix Version/s: 1.0.0 > New LSM tree style archived timeline >

[jira] [Updated] (HUDI-6698) New LSM Timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6698: - Epic Name: hudi-lsm-timeline (was: New Hoodie Timeline) > New LSM Timeline > > >

[jira] [Updated] (HUDI-6698) New LSM Timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6698: - Fix Version/s: 1.0.0 (was: 1.1.0) > New LSM Timeline > > >

[jira] [Updated] (HUDI-6698) New LSM Timeline

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6698: - Summary: New LSM Timeline (was: New Hoodie Timeline) > New LSM Timeline > > >

[jira] [Updated] (HUDI-1238) Perf test env

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1238: - Summary: Perf test env (was: [UMBRELLA] Perf test env) > Perf test env > - > >

[jira] [Updated] (HUDI-6641) Remove the log append and always uses the current instant time in file name

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6641: - Parent: (was: HUDI-6640) Issue Type: Improvement (was: Sub-task) > Remove the log app

[jira] [Updated] (HUDI-6641) Remove the log append and always uses the current instant time in file name

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6641: - Epic Link: HUDI-6242 > Remove the log append and always uses the current instant time in file name

[jira] [Commented] (HUDI-6641) Remove the log append and always uses the current instant time in file name

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755020#comment-17755020 ] Vinoth Chandar commented on HUDI-6641: -- this is a format change? That can be tracked

[jira] [Updated] (HUDI-6551) A new slashed month partition value extractor

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6551: - Fix Version/s: 1.1.0 (was: 1.0.0) > A new slashed month partition value ext

[jira] [Assigned] (HUDI-6498) Being able to write SQL against the metadata table

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6498: Assignee: (was: Vinoth Chandar) > Being able to write SQL against the metadata table >

[jira] [Assigned] (HUDI-6497) Introduce a new HudiFileSystem & HudiPath abstraction to remove Hadoop from hudi-common

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6497: Assignee: (was: Vinoth Chandar) > Introduce a new HudiFileSystem & HudiPath abstraction

[jira] [Assigned] (HUDI-6499) Make a consistent HoodieSchema abstraction across the codebase

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6499: Assignee: (was: Vinoth Chandar) > Make a consistent HoodieSchema abstraction across the

[jira] [Assigned] (HUDI-6488) Hide the Hudi meta fields from the various catalogs

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-6488: Assignee: (was: Vinoth Chandar) > Hide the Hudi meta fields from the various catalogs >

[jira] [Updated] (HUDI-5062) Track and announce breaking changes in 1.0.0

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-5062: - Epic Link: HUDI-6242 > Track and announce breaking changes in 1.0.0 >

[jira] [Assigned] (HUDI-1623) Support start_commit_time & end_commit_times for serializable incremental pull

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1623: Assignee: Danny Chen (was: Vinoth Chandar) > Support start_commit_time & end_commit_times

[jira] [Assigned] (HUDI-4756) Clean up usages of "assume.date.partition" config within hudi

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-4756: Assignee: Lin Liu (was: sivabalan narayanan) > Clean up usages of "assume.date.partition"

[jira] [Assigned] (HUDI-3615) Replace List with HoodieData in HoodieFlink/JavaTable and commit executors

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-3615: Assignee: (was: liujinhui) > Replace List with HoodieData in HoodieFlink/JavaTable and

[jira] [Assigned] (HUDI-3035) Unify Parquet writers

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-3035: Assignee: Ethan Guo (was: Alexey Kudinkin) > Unify Parquet writers > -

[jira] [Assigned] (HUDI-2461) Support lock free multi-writer for metadata table

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-2461: Assignee: Vinoth Chandar (was: sivabalan narayanan) > Support lock free multi-writer for m

[jira] [Assigned] (HUDI-2039) Avoid the raw type usage in hudi codebase

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-2039: Assignee: (was: Wei) > Avoid the raw type usage in hudi codebase >

[jira] [Assigned] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-2635: Assignee: Vinoth Chandar (was: sivabalan narayanan) > Fix double locking issue with multi-

[jira] [Assigned] (HUDI-1045) Support updates during clustering

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1045: Assignee: Vinoth Chandar (was: leesf) > Support updates during clustering > --

[jira] [Assigned] (HUDI-5761) Create "Type" configs for current configs that take in classes

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-5761: Assignee: (was: Jonathan Vexler) > Create "Type" configs for current configs that take

[jira] [Assigned] (HUDI-3049) Use flink table name as default synced hive table name

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-3049: Assignee: (was: Danny Chen) > Use flink table name as default synced hive table name >

[jira] [Assigned] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2023-08-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-2954: Assignee: (was: Ethan Guo) > Code cleanup: HFileDataBock - using integer keys is never

[jira] [Commented] (HUDI-6701) Explore use of UUID-6/7 as a replacement for current auto generated keys

2023-08-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754866#comment-17754866 ] Vinoth Chandar commented on HUDI-6701: -- [~linliu]You may have to write some code, to

[jira] [Updated] (HUDI-6701) Explore use of UUID-6/7 as a replacement for current auto generated keys

2023-08-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-6701: - Description: Today, we auto generate string keys of the form (HoodieRecord#generateSequenceId), w

<    2   3   4   5   6   7   8   9   10   11   >