(hudi) branch branch-0.x updated: [HUDI-7809] Use Spark SerializableConfiguration to avoid NPE in Kryo serde (#11356)

2024-05-29 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch branch-0.x in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/branch-0.x by this push: new bd26797c8e6 [HUDI-7809] Use Spark

(hudi) branch master updated (014f2d789dd -> 4a530837f1b)

2024-05-29 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 014f2d789dd [MINOR] Archive operation only releases lock when holds it (#11160) add 4a530837f1b [HUDI-7809

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Updated] (HUDI-7655) Support configuration for clean to fail execution if there is at least one file is marked as a failed delete

2024-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7655: -- Fix Version/s: 1.0.0 > Support configuration for clean to fail execut

[jira] [Created] (HUDI-7807) spark-sql updates for a pk less table fails w/ partitioned table

2024-05-28 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7807: - Summary: spark-sql updates for a pk less table fails w/ partitioned table Key: HUDI-7807 URL: https://issues.apache.org/jira/browse/HUDI-7807 Project

[jira] [Updated] (HUDI-7807) spark-sql updates for a pk less table fails w/ partitioned table

2024-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7807: -- Fix Version/s: 0.15.0 1.0.0 > spark-sql updates for a pk less ta

[jira] [Assigned] (HUDI-7807) spark-sql updates for a pk less table fails w/ partitioned table

2024-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7807: - Assignee: sivabalan narayanan > spark-sql updates for a pk less table fail

[jira] [Created] (HUDI-7800) Remove usages of instant time with HoodieRecordLocation

2024-05-25 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7800: - Summary: Remove usages of instant time with HoodieRecordLocation Key: HUDI-7800 URL: https://issues.apache.org/jira/browse/HUDI-7800 Project: Apache Hudi

(hudi) branch master updated: Bump h2 in /hudi-platform-service/hudi-metaserver (#9147)

2024-05-24 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new a0d60d98976 Bump h2 in /hudi-platform-service

(hudi) branch master updated (660c6c7a5b0 -> 70a04c6c782)

2024-05-24 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 660c6c7a5b0 [MINOR] Fix the look up warning msg (#11286) add 70a04c6c782 Bump h2 in /packaging/hudi-metaserver

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Assigned] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7779: - Assignee: sivabalan narayanan > Guarding archival to not archive uninten

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Commented] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849139#comment-17849139 ] sivabalan narayanan commented on HUDI-7779: --- Hey Sagar,      I updated the Jira description w

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

(hudi) branch branch-0.x updated: [HUDI-7784] Fix serde of HoodieHadoopConfiguration in Spark (#11270)

2024-05-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch branch-0.x in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/branch-0.x by this push: new 2e39b41be07 [HUDI-7784] Fix serde

(hudi) branch master updated: [HUDI-7784] Fix serde of HoodieHadoopConfiguration in Spark (#11269)

2024-05-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new e1aa1bcb4af [HUDI-7784] Fix serde

[jira] [Created] (HUDI-7780) Avoid 0 record parquet files

2024-05-19 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7780: - Summary: Avoid 0 record parquet files Key: HUDI-7780 URL: https://issues.apache.org/jira/browse/HUDI-7780 Project: Apache Hudi Issue Type

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency

[jira] [Created] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-18 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7779: - Summary: Guarding archival to not archive unintended commits Key: HUDI-7779 URL: https://issues.apache.org/jira/browse/HUDI-7779 Project: Apache Hudi

[jira] [Created] (HUDI-7778) Duplicate Key exception with RLI

2024-05-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7778: - Summary: Duplicate Key exception with RLI Key: HUDI-7778 URL: https://issues.apache.org/jira/browse/HUDI-7778 Project: Apache Hudi Issue Type

[jira] [Assigned] (HUDI-7778) Duplicate Key exception with RLI

2024-05-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7778: - Assignee: sivabalan narayanan > Duplicate Key exception with

[jira] [Assigned] (HUDI-7771) Make default hoodie record payload as OverwriteWithLatestPayload for 0.15.0

2024-05-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7771: - Assignee: sivabalan narayanan > Make default hoodie record payl

[jira] [Updated] (HUDI-7771) Make default hoodie record payload as OverwriteWithLatestPayload for 0.15.0

2024-05-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7771: -- Fix Version/s: 0.15.0 > Make default hoodie record payl

[jira] [Created] (HUDI-7771) Make default hoodie record payload as OverwriteWithLatestPayload for 0.15.0

2024-05-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7771: - Summary: Make default hoodie record payload as OverwriteWithLatestPayload for 0.15.0 Key: HUDI-7771 URL: https://issues.apache.org/jira/browse/HUDI-7771

(hudi) 01/01: Disabling flaky tests

2024-05-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch branch-0.x-failing-tests-test-mult-writer-archival in repository https://gitbox.apache.org/repos/asf/hudi.git commit ebea9b7c5152d135610bb35de89cf1d9e1ab1449 Author: sivabalan AuthorDate

(hudi) branch branch-0.x-failing-tests-test-mult-writer-archival created (now ebea9b7c515)

2024-05-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch branch-0.x-failing-tests-test-mult-writer-archival in repository https://gitbox.apache.org/repos/asf/hudi.git at ebea9b7c515 Disabling flaky tests This branch includes the following

[jira] [Assigned] (HUDI-7768) Fix failing tests for 0.15.0 release (async compaction and metadata num commits check)

2024-05-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7768: - Assignee: sivabalan narayanan > Fix failing tests for 0.15.0 release (as

[jira] [Created] (HUDI-7768) Fix failing tests for 0.15.0 release (async compaction and metadata num commits check)

2024-05-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7768: - Summary: Fix failing tests for 0.15.0 release (async compaction and metadata num commits check) Key: HUDI-7768 URL: https://issues.apache.org/jira/browse/HUDI-7768

[jira] [Created] (HUDI-7756) Audit all base file readers and replace w/ file slice readers

2024-05-14 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7756: - Summary: Audit all base file readers and replace w/ file slice readers Key: HUDI-7756 URL: https://issues.apache.org/jira/browse/HUDI-7756 Project: Apache

(hudi) branch master updated: [HUDI-7654] Optimizing BQ sync for MDT (#11061)

2024-05-10 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c75ae7f9ba2 [HUDI-7654] Optimizing BQ sync

[jira] [Created] (HUDI-7716) Add more logs around index lookup

2024-05-06 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7716: - Summary: Add more logs around index lookup Key: HUDI-7716 URL: https://issues.apache.org/jira/browse/HUDI-7716 Project: Apache Hudi Issue Type

[jira] [Created] (HUDI-7712) Account for file slices instead of just base files while initializing RLI for MOR table

2024-05-05 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7712: - Summary: Account for file slices instead of just base files while initializing RLI for MOR table Key: HUDI-7712 URL: https://issues.apache.org/jira/browse/HUDI-7712

[jira] [Assigned] (HUDI-7673) Enhance RLI validation w/ MDT validator for false positives

2024-05-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7673: - Assignee: sivabalan narayanan > Enhance RLI validation w/ MDT valida

[jira] [Assigned] (HUDI-7687) Instant should not be archived until replaced file groups or older file versions are deleted

2024-04-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7687: - Assignee: sivabalan narayanan > Instant should not be archived until repla

[jira] [Assigned] (HUDI-7655) Support configuration for clean to fail execution if there is at least one file is marked as a failed delete

2024-04-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7655: - Assignee: sivabalan narayanan > Support configuration for clean to f

[jira] [Created] (HUDI-7673) Enhance RLI validation w/ MDT validator for false positives

2024-04-25 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7673: - Summary: Enhance RLI validation w/ MDT validator for false positives Key: HUDI-7673 URL: https://issues.apache.org/jira/browse/HUDI-7673 Project: Apache

[jira] [Created] (HUDI-7659) Update 0.14.0 release docs to call out that row writer w/ clustering is enabled by default

2024-04-23 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7659: - Summary: Update 0.14.0 release docs to call out that row writer w/ clustering is enabled by default Key: HUDI-7659 URL: https://issues.apache.org/jira/browse/HUDI-7659

(hudi) branch master updated: [HUDI-7655] Minor fix to rli validation with MDT validator (#11060)

2024-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0dbeb77b92a [HUDI-7655] Minor fix to rli

[jira] [Created] (HUDI-7645) Optimize BQ sync tool for MDT

2024-04-20 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7645: - Summary: Optimize BQ sync tool for MDT Key: HUDI-7645 URL: https://issues.apache.org/jira/browse/HUDI-7645 Project: Apache Hudi Issue Type

[jira] [Updated] (HUDI-7644) Add record key info with RLI validation in MDT Validator

2024-04-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7644: -- Fix Version/s: 1.0.0 > Add record key info with RLI validation in MDT Valida

[jira] [Updated] (HUDI-7644) Add record key info with RLI validation in MDT Validator

2024-04-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7644: -- Fix Version/s: 0.15.0 > Add record key info with RLI validation in MDT Valida

[jira] [Assigned] (HUDI-7644) Add record key info with RLI validation in MDT Validator

2024-04-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7644: - Assignee: sivabalan narayanan > Add record key info with RLI validation in

[jira] [Created] (HUDI-7644) Add record key info with RLI validation in MDT Validator

2024-04-20 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7644: - Summary: Add record key info with RLI validation in MDT Validator Key: HUDI-7644 URL: https://issues.apache.org/jira/browse/HUDI-7644 Project: Apache Hudi

[jira] [Updated] (HUDI-7641) Add metrics to track what partitions are enabled in MDT

2024-04-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7641: -- Fix Version/s: 0.15.0 > Add metrics to track what partitions are enabled in

[jira] [Assigned] (HUDI-7641) Add metrics to track what partitions are enabled in MDT

2024-04-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7641: - Assignee: sivabalan narayanan > Add metrics to track what partitions are enab

(hudi) branch master updated: [HUDI-7618] Add ability to ignore checkpoints in delta streamer (#11018)

2024-04-19 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new ca77fda51fe [HUDI-7618] Add ability to ignore

[jira] [Created] (HUDI-7641) Add metrics to track what partitions are enabled in MDT

2024-04-18 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7641: - Summary: Add metrics to track what partitions are enabled in MDT Key: HUDI-7641 URL: https://issues.apache.org/jira/browse/HUDI-7641 Project: Apache Hudi

[jira] [Created] (HUDI-7624) Fix index lookup duration to track tag location duration

2024-04-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7624: - Summary: Fix index lookup duration to track tag location duration Key: HUDI-7624 URL: https://issues.apache.org/jira/browse/HUDI-7624 Project: Apache Hudi

(hudi) branch master updated: [HUDI-7606] Unpersist RDDs after table services, mainly compaction and clustering (#11000)

2024-04-14 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 48952ae5dfd [HUDI-7606] Unpersist RDDs after

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2024-04-03 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7507: -- Fix Version/s: 1.0.0 > ongoing concurrent writers with smaller timestamp can ca

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2024-04-03 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7507: -- Fix Version/s: 0.15.0 > ongoing concurrent writers with smaller timestamp can ca

[jira] [Resolved] (HUDI-4699) Primary key-less data model

2024-04-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-4699. --- > Primary key-less data model > --- > >

[jira] [Closed] (HUDI-4699) Primary key-less data model

2024-04-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4699. - Fix Version/s: 0.14.0 Resolution: Fixed > Primary key-less data mo

[jira] [Reopened] (HUDI-4699) Primary key-less data model

2024-04-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reopened HUDI-4699: --- Assignee: sivabalan narayanan > Primary key-less data mo

(hudi) branch master updated: [HUDI-7557] Fix incremental cleaner when commit for savepoint removed (#10946)

2024-04-01 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9efced37f81 [HUDI-7557] Fix incremental

[jira] [Created] (HUDI-7556) Fix MDT validator to account for additional partitions in MDT

2024-03-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7556: - Summary: Fix MDT validator to account for additional partitions in MDT Key: HUDI-7556 URL: https://issues.apache.org/jira/browse/HUDI-7556 Project: Apache

(hudi) branch master updated: [HUDI-7500] fix gaps with deduce schema and null schema (#10858)

2024-03-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 136d0755ad7 [HUDI-7500] fix gaps with deduce

(hudi) branch master updated: [HUDI-7518] Fix HoodieMetadataPayload merging logic around repeated deletes (#10913)

2024-03-26 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8a137631da8 [HUDI-7518] Fix

[jira] [Created] (HUDI-7549) Data inconsistency issue w/ spurious log block detection

2024-03-25 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7549: - Summary: Data inconsistency issue w/ spurious log block detection Key: HUDI-7549 URL: https://issues.apache.org/jira/browse/HUDI-7549 Project: Apache Hudi

[jira] [Created] (HUDI-7532) Fix schedule compact to only consider DCs after last compaction commit

2024-03-22 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7532: - Summary: Fix schedule compact to only consider DCs after last compaction commit Key: HUDI-7532 URL: https://issues.apache.org/jira/browse/HUDI-7532

[jira] [Created] (HUDI-7528) Fix RowCustomColumnsSortPartitioner to use repartition instead of coalesce

2024-03-22 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7528: - Summary: Fix RowCustomColumnsSortPartitioner to use repartition instead of coalesce Key: HUDI-7528 URL: https://issues.apache.org/jira/browse/HUDI-7528

(hudi) branch master updated: [MINOR] Remove redundant fileId from HoodieAppendHandle (#10901)

2024-03-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 135db099afc [MINOR] Remove redundant fileId

[jira] [Created] (HUDI-7526) Fix constructors for all bulk insert sort partitioners to ensure we could use it as user defined partitioners

2024-03-21 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7526: - Summary: Fix constructors for all bulk insert sort partitioners to ensure we could use it as user defined partitioners Key: HUDI-7526 URL: https://issues.apache.org

[jira] [Comment Edited] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2024-03-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828123#comment-17828123 ] sivabalan narayanan edited comment on HUDI-7507 at 3/19/24 1:12 AM

[jira] [Commented] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2024-03-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828142#comment-17828142 ] sivabalan narayanan commented on HUDI-7507: --- We have already fixed it w/ latest master (1.0

[jira] [Commented] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2024-03-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828123#comment-17828123 ] sivabalan narayanan commented on HUDI-7507: --- Just trying to replay the same scenario for data

[jira] [Created] (HUDI-7511) Offset range calculation in kafka should return all topic partitions

2024-03-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7511: - Summary: Offset range calculation in kafka should return all topic partitions Key: HUDI-7511 URL: https://issues.apache.org/jira/browse/HUDI-7511 Project

(hudi) branch master updated (ca2140e2003 -> 3c8488b831c)

2024-03-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from ca2140e2003 [MINOR] rename KeyGenUtils#enableAutoGenerateRecordKeys (#10871) add 3c8488b831c [HUDI-7506

(hudi) branch master updated (2af83e2d9a8 -> dc349f5293f)

2024-03-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 2af83e2d9a8 [HUDI-7411] Meta sync should consider cleaner commit (#10676) add dc349f5293f [ENG-6316] Bump

(hudi) branch asf-site updated: [DOCS] Updated inline and async process with more details (#10664)

2024-03-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 19734b58a62 [DOCS] Updated inline

(hudi) branch master updated: [HUDI-7411] Meta sync should consider cleaner commit (#10676)

2024-03-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2af83e2d9a8 [HUDI-7411] Meta sync should

[jira] [Created] (HUDI-7491) Handle null extra metadata w/ clean commit metadata

2024-03-07 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7491: - Summary: Handle null extra metadata w/ clean commit metadata Key: HUDI-7491 URL: https://issues.apache.org/jira/browse/HUDI-7491 Project: Apache Hudi

[jira] [Updated] (HUDI-7490) Fix archival guarding data files not yet cleaned up by cleaner when savepoint is removed

2024-03-07 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7490: -- Description: We added a fix recently where cleaner will take care of cleaning up

[jira] [Updated] (HUDI-7490) Fix archival guarding data files not yet cleaned up by cleaner when savepoint is removed

2024-03-07 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7490: -- Description: We added a fix recently where cleaner will take care of cleaning up

[jira] [Created] (HUDI-7490) Fix archival guarding data files not yet cleaned up by cleaner when savepoint is removed

2024-03-07 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7490: - Summary: Fix archival guarding data files not yet cleaned up by cleaner when savepoint is removed Key: HUDI-7490 URL: https://issues.apache.org/jira/browse/HUDI-7490

(hudi) branch master updated: [HUDI-7337] Implement MetricsReporter that reports metrics to M3 (#10565)

2024-03-05 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6c7ee8ef210 [HUDI-7337] Implement

[jira] [Created] (HUDI-7478) Fix max delta commits guard check w/ MDT

2024-03-04 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7478: - Summary: Fix max delta commits guard check w/ MDT Key: HUDI-7478 URL: https://issues.apache.org/jira/browse/HUDI-7478 Project: Apache Hudi Issue

[jira] [Created] (HUDI-7460) Fix compaction schedule with pending delta commits

2024-02-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7460: - Summary: Fix compaction schedule with pending delta commits Key: HUDI-7460 URL: https://issues.apache.org/jira/browse/HUDI-7460 Project: Apache Hudi

(hudi) branch master updated (89d267a4bf4 -> 6f7fe8a05a9)

2024-02-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 89d267a4bf4 [HUDI-5823] Partition ttl management (#9723) add 6f7fe8a05a9 [MINOR] Modify filter to allow removal

[jira] [Assigned] (HUDI-7429) Fix avg record size estimation for delta commits and replace commits

2024-02-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7429: - Assignee: sivabalan narayanan > Fix avg record size estimation for delta comm

[jira] [Created] (HUDI-7429) Fix avg record size estimation for delta commits and replace commits

2024-02-20 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7429: - Summary: Fix avg record size estimation for delta commits and replace commits Key: HUDI-7429 URL: https://issues.apache.org/jira/browse/HUDI-7429 Project

(hudi) branch master updated: [MINOR] Clarify config descriptions (#10681)

2024-02-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9da1f2b15e2 [MINOR] Clarify config

[jira] [Assigned] (HUDI-7407) Add optional clean support to standalone compaction and clustering jobs

2024-02-13 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7407: - Assignee: sivabalan narayanan > Add optional clean support to standal

[jira] [Created] (HUDI-7407) Add optional clean support to standalone compaction and clustering jobs

2024-02-13 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7407: - Summary: Add optional clean support to standalone compaction and clustering jobs Key: HUDI-7407 URL: https://issues.apache.org/jira/browse/HUDI-7407

[jira] [Created] (HUDI-7397) Add support to purge a clustering instant

2024-02-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7397: - Summary: Add support to purge a clustering instant Key: HUDI-7397 URL: https://issues.apache.org/jira/browse/HUDI-7397 Project: Apache Hudi Issue

(hudi) branch master updated (734015f750d -> 635d84a5301)

2024-02-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 734015f750d [HUDI-7394] Fix run script of hudi-timeline-server-bundle (#10640) add 635d84a5301 [HUDI-7373

(hudi) branch master updated: [Hudi-6902] Fix the timestamp format in hive test (#10610)

2024-02-02 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0ac2051ed09 [Hudi-6902] Fix the timestamp

[jira] [Commented] (HUDI-7320) hive-sync unexpectedly loads archived timeline

2024-01-30 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812499#comment-17812499 ] sivabalan narayanan commented on HUDI-7320: --- We did fix something on these lines already. Can

(hudi) branch master updated: [HUDI-7327] remove meta cols from incoming schema in stream sync (#10556)

2024-01-25 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 39123cd5ac5 [HUDI-7327] remove meta cols from

[jira] [Created] (HUDI-7331) Test and certify col stats integration with MOR table

2024-01-24 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7331: - Summary: Test and certify col stats integration with MOR table Key: HUDI-7331 URL: https://issues.apache.org/jira/browse/HUDI-7331 Project: Apache Hudi

(hudi) branch master updated: [HUDI-7237] Hudi Streamer: Handle edge case with null schema, minor cleanups (#10342)

2024-01-23 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b420850430a [HUDI-7237] Hudi Streamer: Handle

(hudi) branch master updated (48ce342a19b -> 9111d430a1c)

2024-01-18 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 48ce342a19b [HUDI-7170] Implement HFile reader independent of HBase (#10241) add 9111d430a1c [HUDI-6902] Fix

(hudi) branch master updated (9111d430a1c -> 407366447de)

2024-01-18 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 9111d430a1c [HUDI-6902] Fix a unit test (#10513) add 407366447de [HUDI-6902] Shutdown metric hooks properly

(hudi) branch master updated: [MINOR] Avoid resource leaks (#10345)

2024-01-10 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new d22bdc59843 [MINOR] Avoid resource leaks

(hudi) branch release-0.14.1 updated (66cff7d7642 -> 5b0d67bc798)

2024-01-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch release-0.14.1 in repository https://gitbox.apache.org/repos/asf/hudi.git from 66cff7d7642 Bumping release candidate number 2 add 5b0d67bc798 [MINOR] Update release version

(hudi) branch master updated: [MINOR] Disable flaky test (#10449)

2024-01-08 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b54381ff9e6 [MINOR] Disable flaky test (#10449

(hudi) branch master updated (a7f248f2461 -> 397a2fdc5d9)

2024-01-04 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from a7f248f2461 [HUDI-7271] Copy a conf in ClusteringOperator to avoid configuration leak (#10441) add 397a2fdc5d9

  1   2   3   4   5   6   7   8   9   10   >