[GitHub] [hudi] hudi-bot commented on pull request #3071: [HUDI-1976] Resolve Hive and Jackson vulnerability

2022-03-28 Thread GitBox
hudi-bot commented on pull request #3071: URL: https://github.com/apache/hudi/pull/3071#issuecomment-1081434814 ## CI report: * 9a8be2fd9d42d207314efa88f5315a435f1c917d Azure:

[jira] [Commented] (HUDI-3721) Metadata table blocks rollback and restore to savepoint before bootstrapped/init commit

2022-03-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513824#comment-17513824 ] Ethan Guo commented on HUDI-3721: - Per discussion offline, a simpler approach would be to delete the MDT

[jira] [Updated] (HUDI-3632) ensure Deltastreamer writes succeed if a target base path exists, but w/ no contents

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3632: - Issue Type: Improvement (was: Task) > ensure Deltastreamer writes succeed if a target base path exists,

[jira] [Updated] (HUDI-3632) ensure Deltastreamer writes succeed if a target base path exists, but w/ no contents

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3632: - Priority: Minor (was: Major) > ensure Deltastreamer writes succeed if a target base path exists, but w/

[jira] [Updated] (HUDI-3632) ensure Deltastreamer writes succeed if a target base path exists, but w/ no contents

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3632: - Fix Version/s: 0.12.0 (was: 0.11.0) > ensure Deltastreamer writes succeed if a

[jira] [Updated] (HUDI-3616) Ingestigate mor async compact integ test failure

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3616: - Fix Version/s: 0.12.0 (was: 0.11.0) > Ingestigate mor async compact integ test

[jira] [Updated] (HUDI-3616) Ingestigate mor async compact integ test failure

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3616: - Priority: Critical (was: Major) > Ingestigate mor async compact integ test failure >

[jira] [Updated] (HUDI-3609) Create scala version specific artifacts for hudi-spark-client

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3609: - Epic Link: HUDI-3679 > Create scala version specific artifacts for hudi-spark-client >

[jira] [Updated] (HUDI-3609) Create scala version specific artifacts for hudi-spark-client

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3609: - Priority: Blocker (was: Critical) > Create scala version specific artifacts for hudi-spark-client >

[jira] [Updated] (HUDI-3571) Add failure injection tests for spark datasource

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3571: - Priority: Major (was: Blocker) > Add failure injection tests for spark datasource >

[jira] [Updated] (HUDI-3571) Add failure injection tests for spark datasource

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3571: - Issue Type: Improvement (was: Task) > Add failure injection tests for spark datasource >

[jira] [Updated] (HUDI-3560) Add docker image for spark3 hadoop3 and hive3

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3560: - Reviewers: Alexey Kudinkin > Add docker image for spark3 hadoop3 and hive3 >

[jira] [Assigned] (HUDI-3524) Decouple basic and advanced configs in website

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3524: Assignee: Kyle Weller (was: sivabalan narayanan) > Decouple basic and advanced configs in website

[jira] [Updated] (HUDI-3517) Unicode in partition path causes it to be resolved wrongly

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3517: - Component/s: spark-sql > Unicode in partition path causes it to be resolved wrongly >

[jira] [Updated] (HUDI-3517) Unicode in partition path causes it to be resolved wrongly

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3517: - Issue Type: Improvement (was: Bug) > Unicode in partition path causes it to be resolved wrongly >

[jira] [Updated] (HUDI-3517) Unicode in partition path causes it to be resolved wrongly

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3517: - Fix Version/s: 0.12.0 (was: 0.11.0) > Unicode in partition path causes it to be

[jira] [Assigned] (HUDI-3485) Add support for scheduler configs for async clustering w/ deltastreamer and spark streamign

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3485: Assignee: Sagar Sumit (was: sivabalan narayanan) > Add support for scheduler configs for async

[jira] [Updated] (HUDI-3462) List of fixes to Metadata table after 0.10.1

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3462: - Fix Version/s: (was: 0.11.0) > List of fixes to Metadata table after 0.10.1 >

[jira] [Updated] (HUDI-3462) List of fixes to Metadata table after 0.10.1

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3462: - Priority: Blocker (was: Critical) > List of fixes to Metadata table after 0.10.1 >

[jira] [Closed] (HUDI-3435) Do not throw exception when instant to rollback does not exist in metadata table active timeline

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3435. Resolution: Fixed > Do not throw exception when instant to rollback does not exist in metadata > table

[jira] [Updated] (HUDI-3462) List of fixes to Metadata table after 0.10.1

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3462: - Priority: Major (was: Blocker) > List of fixes to Metadata table after 0.10.1 >

[jira] [Closed] (HUDI-3436) 0.11.0/0.10.2 release notes prep ticket

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3436. Resolution: Abandoned > 0.11.0/0.10.2 release notes prep ticket > --- >

[jira] [Updated] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3425: - Fix Version/s: 0.12.0 (was: 0.11.0) > Clean up spill path created by Hudi during

[jira] [Updated] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3425: - Issue Type: Improvement (was: Task) > Clean up spill path created by Hudi during uneventful shutdown >

[jira] [Updated] (HUDI-3436) 0.11.0/0.10.2 release notes prep ticket

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3436: - Fix Version/s: (was: 0.11.0) > 0.11.0/0.10.2 release notes prep ticket >

[jira] [Updated] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3425: - Priority: Major (was: Critical) > Clean up spill path created by Hudi during uneventful shutdown >

[jira] [Closed] (HUDI-3387) Enable async timeline server by default

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3387. Fix Version/s: (was: 0.11.0) Resolution: Duplicate > Enable async timeline server by default >

[jira] [Updated] (HUDI-3340) Fix deploy_staging_jars for diff spark versions

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3340: - Priority: Minor (was: Critical) > Fix deploy_staging_jars for diff spark versions >

[jira] [Assigned] (HUDI-3340) Fix deploy_staging_jars for diff spark versions

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3340: Assignee: Raymond Xu (was: sivabalan narayanan) > Fix deploy_staging_jars for diff spark versions

[jira] [Updated] (HUDI-3291) Flip Default record paylod to DefaultHoodieRecordPayload

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3291: - Epic Link: HUDI-3217 > Flip Default record paylod to DefaultHoodieRecordPayload >

[jira] [Updated] (HUDI-3291) Flip Default record paylod to DefaultHoodieRecordPayload

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3291: - Issue Type: Improvement (was: Task) > Flip Default record paylod to DefaultHoodieRecordPayload >

[jira] [Updated] (HUDI-3291) Flip Default record paylod to DefaultHoodieRecordPayload

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3291: - Fix Version/s: 0.12.0 (was: 0.11.0) > Flip Default record paylod to

[jira] [Closed] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3242. Resolution: Information Provided Resolved for user > Checkpoint 0 is ignored -Partial parquet file

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3242: - Fix Version/s: (was: 0.12.0) > Checkpoint 0 is ignored -Partial parquet file discovery after the

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3242: - Fix Version/s: 0.12.0 > Checkpoint 0 is ignored -Partial parquet file discovery after the first commit >

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3242: - Fix Version/s: (was: 0.11.0) > Checkpoint 0 is ignored -Partial parquet file discovery after the

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3242: - Priority: Minor (was: Major) > Checkpoint 0 is ignored -Partial parquet file discovery after the first

[GitHub] [hudi] codope commented on a change in pull request #5043: [HUDI-3485] Adding scheduler pool configs for async clustering

2022-03-28 Thread GitBox
codope commented on a change in pull request #5043: URL: https://github.com/apache/hudi/pull/5043#discussion_r837068928 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -388,6 +388,14 @@ private boolean

[jira] [Updated] (HUDI-3216) Support timestamp with microseconds precision

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3216: - Priority: Critical (was: Major) > Support timestamp with microseconds precision >

[jira] [Updated] (HUDI-3216) Support timestamp with microseconds precision

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3216: - Issue Type: Improvement (was: Task) > Support timestamp with microseconds precision >

[jira] [Updated] (HUDI-3216) Support timestamp with microseconds precision

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3216: - Fix Version/s: 0.12.0 (was: 0.11.0) > Support timestamp with microseconds

[jira] [Updated] (HUDI-3068) Add support to sync all partitions in hive sync tool

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3068: - Priority: Blocker (was: Major) > Add support to sync all partitions in hive sync tool >

[jira] [Updated] (HUDI-3068) Add support to sync all partitions in hive sync tool

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3068: - Component/s: meta-sync (was: hive) > Add support to sync all partitions in hive sync

[jira] [Updated] (HUDI-3068) Add support to sync all partitions in hive sync tool

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3068: - Issue Type: New Feature (was: Improvement) > Add support to sync all partitions in hive sync tool >

[jira] [Updated] (HUDI-3068) Add support to sync all partitions in hive sync tool

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3068: - Fix Version/s: 0.12.0 (was: 0.11.0) > Add support to sync all partitions in hive

[jira] [Updated] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3062: - Priority: Blocker (was: Critical) > savepoint rollback of last but one savepoint fails >

[jira] [Updated] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3054: - Issue Type: Test (was: Task) > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter >

[jira] [Updated] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3054: - Fix Version/s: 0.12.0 (was: 0.11.0) > Fix flaky TestHoodieClientMultiWriter.

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Component/s: flink metadata > Get Metadata table bootstrapping in Flink in parity with

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Issue Type: Improvement (was: New Feature) > Get Metadata table bootstrapping in Flink in parity with

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Issue Type: New Feature (was: Task) > Get Metadata table bootstrapping in Flink in parity with spark >

[jira] [Updated] (HUDI-2782) Fix marker based strategy for structured streaming

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2782: - Fix Version/s: 0.12.0 (was: 0.11.0) > Fix marker based strategy for structured

[jira] [Commented] (HUDI-2768) Enable async timeline server by default

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513805#comment-17513805 ] Raymond Xu commented on HUDI-2768: -- [https://github.com/apache/hudi/pull/4807] WIP PR   > Enable async

[jira] [Updated] (HUDI-2768) Enable async timeline server by default

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2768: - Fix Version/s: 0.12.0 (was: 0.11.0) > Enable async timeline server by default >

[jira] [Updated] (HUDI-1456) [UMBRELLA] Concurrency Control for Hudi writers and table services

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1456: - Priority: Blocker (was: Major) > [UMBRELLA] Concurrency Control for Hudi writers and table services >

[jira] [Updated] (HUDI-1456) [UMBRELLA] Concurrency Control for Hudi writers and table services

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1456: - Fix Version/s: 0.12.0 > [UMBRELLA] Concurrency Control for Hudi writers and table services >

[jira] [Updated] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2635: - Issue Type: Improvement (was: Task) > Fix double locking issue with multi-writers with proper

[jira] [Updated] (HUDI-2613) Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2613: - Issue Type: Improvement (was: Task) > Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus >

[jira] [Updated] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2635: - Fix Version/s: 0.12.0 (was: 0.11.0) > Fix double locking issue with multi-writers

[jira] [Updated] (HUDI-2635) Fix double locking issue with multi-writers with proper abstraction around trnx manager

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2635: - Component/s: multi-writer > Fix double locking issue with multi-writers with proper abstraction around >

[GitHub] [hudi] hudi-bot removed a comment on pull request #5164: [HUDI-3741] Fix flink bucket index bulk insert generates too many sma…

2022-03-28 Thread GitBox
hudi-bot removed a comment on pull request #5164: URL: https://github.com/apache/hudi/pull/5164#issuecomment-1081411848 ## CI report: * d7552a06e27b4ecb13d6fd290a48bad1cfddb58f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot

[jira] [Updated] (HUDI-2613) Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2613: - Fix Version/s: 0.12.0 (was: 0.11.0) > Fix usages of RealtimeSplit to use the new

[jira] [Commented] (HUDI-2559) Ensure unique timestamps are generated for commit times with concurrent writers

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513803#comment-17513803 ] Raymond Xu commented on HUDI-2559: -- We need to eliminate the issue with locks or giving identifier to

[GitHub] [hudi] hudi-bot commented on pull request #5159: [HUDI-3731] Fixing Column Stats Index record Merging sequence missing `columnName`

2022-03-28 Thread GitBox
hudi-bot commented on pull request #5159: URL: https://github.com/apache/hudi/pull/5159#issuecomment-1081413544 ## CI report: * f9075077ff6d7b14bfaebe9d62b10b141ee9738c Azure:

[jira] [Updated] (HUDI-2559) Ensure unique timestamps are generated for commit times with concurrent writers

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2559: - Fix Version/s: 0.12.0 (was: 0.11.0) > Ensure unique timestamps are generated for

[jira] [Updated] (HUDI-2613) Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2613: - Component/s: code-quality > Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus >

[GitHub] [hudi] hudi-bot commented on pull request #5164: [HUDI-3741] Fix flink bucket index bulk insert generates too many sma…

2022-03-28 Thread GitBox
hudi-bot commented on pull request #5164: URL: https://github.com/apache/hudi/pull/5164#issuecomment-1081413569 ## CI report: * d7552a06e27b4ecb13d6fd290a48bad1cfddb58f Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #5159: [HUDI-3731] Fixing Column Stats Index record Merging sequence missing `columnName`

2022-03-28 Thread GitBox
hudi-bot removed a comment on pull request #5159: URL: https://github.com/apache/hudi/pull/5159#issuecomment-1081318425 ## CI report: * f9075077ff6d7b14bfaebe9d62b10b141ee9738c Azure:

[jira] [Updated] (HUDI-2559) Ensure unique timestamps are generated for commit times with concurrent writers

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2559: - Issue Type: Improvement (was: Task) > Ensure unique timestamps are generated for commit times with

[jira] [Updated] (HUDI-2473) Fix compaction action type in commit metadata

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2473: - Priority: Blocker (was: Major) > Fix compaction action type in commit metadata >

[GitHub] [hudi] hudi-bot commented on pull request #5164: [HUDI-3741] Fix flink bucket index bulk insert generates too many sma…

2022-03-28 Thread GitBox
hudi-bot commented on pull request #5164: URL: https://github.com/apache/hudi/pull/5164#issuecomment-1081411848 ## CI report: * d7552a06e27b4ecb13d6fd290a48bad1cfddb58f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[jira] [Updated] (HUDI-2466) Add and validate comprehensive yamls for spark dml

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2466: - Fix Version/s: 0.12.0 (was: 0.11.0) > Add and validate comprehensive yamls for

[jira] [Updated] (HUDI-2466) Add and validate comprehensive yamls for spark dml

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2466: - Issue Type: Test (was: Task) > Add and validate comprehensive yamls for spark dml >

[jira] [Updated] (HUDI-2464) Create comprehensive spark datasource yamls similar to deltastreamer

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2464: - Fix Version/s: 0.12.0 (was: 0.11.0) > Create comprehensive spark datasource yamls

[jira] [Updated] (HUDI-2151) Make performant out-of-box configs

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2151: - Fix Version/s: 0.12.0 (was: 0.11.0) > Make performant out-of-box configs >

[jira] [Updated] (HUDI-2151) Make performant out-of-box configs

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2151: - Issue Type: Improvement (was: Task) > Make performant out-of-box configs >

[jira] [Updated] (HUDI-2151) Make performant out-of-box configs

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2151: - Priority: Blocker (was: Critical) > Make performant out-of-box configs >

[jira] [Updated] (HUDI-1887) Make schema post processor's default as disabled

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1887: - Issue Type: Improvement (was: Task) > Make schema post processor's default as disabled >

[jira] [Updated] (HUDI-1887) Make schema post processor's default as disabled

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1887: - Component/s: spark > Make schema post processor's default as disabled >

[jira] [Updated] (HUDI-1887) Make schema post processor's default as disabled

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1887: - Fix Version/s: 0.12.0 (was: 0.11.0) > Make schema post processor's default as

[jira] [Updated] (HUDI-3741) Fix flink bucket index bulk insert generates too many small files

2022-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3741: - Labels: pull-request-available (was: ) > Fix flink bucket index bulk insert generates too many

[GitHub] [hudi] danny0405 opened a new pull request #5164: [HUDI-3741] Fix flink bucket index bulk insert generates too many sma…

2022-03-28 Thread GitBox
danny0405 opened a new pull request #5164: URL: https://github.com/apache/hudi/pull/5164 …ll files ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ##

[jira] [Created] (HUDI-3741) Fix flink bucket index bulk insert generates too many small files

2022-03-28 Thread Danny Chen (Jira)
Danny Chen created HUDI-3741: Summary: Fix flink bucket index bulk insert generates too many small files Key: HUDI-3741 URL: https://issues.apache.org/jira/browse/HUDI-3741 Project: Apache Hudi

[jira] [Updated] (HUDI-1549) Programmatic way to fetch earliest commit retained

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1549: - Fix Version/s: 0.12.0 (was: 0.11.0) > Programmatic way to fetch earliest commit

[jira] [Updated] (HUDI-1549) Programmatic way to fetch earliest commit retained

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1549: - Issue Type: New Feature (was: Improvement) > Programmatic way to fetch earliest commit retained >

[jira] [Updated] (HUDI-1549) Programmatic way to fetch earliest commit retained

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1549: - Component/s: timeline-server > Programmatic way to fetch earliest commit retained >

[jira] [Updated] (HUDI-1038) Adding perf benchmark using jmh to Hudi

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1038: - Description: Add benchmark code to the repo to be reused. > Adding perf benchmark using jmh to Hudi >

[jira] [Updated] (HUDI-1038) Adding perf benchmark using jmh to Hudi

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1038: - Priority: Major (was: Critical) > Adding perf benchmark using jmh to Hudi >

[jira] [Updated] (HUDI-1038) Adding perf benchmark using jmh to Hudi

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1038: - Priority: Critical (was: Major) > Adding perf benchmark using jmh to Hudi >

[jira] [Updated] (HUDI-945) Cleanup spillable map files eagerly as part of close

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-945: Priority: Blocker (was: Major) > Cleanup spillable map files eagerly as part of close >

[jira] [Updated] (HUDI-1038) Adding perf benchmark using jmh to Hudi

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1038: - Fix Version/s: 0.12.0 (was: 0.11.0) > Adding perf benchmark using jmh to Hudi >

[jira] [Updated] (HUDI-945) Cleanup spillable map files eagerly as part of close

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-945: Fix Version/s: 0.12.0 (was: 0.11.0) > Cleanup spillable map files eagerly as part of

[GitHub] [hudi] hudi-bot commented on pull request #4962: [HUDI-3355] Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

2022-03-28 Thread GitBox
hudi-bot commented on pull request #4962: URL: https://github.com/apache/hudi/pull/4962#issuecomment-1081401588 ## CI report: * bb65f08889055d1ed1908b858a398a98e9bfac64 UNKNOWN * bd83cf3b8dcf7ae81e54c1d0c9b19e75aa087eec UNKNOWN * 2a8b30e4c3361e7ccfc528be2c455008f56578eb

[GitHub] [hudi] hudi-bot removed a comment on pull request #4962: [HUDI-3355] Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

2022-03-28 Thread GitBox
hudi-bot removed a comment on pull request #4962: URL: https://github.com/apache/hudi/pull/4962#issuecomment-1081319613 ## CI report: * bb65f08889055d1ed1908b858a398a98e9bfac64 UNKNOWN * bd83cf3b8dcf7ae81e54c1d0c9b19e75aa087eec UNKNOWN *

[jira] [Updated] (HUDI-3738) Perf comparison between parquet and hudi for COW snapshot and MOR read optimized

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3738: - Sprint: Hudi-Sprint-Mar-22 > Perf comparison between parquet and hudi for COW snapshot and MOR read >

[jira] [Updated] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3650: - Sprint: Hudi-Sprint-Mar-22 > Revisit all usages of filterPendingCompactionTimeline() >

[jira] [Updated] (HUDI-3135) Fix Show Partitions Command's Result after drop partition

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3135: - Priority: Blocker (was: Critical) > Fix Show Partitions Command's Result after drop partition >

[jira] [Updated] (HUDI-3135) Fix Show Partitions Command's Result after drop partition

2022-03-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3135: - Sprint: Cont' improve - 2021/01/10, Cont' improve - 2021/01/18, Cont' improve - 2021/01/24, Cont'

[jira] [Assigned] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

2022-03-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-3650: --- Assignee: Yue Zhang > Revisit all usages of filterPendingCompactionTimeline() >

[jira] [Updated] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

2022-03-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3650: Priority: Blocker (was: Critical) > Revisit all usages of filterPendingCompactionTimeline() >

  1   2   3   4   5   6   7   8   >