[GitHub] [hudi] hudi-bot commented on pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
hudi-bot commented on PR #6580: URL: https://github.com/apache/hudi/pull/6580#issuecomment-1250004648 ## CI report: * ff98ae0dda69ee611e4814fbae9c8ddc0a93a4f1 UNKNOWN * 99451dc89547f803eb6823b2baa620096e76459e UNKNOWN * 11ba7cd991ca83773aae03b1fd7271364079be21 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1250004590 ## CI report: * 9f7d93d6247a1b523b6f52c41bd2bc2f5f1ed23e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1250003932 ## CI report: * 9f7d93d6247a1b523b6f52c41bd2bc2f5f1ed23e Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

2022-09-16 Thread GitBox
yihua commented on code in PR #5052: URL: https://github.com/apache/hudi/pull/5052#discussion_r973541985 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java: ## @@ -232,16 +232,21 @@ protected synchronized void scanInternal(Option

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1250003753 ## CI report: * 5944f5cbe9ce73fe6b7e27a0d381eaeb80dead38 UNKNOWN * 4ef7b451c3dd795906f3f68571256baeb330a59f UNKNOWN * 6aeb3d0d8f09aeab2a5766cf9d25ecb30537 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6702: URL: https://github.com/apache/hudi/pull/6702#issuecomment-1250003369 ## CI report: * 3fd138c9b0f6a49c0c231a7c9bb17d8cc90b4c4c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6672: [HUDI-4757] Create pyspark examples

2022-09-16 Thread GitBox
hudi-bot commented on PR #6672: URL: https://github.com/apache/hudi/pull/6672#issuecomment-1250003354 ## CI report: * 64f7aa6d99a87fb98bfcd25dd1ed06048b9487b6 UNKNOWN * 0a3ab818de34d0f9b3db40fe4cbe47e32363d27f Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

2022-09-16 Thread GitBox
yihua commented on code in PR #5052: URL: https://github.com/apache/hudi/pull/5052#discussion_r973541666 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java: ## @@ -361,6 +366,19 @@ protected synchronized void scanInternal(Option

[GitHub] [hudi] scxwhite commented on a diff in pull request #6670: [HUDI-4842] Support compression strategy based on delte file length

2022-09-16 Thread GitBox
scxwhite commented on code in PR #6670: URL: https://github.com/apache/hudi/pull/6670#discussion_r973539801 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java: ## @@ -106,6 +106,12 @@ public class HoodieCompactionConfig extends

[GitHub] [hudi] hudi-bot commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-09-16 Thread GitBox
hudi-bot commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1249996254 ## CI report: * 5a16d35ec42bf86e5759ebb155cad40e83aba9f9 UNKNOWN * 1587f472f18d7b524971637abe64d171c9799818 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1249996107 ## CI report: * 5944f5cbe9ce73fe6b7e27a0d381eaeb80dead38 UNKNOWN * 4ef7b451c3dd795906f3f68571256baeb330a59f UNKNOWN * 6aeb3d0d8f09aeab2a5766cf9d25ecb30537 UNKNOWN *

[GitHub] [hudi] 5herhom commented on pull request #6031: [HUDI-4282] Repair IOException in some other dfs, except hdfs,when check block corrupted in HoodieLogFileReader

2022-09-16 Thread GitBox
5herhom commented on PR #6031: URL: https://github.com/apache/hudi/pull/6031#issuecomment-1249995668 > 2 minor comments. and I am assuming you have tested the patch in your env (CHDS) and its working as expected? LGTM The code has been updated. Please review again. Thank you. --

[GitHub] [hudi] fengjian428 commented on a diff in pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
fengjian428 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r973537469 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartialUpdateAvroPayload.java: ## @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hudi] fengjian428 commented on a diff in pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
fengjian428 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r973537469 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartialUpdateAvroPayload.java: ## @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hudi] hudi-bot commented on pull request #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6702: URL: https://github.com/apache/hudi/pull/6702#issuecomment-1249994987 ## CI report: * 3fd138c9b0f6a49c0c231a7c9bb17d8cc90b4c4c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5573: [HUDI-4093] fix NPE when insert records that partition column is null…

2022-09-16 Thread GitBox
hudi-bot commented on PR #5573: URL: https://github.com/apache/hudi/pull/5573#issuecomment-1249994728 ## CI report: * a41a8e706361e06594d4dae042c838a243016c64 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6702: URL: https://github.com/apache/hudi/pull/6702#issuecomment-1249994205 ## CI report: * 3fd138c9b0f6a49c0c231a7c9bb17d8cc90b4c4c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5573: [HUDI-4093] fix NPE when insert records that partition column is null…

2022-09-16 Thread GitBox
hudi-bot commented on PR #5573: URL: https://github.com/apache/hudi/pull/5573#issuecomment-1249993976 ## CI report: * a41a8e706361e06594d4dae042c838a243016c64 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1249993867 ## CI report: * 5944f5cbe9ce73fe6b7e27a0d381eaeb80dead38 UNKNOWN * 4ef7b451c3dd795906f3f68571256baeb330a59f UNKNOWN * 6aeb3d0d8f09aeab2a5766cf9d25ecb30537 UNKNOWN *

[GitHub] [hudi] brightwon commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
brightwon commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1249993843 It could be a spring shell issue! In order to clearly understand this problem, I need to log and check all the parameters passed from the cli. If I can test in my local machine, it

[GitHub] [hudi] hudi-bot commented on pull request #6703: [HUDI-4866] Fixes the issue that the archive operation is invalid whe…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6703: URL: https://github.com/apache/hudi/pull/6703#issuecomment-1249993405 ## CI report: * a08d4c149dd04fafb68d0cac6f4d2bbe7da3a520 Azure:

[GitHub] [hudi] yihua commented on pull request #6525: [HUDI-4237] should not sync partition parameters when create non-partition table in spark

2022-09-16 Thread GitBox
yihua commented on PR #6525: URL: https://github.com/apache/hudi/pull/6525#issuecomment-1249990712 @dujl It's likely due to CI flakiness. Could you rebase this PR on the latest master? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Created] (HUDI-4867) Fixed the issue that compaction is invalid when the last commit action is replace commit.

2022-09-16 Thread YangXuan (Jira)
YangXuan created HUDI-4867: -- Summary: Fixed the issue that compaction is invalid when the last commit action is replace commit. Key: HUDI-4867 URL: https://issues.apache.org/jira/browse/HUDI-4867 Project:

[GitHub] [hudi] yihua commented on a diff in pull request #6670: [HUDI-4842] Support compression strategy based on delte file length

2022-09-16 Thread GitBox
yihua commented on code in PR #6670: URL: https://github.com/apache/hudi/pull/6670#discussion_r973534451 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java: ## @@ -106,6 +106,12 @@ public class HoodieCompactionConfig extends

[GitHub] [hudi] yihua commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
yihua commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1249989632 > @yihua @codope I have found the cause of the above ci failed. > > If an empty string is passed to the `duplicatedPartitionPath` parameter of the **repair deduplicate** command, the

[GitHub] [hudi] yihua commented on a diff in pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
yihua commented on code in PR #6349: URL: https://github.com/apache/hudi/pull/6349#discussion_r973533659 ## hudi-cli/src/test/java/org/apache/hudi/cli/integ/ITTestRepairsCommand.java: ## @@ -102,9 +104,12 @@ public void init() throws Exception {

[GitHub] [hudi] brightwon commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
brightwon commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1249989142 @yihua @codope I have found the cause of the above ci failed. If an empty string is passed to the `duplicatedPartitionPath` parameter of the **repair deduplicate** command, the

[GitHub] [hudi] dujl commented on pull request #6525: [HUDI-4237] should not sync partition parameters when create non-partition table in spark

2022-09-16 Thread GitBox
dujl commented on PR #6525: URL: https://github.com/apache/hudi/pull/6525#issuecomment-1249988753 ok,i will fix it Replied Message From

[GitHub] [hudi] brightwon commented on a diff in pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
brightwon commented on code in PR #6349: URL: https://github.com/apache/hudi/pull/6349#discussion_r973533368 ## hudi-cli/src/test/java/org/apache/hudi/cli/integ/ITTestRepairsCommand.java: ## @@ -102,9 +104,12 @@ public void init() throws Exception {

[GitHub] [hudi] watermelon12138 commented on pull request #5573: [HUDI-4093] fix NPE when insert records that partition column is null…

2022-09-16 Thread GitBox
watermelon12138 commented on PR #5573: URL: https://github.com/apache/hudi/pull/5573#issuecomment-1249987019 > high level question/thoughts. is this about partition column value being null or partition path field config value set to null? also, can we ensure we look at all write paths?

[GitHub] [hudi] hudi-bot commented on pull request #6703: [HUDI-4866] Fixes the issue that the archive operation is invalid whe…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6703: URL: https://github.com/apache/hudi/pull/6703#issuecomment-1249986385 ## CI report: * a08d4c149dd04fafb68d0cac6f4d2bbe7da3a520 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6031: [HUDI-4282] Repair IOException in some other dfs, except hdfs,when check block corrupted in HoodieLogFileReader

2022-09-16 Thread GitBox
hudi-bot commented on PR #6031: URL: https://github.com/apache/hudi/pull/6031#issuecomment-1249985588 ## CI report: * f3a3f4b9e27b5daaeca40070626a80c7e80bd479 Azure:

[jira] [Updated] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4866: - Labels: pull-request-available (was: ) > Fixes the issue that the archive operation is invalid

[GitHub] [hudi] watermelon12138 opened a new pull request, #6703: [HUDI-4866] Fixes the issue that the archive operation is invalid whe…

2022-09-16 Thread GitBox
watermelon12138 opened a new pull request, #6703: URL: https://github.com/apache/hudi/pull/6703 ### Change Logs The result of filter will be NOT. Therefore, an instant whose value is less than firstSavepoint is selected for archiving. The expected result is that an instant

[GitHub] [hudi] hudi-bot commented on pull request #6031: [HUDI-4282] Repair IOException in some other dfs, except hdfs,when check block corrupted in HoodieLogFileReader

2022-09-16 Thread GitBox
hudi-bot commented on PR #6031: URL: https://github.com/apache/hudi/pull/6031#issuecomment-1249984908 ## CI report: * f3a3f4b9e27b5daaeca40070626a80c7e80bd479 Azure:

[GitHub] [hudi] XuQianJin-Stars commented on pull request #6525: [HUDI-4237] should not sync partition parameters when create non-partition table in spark

2022-09-16 Thread GitBox
XuQianJin-Stars commented on PR #6525: URL: https://github.com/apache/hudi/pull/6525#issuecomment-1249984566 hi @dujl the ci is failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #6697: [HUDI-3478] Spark CDC Write

2022-09-16 Thread GitBox
hudi-bot commented on PR #6697: URL: https://github.com/apache/hudi/pull/6697#issuecomment-1249984311 ## CI report: * 60b5ed8092b288f8ef11887ca5e438ebf60669b8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6031: [HUDI-4282] Repair IOException in some other dfs, except hdfs,when check block corrupted in HoodieLogFileReader

2022-09-16 Thread GitBox
hudi-bot commented on PR #6031: URL: https://github.com/apache/hudi/pull/6031#issuecomment-1249984154 ## CI report: * f3a3f4b9e27b5daaeca40070626a80c7e80bd479 Azure:

[jira] [Updated] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4866: --- Description: When a savepoint exists, the archiving condition is incorrect.

[jira] [Updated] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4866: --- Attachment: image-2022-09-17-10-23-03-977.png > Fixes the issue that the archive operation is invalid when

[jira] [Updated] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4866: --- Fix Version/s: 0.12.0 > Fixes the issue that the archive operation is invalid when the savepoint is >

[jira] [Updated] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4866: --- Affects Version/s: 0.12.0 0.11.0 > Fixes the issue that the archive operation is

[jira] [Created] (HUDI-4866) Fixes the issue that the archive operation is invalid when the savepoint is created.

2022-09-16 Thread YangXuan (Jira)
YangXuan created HUDI-4866: -- Summary: Fixes the issue that the archive operation is invalid when the savepoint is created. Key: HUDI-4866 URL: https://issues.apache.org/jira/browse/HUDI-4866 Project: Apache

[GitHub] [hudi] hudi-bot commented on pull request #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6702: URL: https://github.com/apache/hudi/pull/6702#issuecomment-1249974841 ## CI report: * 3fd138c9b0f6a49c0c231a7c9bb17d8cc90b4c4c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6702: URL: https://github.com/apache/hudi/pull/6702#issuecomment-1249973950 ## CI report: * 3fd138c9b0f6a49c0c231a7c9bb17d8cc90b4c4c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6536: [HUDI-4736] Fix inflight clean action preventing clean service to continue when multiple cleans are not allowed

2022-09-16 Thread GitBox
hudi-bot commented on PR #6536: URL: https://github.com/apache/hudi/pull/6536#issuecomment-1249973885 ## CI report: * dc3daf9826dea5c5b2c09dec9e2b9b0f08048c16 Azure:

[hudi] branch master updated: [HUDI-4864] Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations with test (#6688)

2022-09-16 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 21cbfce617 [HUDI-4864] Fix

[GitHub] [hudi] yihua merged pull request #6688: [HUDI-4864] Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations

2022-09-16 Thread GitBox
yihua merged PR #6688: URL: https://github.com/apache/hudi/pull/6688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on a diff in pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
yihua commented on code in PR #6349: URL: https://github.com/apache/hudi/pull/6349#discussion_r973524308 ## hudi-cli/src/test/java/org/apache/hudi/cli/integ/ITTestRepairsCommand.java: ## @@ -102,9 +104,12 @@ public void init() throws Exception {

[GitHub] [hudi] hudi-bot commented on pull request #6536: [HUDI-4736] Fix inflight clean action preventing clean service to continue when multiple cleans are not allowed

2022-09-16 Thread GitBox
hudi-bot commented on PR #6536: URL: https://github.com/apache/hudi/pull/6536#issuecomment-1249973106 ## CI report: * dc3daf9826dea5c5b2c09dec9e2b9b0f08048c16 Azure:

[jira] [Updated] (HUDI-4865) Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity

2022-09-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4865: - Labels: pull-request-available (was: ) > Optimize HoodieAvroUtils#isMetadataField to use O(1)

[jira] [Updated] (HUDI-4865) Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity

2022-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4865: - Fix Version/s: 0.12.1 > Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity >

[GitHub] [hudi] danny0405 opened a new pull request, #6702: [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) comp…

2022-09-16 Thread GitBox
danny0405 opened a new pull request, #6702: URL: https://github.com/apache/hudi/pull/6702 …lexity ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change

[GitHub] [hudi] hudi-bot commented on pull request #6677: [HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

2022-09-16 Thread GitBox
hudi-bot commented on PR #6677: URL: https://github.com/apache/hudi/pull/6677#issuecomment-1249972309 ## CI report: * 146f2f91d95aebce0d958685bcbf65fd6c8b0f04 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5933: [HUDI-4293] Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-09-16 Thread GitBox
hudi-bot commented on PR #5933: URL: https://github.com/apache/hudi/pull/5933#issuecomment-1249972137 ## CI report: * 6bcd009993e304b5bbe15eabc062f4a5e5ed74ee Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

2022-09-16 Thread GitBox
yihua commented on code in PR #6669: URL: https://github.com/apache/hudi/pull/6669#discussion_r973522477 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java: ## @@ -214,13 +213,7 @@ public FileInputSplit[]

[GitHub] [hudi] yihua commented on pull request #6536: [HUDI-4736] Fix inflight clean action preventing clean service to continue when multiple cleans are not allowed

2022-09-16 Thread GitBox
yihua commented on PR #6536: URL: https://github.com/apache/hudi/pull/6536#issuecomment-1249969384 @nsivabalan I fixed the test which covers the change logic around allowing multiple cleans. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Created] (HUDI-4865) Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity

2022-09-16 Thread Danny Chen (Jira)
Danny Chen created HUDI-4865: Summary: Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity Key: HUDI-4865 URL: https://issues.apache.org/jira/browse/HUDI-4865 Project: Apache Hudi

[GitHub] [hudi] danny0405 commented on a diff in pull request #4676: [HUDI-3304] Support partial update payload

2022-09-16 Thread GitBox
danny0405 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r973518456 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartialUpdateAvroPayload.java: ## @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

2022-09-16 Thread GitBox
hudi-bot commented on PR #6669: URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249960054 ## CI report: * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #6630: [HUDI-4808] Fix HoodieSimpleBucketIndex not consider bucket num in lo…

2022-09-16 Thread GitBox
danny0405 commented on code in PR #6630: URL: https://github.com/apache/hudi/pull/6630#discussion_r973515665 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/HoodieSimpleBucketIndex.java: ## @@ -52,10 +54,20 @@ private Map

[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

2022-09-16 Thread GitBox
hudi-bot commented on PR #6669: URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249958394 ## CI report: * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6662: [HUDI-4832] Fix drop partition meta sync

2022-09-16 Thread GitBox
hudi-bot commented on PR #6662: URL: https://github.com/apache/hudi/pull/6662#issuecomment-1249958377 ## CI report: * a6b676c8a983aad9ef485d73ec1dc7dd462a055a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6349: [HUDI-4433] Hudi-CLI repair deduplicate not working with non-partitio…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6349: URL: https://github.com/apache/hudi/pull/6349#issuecomment-1249958197 ## CI report: * 9f7d93d6247a1b523b6f52c41bd2bc2f5f1ed23e Azure:

[jira] [Updated] (HUDI-4856) Missing option for HoodieCatalogFactory

2022-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4856: - Fix Version/s: 0.12.1 > Missing option for HoodieCatalogFactory > ---

[jira] [Commented] (HUDI-4856) Missing option for HoodieCatalogFactory

2022-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606021#comment-17606021 ] Danny Chen commented on HUDI-4856: -- Fixed via master branch: 36fe10a0b2018aaaedd8c6813c45d0dc26cf401d >

[GitHub] [hudi] hudi-bot commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-09-16 Thread GitBox
hudi-bot commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1249958059 ## CI report: * 5a16d35ec42bf86e5759ebb155cad40e83aba9f9 UNKNOWN * f2bb9e61707199197f30eef79e80db3e1241b3a0 Azure:

[jira] [Resolved] (HUDI-4856) Missing option for HoodieCatalogFactory

2022-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4856. -- > Missing option for HoodieCatalogFactory > --- > >

[GitHub] [hudi] danny0405 merged pull request #6693: [HUDI-4856] Missing option for HoodieCatalogFactory

2022-09-16 Thread GitBox
danny0405 merged PR #6693: URL: https://github.com/apache/hudi/pull/6693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (3faddb7da0 -> 36fe10a0b2)

2022-09-16 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 3faddb7da0 [HUDI-4813] Fix infer keygen not work in sparksql side issue (#6634) add 36fe10a0b2 [HUDI-4856]

[GitHub] [hudi] hudi-bot commented on pull request #6700: [HUDI-4851] Handle UTF8String when match InSet for dataskipping

2022-09-16 Thread GitBox
hudi-bot commented on PR #6700: URL: https://github.com/apache/hudi/pull/6700#issuecomment-1249956732 ## CI report: * fbbe63e493acf95b9d74f1a1bc9133646cd64812 Azure:

[GitHub] [hudi] voonhous commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

2022-09-16 Thread GitBox
voonhous commented on code in PR #6669: URL: https://github.com/apache/hudi/pull/6669#discussion_r973513763 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java: ## @@ -214,13 +213,7 @@ public FileInputSplit[]

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6196: [HUDI-4071] Enable schema reconciliation by default

2022-09-16 Thread GitBox
alexeykudinkin commented on code in PR #6196: URL: https://github.com/apache/hudi/pull/6196#discussion_r973511372 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieCommonConfig.java: ## @@ -38,7 +38,7 @@ public class HoodieCommonConfig extends HoodieConfig {

[GitHub] [hudi] yihua commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

2022-09-16 Thread GitBox
yihua commented on code in PR #6669: URL: https://github.com/apache/hudi/pull/6669#discussion_r973507946 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java: ## @@ -214,13 +213,7 @@ public FileInputSplit[]

[GitHub] [hudi] hudi-bot commented on pull request #6688: [HUDI-4864] Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations

2022-09-16 Thread GitBox
hudi-bot commented on PR #6688: URL: https://github.com/apache/hudi/pull/6688#issuecomment-1249932808 ## CI report: * fff1405467fb5f6a7fdb6d3d043714e268f1c875 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
hudi-bot commented on PR #6580: URL: https://github.com/apache/hudi/pull/6580#issuecomment-1249932711 ## CI report: * ff98ae0dda69ee611e4814fbae9c8ddc0a93a4f1 UNKNOWN * c292838205bb8eb57c529808c6b6da98635ac17d Azure:

[jira] [Updated] (HUDI-4864) Fix AWSDmsAvroPayload during delete operations with MOR snapshot query

2022-09-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4864: - Labels: pull-request-available (was: ) > Fix AWSDmsAvroPayload during delete operations with MOR

[GitHub] [hudi] hudi-bot commented on pull request #6688: [HUDI-4864] Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations

2022-09-16 Thread GitBox
hudi-bot commented on PR #6688: URL: https://github.com/apache/hudi/pull/6688#issuecomment-1249930871 ## CI report: * fff1405467fb5f6a7fdb6d3d043714e268f1c875 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
hudi-bot commented on PR #6580: URL: https://github.com/apache/hudi/pull/6580#issuecomment-1249930760 ## CI report: * ff98ae0dda69ee611e4814fbae9c8ddc0a93a4f1 UNKNOWN * c292838205bb8eb57c529808c6b6da98635ac17d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-09-16 Thread GitBox
hudi-bot commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1249928256 ## CI report: * 5a16d35ec42bf86e5759ebb155cad40e83aba9f9 UNKNOWN * 988e4874af3065d6879f9adc40c7483a84467f72 Azure:

[jira] [Created] (HUDI-4864) Fix AWSDmsAvroPayload during delete operations with MOR snapshot query

2022-09-16 Thread Rahil Chertara (Jira)
Rahil Chertara created HUDI-4864: Summary: Fix AWSDmsAvroPayload during delete operations with MOR snapshot query Key: HUDI-4864 URL: https://issues.apache.org/jira/browse/HUDI-4864 Project: Apache

[GitHub] [hudi] parisni commented on pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
parisni commented on PR #6580: URL: https://github.com/apache/hudi/pull/6580#issuecomment-1249917468 @nsivabalan I applied your review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] rahil-c commented on pull request #6688: Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations

2022-09-16 Thread GitBox
rahil-c commented on PR #6688: URL: https://github.com/apache/hudi/pull/6688#issuecomment-1249916684 @yihua Was able to add test which exposed the constructor issue as well as the same `combineAndGetValue` issue that aws customer saw ``` Caused by:

[GitHub] [hudi] tarunguptanit closed issue #6174: Hudi Read Performance : Partition pruning not happening when reading Hudi table

2022-09-16 Thread GitBox
tarunguptanit closed issue #6174: Hudi Read Performance : Partition pruning not happening when reading Hudi table URL: https://github.com/apache/hudi/issues/6174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
hudi-bot commented on PR #6580: URL: https://github.com/apache/hudi/pull/6580#issuecomment-1249906693 ## CI report: * ff98ae0dda69ee611e4814fbae9c8ddc0a93a4f1 UNKNOWN * c292838205bb8eb57c529808c6b6da98635ac17d Azure:

[GitHub] [hudi] parisni commented on a diff in pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
parisni commented on code in PR #6580: URL: https://github.com/apache/hudi/pull/6580#discussion_r973465697 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -735,6 +735,34 @@ public final Stream getAllFileGroups(String

[GitHub] [hudi] hudi-bot commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-09-16 Thread GitBox
hudi-bot commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1249889804 ## CI report: * 5a16d35ec42bf86e5759ebb155cad40e83aba9f9 UNKNOWN * 7300c9eb17c30e11aaeb9cd768b15585536ab5f9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6693: [HUDI-4856] Missing option for HoodieCatalogFactory

2022-09-16 Thread GitBox
hudi-bot commented on PR #6693: URL: https://github.com/apache/hudi/pull/6693#issuecomment-1249869868 ## CI report: * d543dcbb0210ad1e798af374efe7bae79065bbde Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6694: [DO NOT MERGE][HUDI-4855] Add missing table configs for bootstrap in Deltastreamer

2022-09-16 Thread GitBox
hudi-bot commented on PR #6694: URL: https://github.com/apache/hudi/pull/6694#issuecomment-1249869916 ## CI report: * 5a6eed936fc08b943370db12c258ea6e75430912 Azure:

[GitHub] [hudi] parisni commented on a diff in pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
parisni commented on code in PR #6580: URL: https://github.com/apache/hudi/pull/6580#discussion_r973441831 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java: ## @@ -110,9 +112,15 @@ HoodieCleanerPlan

[GitHub] [hudi] parisni commented on a diff in pull request #6580: [HUDI-4792] Batch clean files to delete

2022-09-16 Thread GitBox
parisni commented on code in PR #6580: URL: https://github.com/apache/hudi/pull/6580#discussion_r973439309 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java: ## @@ -110,9 +112,15 @@ HoodieCleanerPlan

[GitHub] [hudi] CTTY commented on a diff in pull request #5113: [HUDI-3625] [RFC-60] Optimized storage layout for Cloud Object Stores

2022-09-16 Thread GitBox
CTTY commented on code in PR #5113: URL: https://github.com/apache/hudi/pull/5113#discussion_r973438573 ## rfc/rfc-56/rfc-56.md: ## @@ -0,0 +1,226 @@ + + +# RFC-56: Federated Storage Layer + +## Proposers +- @umehrot2 + +## Approvers +- @vinoth +- @shivnarayan + +## Status +

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6361: [WIP][HUDI-4690][HUDI-4503][Stacked on 6550] Cleaning up Hudi custom Spark `Rule`s

2022-09-16 Thread GitBox
alexeykudinkin commented on code in PR #6361: URL: https://github.com/apache/hudi/pull/6361#discussion_r971019994 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala: ## @@ -65,35 +68,33 @@ object HoodieSparkUtils extends SparkAdapterSupport

[GitHub] [hudi] parisni closed pull request #6537: [HUDI-4762] Avoid update metastore schema if only missing column in input

2022-09-16 Thread GitBox
parisni closed pull request #6537: [HUDI-4762] Avoid update metastore schema if only missing column in input URL: https://github.com/apache/hudi/pull/6537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] parisni commented on pull request #6537: [HUDI-4762] Avoid update metastore schema if only missing column in input

2022-09-16 Thread GitBox
parisni commented on PR #6537: URL: https://github.com/apache/hudi/pull/6537#issuecomment-1249828151 Agreed, I will close this On Thu, 2022-09-15 at 22:47 -0700, Shiyan Xu wrote: > @xushiyan commented on this pull request. > > > > > @@ -286,7 +286,11 @@ private

[jira] [Created] (HUDI-4863) Deprecate `hoodie.compaction.payload.class` and re-use hoodie.datasource.write.payload.class

2022-09-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4863: - Summary: Deprecate `hoodie.compaction.payload.class` and re-use hoodie.datasource.write.payload.class Key: HUDI-4863 URL:

[GitHub] [hudi] hudi-bot commented on pull request #6630: [HUDI-4808] Fix HoodieSimpleBucketIndex not consider bucket num in lo…

2022-09-16 Thread GitBox
hudi-bot commented on PR #6630: URL: https://github.com/apache/hudi/pull/6630#issuecomment-1249800719 ## CI report: * 8746849f96d764a6203009df3f856ef564ff0931 Azure:

[jira] [Updated] (HUDI-4862) Add an option leverage RDDCustomColumnsSortPartitioner in bulk-insert

2022-09-16 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4862: -- Component/s: deltastreamer > Add an option leverage RDDCustomColumnsSortPartitioner in

[jira] [Created] (HUDI-4862) Add an option leverage RDDCustomColumnsSortPartitioner in bulk-insert

2022-09-16 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-4862: - Summary: Add an option leverage RDDCustomColumnsSortPartitioner in bulk-insert Key: HUDI-4862 URL: https://issues.apache.org/jira/browse/HUDI-4862 Project: Apache

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6516: [HUDI-4729] Fix fq can not be queried in pending compaction when query ro table with spark

2022-09-16 Thread GitBox
alexeykudinkin commented on code in PR #6516: URL: https://github.com/apache/hudi/pull/6516#discussion_r973345568 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -665,13 +665,25 @@ public final Stream

  1   2   3   >