[GitHub] [hudi] yihua commented on a diff in pull request #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
yihua commented on code in PR #8156: URL: https://github.com/apache/hudi/pull/8156#discussion_r1133047376 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java: ## @@ -533,7 +533,19 @@ private List

[GitHub] [hudi] hudi-bot commented on pull request #8157: [HUDI-5920] Improve documentation of parallelism configs

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8157: URL: https://github.com/apache/hudi/pull/8157#issuecomment-1464849404 ## CI report: * 5075feb0a984758ac4dc2999bf503d0df3b1dbd1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8157: [HUDI-5920] Improve documentation of parallelism configs

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8157: URL: https://github.com/apache/hudi/pull/8157#issuecomment-1464848306 ## CI report: * 5075feb0a984758ac4dc2999bf503d0df3b1dbd1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8157: [HUDI-5920] Improve documentation of parallelism configs

2023-03-10 Thread via GitHub
nsivabalan commented on code in PR #8157: URL: https://github.com/apache/hudi/pull/8157#discussion_r1133042083 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -247,13 +247,29 @@ public class HoodieWriteConfig extends

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
nsivabalan commented on code in PR #8156: URL: https://github.com/apache/hudi/pull/8156#discussion_r1133041781 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java: ## @@ -533,7 +533,19 @@ private List

[jira] [Updated] (HUDI-5920) Improve documentation of parallelism configs

2023-03-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5920: - Labels: pull-request-available (was: ) > Improve documentation of parallelism configs >

[GitHub] [hudi] yihua opened a new pull request, #8157: [HUDI-5920] Improve documentation of parallelism configs

2023-03-10 Thread via GitHub
yihua opened a new pull request, #8157: URL: https://github.com/apache/hudi/pull/8157 ### Change Logs This PR improves the documentation for the following parallelism configs: ``` hoodie.archive.delete.parallelism hoodie.bloom.index.parallelism

[jira] [Updated] (HUDI-5920) Improve documentation of parallelism configs

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5920: Story Points: 1 > Improve documentation of parallelism configs >

[jira] [Assigned] (HUDI-5920) Improve documentation of parallelism configs

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-5920: --- Assignee: Ethan Guo > Improve documentation of parallelism configs >

[jira] [Created] (HUDI-5920) Improve documentation of parallelism configs

2023-03-10 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-5920: --- Summary: Improve documentation of parallelism configs Key: HUDI-5920 URL: https://issues.apache.org/jira/browse/HUDI-5920 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-5920) Improve documentation of parallelism configs

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5920: Fix Version/s: 0.13.1 > Improve documentation of parallelism configs >

[GitHub] [hudi] hudi-bot commented on pull request #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8156: URL: https://github.com/apache/hudi/pull/8156#issuecomment-1464819336 ## CI report: * c2494f74ce709a29f512fc165dfe1b12b99aa244 Azure:

[GitHub] [hudi] BalaMahesh commented on issue #7595: [SUPPORT] Hudi Clean and Delta commits taking ~50 mins to finish frequently

2023-03-10 Thread via GitHub
BalaMahesh commented on issue #7595: URL: https://github.com/apache/hudi/issues/7595#issuecomment-1464816071 > > We were running with > > hoodie.metadata.index.bloom.filter.enable=false hoodie.metadata.index.column.stats.enable=false > > I hope that answers your question. We have set

[GitHub] [hudi] danny0405 commented on pull request #7955: [HUDI-5649] Unify all the loggers to slf4j

2023-03-10 Thread via GitHub
danny0405 commented on PR #7955: URL: https://github.com/apache/hudi/pull/7955#issuecomment-1464810336 > Few questions : > > 1. Can you give me an example of these conflicts? (GH issues or slack threads) > 2. How does this affect bundling? we bundle slf4j? we need to think if

[GitHub] [hudi] danny0405 commented on issue #8071: [SUPPORT]How to improve the speed of Flink writing to hudi ?

2023-03-10 Thread via GitHub
danny0405 commented on issue #8071: URL: https://github.com/apache/hudi/issues/8071#issuecomment-1464808238 ![image](https://user-images.githubusercontent.com/7644508/224461977-513298fb-10f3-4a52-90b0-b8b20ba6b0aa.png) From the DAG of this diagram, it seems you are using the MOR table

[GitHub] [hudi] danny0405 commented on issue #8153: [SUPPORT] Async Clustering failing for MoR in 0.13.0

2023-03-10 Thread via GitHub
danny0405 commented on issue #8153: URL: https://github.com/apache/hudi/issues/8153#issuecomment-1464807143 Seems a jar conflict, cc @umehrot2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] danny0405 commented on issue #8118: [SUPPORT] error in run_sync_tool.sh

2023-03-10 Thread via GitHub
danny0405 commented on issue #8118: URL: https://github.com/apache/hudi/issues/8118#issuecomment-1464806799 > Caused by: java.sql.SQLException: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.AnalysisException: Cannot persist persis.tempstream_hudi

[GitHub] [hudi] danny0405 commented on issue #8144: [SUPPORT]Unable to connect to an s3 hudi table

2023-03-10 Thread via GitHub
danny0405 commented on issue #8144: URL: https://github.com/apache/hudi/issues/8144#issuecomment-1464804750 Seems a permission setup issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Closed] (HUDI-5909) Reuse hive client if possible

2023-03-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-5909. Fix Version/s: 0.13.1 0.14.0 Resolution: Fixed Fixed via master branch:

[hudi] branch master updated (5416f001dc7 -> 9c44339e4bb)

2023-03-10 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 5416f001dc7 [HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication (#8150) add 9c44339e4bb

[GitHub] [hudi] danny0405 merged pull request #8139: [HUDI-5909] Reuse hive client if possible

2023-03-10 Thread via GitHub
danny0405 merged PR #8139: URL: https://github.com/apache/hudi/pull/8139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Closed] (HUDI-5917) MOR table log file has only one replication

2023-03-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-5917. Fix Version/s: 0.14.0 Resolution: Fixed Fixed via master branch:

[hudi] branch master updated (84fcc3851cc -> 5416f001dc7)

2023-03-10 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 84fcc3851cc [HUDI-5913] Table can not read correctly when computed column is in the midst (#8154) add

[GitHub] [hudi] danny0405 merged pull request #8150: [HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication

2023-03-10 Thread via GitHub
danny0405 merged PR #8150: URL: https://github.com/apache/hudi/pull/8150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Closed] (HUDI-5913) Table can not read correctly when computed column is in the midst

2023-03-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-5913. Fix Version/s: 0.14.0 Resolution: Fixed Fixed via master branch:

[hudi] branch master updated (6589ebe8486 -> 84fcc3851cc)

2023-03-10 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 6589ebe8486 [HUDI-5904] support more than one update actions in merge into table (#8133) add 84fcc3851cc

[GitHub] [hudi] danny0405 merged pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
danny0405 merged PR #8154: URL: https://github.com/apache/hudi/pull/8154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
danny0405 commented on PR #8154: URL: https://github.com/apache/hudi/pull/8154#issuecomment-1464802437 The failed test case should not be caused by this patch:

[hudi] branch master updated: [HUDI-5904] support more than one update actions in merge into table (#8133)

2023-03-10 Thread forwardxu
This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6589ebe8486 [HUDI-5904] support more than one

[GitHub] [hudi] XuQianJin-Stars merged pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
XuQianJin-Stars merged PR #8133: URL: https://github.com/apache/hudi/pull/8133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8156: URL: https://github.com/apache/hudi/pull/8156#issuecomment-1464746277 ## CI report: * c2494f74ce709a29f512fc165dfe1b12b99aa244 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8156: URL: https://github.com/apache/hudi/pull/8156#issuecomment-1464740665 ## CI report: * c2494f74ce709a29f512fc165dfe1b12b99aa244 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5919: - Labels: pull-request-available (was: ) > Fix the validation of partition listing in metadata

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5919: Description: In HoodieMetadataTableValidator, we compare the partition listing between MDT and file

[GitHub] [hudi] yihua opened a new pull request, #8156: [HUDI-5919] Fix the validation of partition listing in metadata table validator

2023-03-10 Thread via GitHub
yihua opened a new pull request, #8156: URL: https://github.com/apache/hudi/pull/8156 ### Change Logs In the following scenario, before this fix, the validation job fires a false alarm complaining that the partition list returned by the file system and the metadata table: -

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5919: Description: In HoodieMetadataTableValidator, we compare the partition listing between MDT and file

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5919: Priority: Blocker (was: Major) > Fix the validation of partition listing in metadata table validator >

[jira] [Assigned] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-5919: --- Assignee: Ethan Guo > Fix the validation of partition listing in metadata table validator >

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5919: Fix Version/s: 0.13.1 > Fix the validation of partition listing in metadata table validator >

[jira] [Updated] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5919: Story Points: 0.5 > Fix the validation of partition listing in metadata table validator >

[jira] [Created] (HUDI-5919) Fix the validation of partition listing in metadata table validator

2023-03-10 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-5919: --- Summary: Fix the validation of partition listing in metadata table validator Key: HUDI-5919 URL: https://issues.apache.org/jira/browse/HUDI-5919 Project: Apache Hudi

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1464477812 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * f51a45589d72a12c4e894e17bb90a56bca44fca6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1464219994 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * f51a45589d72a12c4e894e17bb90a56bca44fca6 Azure:

[GitHub] [hudi] xuzifu666 commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
xuzifu666 commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1464216645 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1464204373 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * f51a45589d72a12c4e894e17bb90a56bca44fca6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1464184319 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 89868d4c16c65b4d118c4b061b1db8232e5abc34 UNKNOWN * 0c5dea833612c53880831dccb652494f1a00c192 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8101: [HUDI-5879] Extends evaluators to support evaluate based on column values

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8101: URL: https://github.com/apache/hudi/pull/8101#issuecomment-1464112889 ## CI report: * 5e06622760d679a66c9f9b17028bf69fb2ffab93 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8154: URL: https://github.com/apache/hudi/pull/8154#issuecomment-1464031652 ## CI report: * 6aa5e494711eeced312b35b2042a0a21bf1a877e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8150: [HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8150: URL: https://github.com/apache/hudi/pull/8150#issuecomment-1464031570 ## CI report: * b822947584be483fcc23fd1880d2212f31ae386d UNKNOWN * 2cd528589a68a035631545bfb5df9a56d7c0148b Azure:

[GitHub] [hudi] nsivabalan closed issue #8151: [SUPPORT]Could not compact /.hoodie/metadata in 0.13.0

2023-03-10 Thread via GitHub
nsivabalan closed issue #8151: [SUPPORT]Could not compact /.hoodie/metadata in 0.13.0 URL: https://github.com/apache/hudi/issues/8151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] nsivabalan commented on issue #8151: [SUPPORT]Could not compact /.hoodie/metadata in 0.13.0

2023-03-10 Thread via GitHub
nsivabalan commented on issue #8151: URL: https://github.com/apache/hudi/issues/8151#issuecomment-1464012063 its a known limitation that MDT may not work w/ consistent hashing index. we did call out in our 0.13.0 release docs

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463803982 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * f51a45589d72a12c4e894e17bb90a56bca44fca6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463793039 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * f51a45589d72a12c4e894e17bb90a56bca44fca6 UNKNOWN Bot commands @hudi-bot supports the

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463791944 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 89868d4c16c65b4d118c4b061b1db8232e5abc34 UNKNOWN * 67e4d83e617d62ef77f3f1dae524dac486769cc2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463784272 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
XuQianJin-Stars commented on code in PR #8133: URL: https://github.com/apache/hudi/pull/8133#discussion_r1132353849 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -19,7 +19,8 @@ package org.apache.spark.sql.hudi

[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
XuQianJin-Stars commented on code in PR #8133: URL: https://github.com/apache/hudi/pull/8133#discussion_r1132353849 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -19,7 +19,8 @@ package org.apache.spark.sql.hudi

[GitHub] [hudi] xuzifu666 commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
xuzifu666 commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463779687 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] xuzifu666 commented on a diff in pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
xuzifu666 commented on code in PR #8133: URL: https://github.com/apache/hudi/pull/8133#discussion_r1132350431 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -115,6 +116,65 @@ class TestMergeIntoTable extends

[GitHub] [hudi] leesf commented on a diff in pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
leesf commented on code in PR #8133: URL: https://github.com/apache/hudi/pull/8133#discussion_r1132348375 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -115,6 +116,65 @@ class TestMergeIntoTable extends

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463737743 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 89868d4c16c65b4d118c4b061b1db8232e5abc34 UNKNOWN * 67e4d83e617d62ef77f3f1dae524dac486769cc2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463727525 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 3420bb3b6ef5fbdf76aa594e00a3a6764d98e955 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463719652 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 3420bb3b6ef5fbdf76aa594e00a3a6764d98e955 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463673303 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 3420bb3b6ef5fbdf76aa594e00a3a6764d98e955 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8149: [HUDI-5915] Fixed load ckpMeatadata error when using minio

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8149: URL: https://github.com/apache/hudi/pull/8149#issuecomment-1463665768 ## CI report: * 64fff59128deb511ed29c4ac7972345e6dab1bd7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463664474 ## CI report: * 41d7a2a249bf0c75491790a7aafc95761e9d49d4 UNKNOWN * 3420bb3b6ef5fbdf76aa594e00a3a6764d98e955 Azure:

[jira] [Commented] (HUDI-5824) COMBINE_BEFORE_UPSERT=false option does not work for upsert

2023-03-10 Thread kazdy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698867#comment-17698867 ] kazdy commented on HUDI-5824: - [~xushiyan] I see that you marked this as critical, but after having a

[GitHub] [hudi] pete91z commented on issue #8118: [SUPPORT] error in run_sync_tool.sh

2023-03-10 Thread via GitHub
pete91z commented on issue #8118: URL: https://github.com/apache/hudi/issues/8118#issuecomment-1463613215 Workaround I'm using at the moment is to create the table in spark-sql, but omitting the tblproperties clause: CREATE EXTERNAL TABLE IF NOT EXISTS persis.tempstream_hudi(

[GitHub] [hudi] hudi-bot commented on pull request #8101: [HUDI-5879] Extends evaluators to support evaluate based on column values

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8101: URL: https://github.com/apache/hudi/pull/8101#issuecomment-1463602325 ## CI report: * ff2a6fa29a3ea2b46532420f218859f6e59f10de Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8101: [HUDI-5879] Extends evaluators to support evaluate based on column values

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8101: URL: https://github.com/apache/hudi/pull/8101#issuecomment-1463593084 ## CI report: * ff2a6fa29a3ea2b46532420f218859f6e59f10de Azure:

[GitHub] [hudi] caokz commented on issue #7047: [SUPPORT] HoodieFlinkCompactor with NoSuchMethodError: org.apache.hudi.org.apache.avro.specific.SpecificRecordBuilderBase

2023-03-10 Thread via GitHub
caokz commented on issue #7047: URL: https://github.com/apache/hudi/issues/7047#issuecomment-1463591667 I also encountered this problem and found that the reason is that the avro version used to package the hudi-common module and the avro version used to package the hudi-flink-bundle

[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8154: URL: https://github.com/apache/hudi/pull/8154#issuecomment-1463580844 ## CI report: * 6aa5e494711eeced312b35b2042a0a21bf1a877e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8154: URL: https://github.com/apache/hudi/pull/8154#issuecomment-1463566835 ## CI report: * 6aa5e494711eeced312b35b2042a0a21bf1a877e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-5917) MOR table log file has only one replication

2023-03-10 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-5917: - Fix Version/s: 0.13.1 0.12.3 > MOR table log file has only one replication >

[GitHub] [hudi] DavidZ1 commented on issue #8071: [SUPPORT]How to improve the speed of Flink writing to hudi ?

2023-03-10 Thread via GitHub
DavidZ1 commented on issue #8071: URL: https://github.com/apache/hudi/issues/8071#issuecomment-1463521133 Thanks ,we use `insert + COW + bucket Index`,so we the write throughput should be higt. In fact,we flink job of running , throughput was bad,i do not understand why ? -- This is an

[GitHub] [hudi] hudi-bot commented on pull request #8150: [HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8150: URL: https://github.com/apache/hudi/pull/8150#issuecomment-1463515581 ## CI report: * b822947584be483fcc23fd1880d2212f31ae386d UNKNOWN * 6dc5a2866114879b660baceae026bf8574126af3 Azure:

[GitHub] [hudi] danny0405 closed pull request #8098: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
danny0405 closed pull request #8098: [HUDI-5913] Table can not read correctly when computed column is in the midst URL: https://github.com/apache/hudi/pull/8098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #8150: [HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8150: URL: https://github.com/apache/hudi/pull/8150#issuecomment-1463506807 ## CI report: * b822947584be483fcc23fd1880d2212f31ae386d UNKNOWN * 6dc5a2866114879b660baceae026bf8574126af3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8139: [HUDI-5909] Reuse hive client if possible

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8139: URL: https://github.com/apache/hudi/pull/8139#issuecomment-1463506676 ## CI report: * 075563866d156e36afe34780d5fb132d6da57251 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463506582 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * 9795e86a4071d23307852ce59a09af8d5e41ffae Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463493476 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * 9795e86a4071d23307852ce59a09af8d5e41ffae Azure:

[GitHub] [hudi] XuQianJin-Stars commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
XuQianJin-Stars commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463492838 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HUDI-5781) Refactor other classes to use HoodieConfig/ConfigProperty

2023-03-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5781: - Labels: pull-request-available (was: ) > Refactor other classes to use

[GitHub] [hudi] lokeshj1703 opened a new pull request, #8155: [HUDI-5781] Refactor other classes to use HoodieConfig/ConfigProperty

2023-03-10 Thread via GitHub
lokeshj1703 opened a new pull request, #8155: URL: https://github.com/apache/hudi/pull/8155 ### Change Logs Relevant classes: SqlFileBasedTransformer SqlQueryBasedTransformer The following can be skipped FlinkClusteringConfig FlinkCompactionConfig

[hudi] branch master updated: [HUDI-5857] Insert overwrite into bucket table would generate new file group id (#8072)

2023-03-10 Thread forwardxu
This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 51d03515728 [HUDI-5857] Insert overwrite into

[GitHub] [hudi] XuQianJin-Stars merged pull request #8072: [HUDI-5857] Insert overwrite into bucket table would generate new file group id

2023-03-10 Thread via GitHub
XuQianJin-Stars merged PR #8072: URL: https://github.com/apache/hudi/pull/8072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] 1032851561 commented on pull request #8098: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
1032851561 commented on PR #8098: URL: https://github.com/apache/hudi/pull/8098#issuecomment-1463458585 > Thanks for the feedback, you are right, the test case passed because the computed column is the last column within the schema, I have created a patch and attach it here:

[GitHub] [hudi] 1032851561 opened a new pull request, #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
1032851561 opened a new pull request, #8154: URL: https://github.com/apache/hudi/pull/8154 ### Change Logs The computed column(eg : process_time as proctime() ) will cause a select exception because it is not included in the data file schema, so we need to ignore the computed column

[GitHub] [hudi] hudi-bot commented on pull request #8098: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8098: URL: https://github.com/apache/hudi/pull/8098#issuecomment-1463450662 ## CI report: * f742eac87e222999b532288b8ae733a69520b031 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463442675 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * 9795e86a4071d23307852ce59a09af8d5e41ffae Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8098: [HUDI-5913] Table can not read correctly when computed column is in the midst

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8098: URL: https://github.com/apache/hudi/pull/8098#issuecomment-1463442534 ## CI report: * f742eac87e222999b532288b8ae733a69520b031 Azure:

[GitHub] [hudi] haripriyarhp opened a new issue, #8153: [SUPPORT]

2023-03-10 Thread via GitHub
haripriyarhp opened a new issue, #8153: URL: https://github.com/apache/hudi/issues/8153 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

[GitHub] [hudi] hudi-bot commented on pull request #8133: [HUDI-5904] support more than one update actions in merge into table

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8133: URL: https://github.com/apache/hudi/pull/8133#issuecomment-1463434887 ## CI report: * 8e3fad5fa9e9c64e7e345a317865f6fe6a9a7620 UNKNOWN * a690c5122694914f975ebbb717e06630ac3b5902 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8107: [HUDI-5514] Adding auto generation of record keys support to Hudi

2023-03-10 Thread via GitHub
hudi-bot commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1463434776 ## CI report: * 9dfbe3e6135456e7f8c79513270eb5e7e4ed123d Azure:

[GitHub] [hudi] bvaradar commented on pull request #7680: [HUDI-5548] spark sql show | update hudi's table properties

2023-03-10 Thread via GitHub
bvaradar commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1463434033 @XuQianJin-Stars : Can you add tests for set/unset commands. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[hudi] branch master updated: [HUDI-5333] Ignore file system type of basePath when using RocksDbBasedFileSystemView (#7387)

2023-03-10 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8129a85eebb [HUDI-5333] Ignore file system type

[GitHub] [hudi] bvaradar merged pull request #7387: [HUDI-5333] Ignore file system type of basePath when using RocksDbBasedFileSystemView

2023-03-10 Thread via GitHub
bvaradar merged PR #7387: URL: https://github.com/apache/hudi/pull/7387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: