[GitHub] [hudi] danny0405 merged pull request #6306: [HUDI-4545] Do not modify the current record directly for OverwriteNo…

2022-08-05 Thread GitBox
danny0405 merged PR #6306: URL: https://github.com/apache/hudi/pull/6306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Resolved] (HUDI-4545) Do not modify the current record directly for OverwriteNonDefaultsWithLatestAvroPayload

2022-08-05 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4545. -- > Do not modify the current record directly for > OverwriteNonDefaultsWithLatestAvroPayload >

[jira] [Updated] (HUDI-4550) Investigate why rollback is triggered for completed instant

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4550: -- Fix Version/s: 0.13.0 > Investigate why rollback is triggered for completed instant >

[jira] [Assigned] (HUDI-4551) The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-4551: Assignee: Nicholas Jiang > The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS

[jira] [Created] (HUDI-4551) The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread Nicholas Jiang (Jira)
Nicholas Jiang created HUDI-4551: Summary: The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment Key: HUDI-4551 URL:

[GitHub] [hudi] garyli1019 merged pull request #6300: [HUDI-4544] support retain hour cleaning policy for flink

2022-08-05 Thread GitBox
garyli1019 merged PR #6300: URL: https://github.com/apache/hudi/pull/6300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (e03cd0a198 -> fcdd4cf06c)

2022-08-05 Thread garyli
This is an automated email from the ASF dual-hosted git repository. garyli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from e03cd0a198 [HUDI-4545] Do not modify the current record directly for OverwriteNonDefaultsWithLatestAvroPayload (#6306)

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206157272 ## CI report: * 666088efaacc584a5f36db4df2f44f358e1ba53c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6311: [HUDI-4548] Unpack the column max/min to string instead of Utf8 for M…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6311: URL: https://github.com/apache/hudi/pull/6311#issuecomment-1206157321 ## CI report: * 17848d0f924115607c4144b3fa0a218333e89c99 Azure:

[hudi] branch master updated: [HUDI-4545] Do not modify the current record directly for OverwriteNonDefaultsWithLatestAvroPayload (#6306)

2022-08-05 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new e03cd0a198 [HUDI-4545] Do not modify the

[jira] [Updated] (HUDI-4505) Returns instead of throws if lock file exists for FileSystemBasedLockProvider

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4505: -- Priority: Blocker (was: Major) > Returns instead of throws if lock file exists for

[jira] [Commented] (HUDI-4545) Do not modify the current record directly for OverwriteNonDefaultsWithLatestAvroPayload

2022-08-05 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575596#comment-17575596 ] Danny Chen commented on HUDI-4545: -- Fixed via master branch: e03cd0a198f63df7fb7ba71d1c9a0b01ae33f021 >

[jira] [Updated] (HUDI-4504) Disable metadata table by default for flink

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4504: -- Priority: Blocker (was: Major) > Disable metadata table by default for flink >

[jira] [Closed] (HUDI-4536) ClusteringOperator causes the NullPointerException when writing with BulkInsertWriterHelper in clustering

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang closed HUDI-4536. Reviewers: Danny Chen Resolution: Fixed > ClusteringOperator causes the NullPointerException

[jira] [Created] (HUDI-4550) Investigate why rollback is triggered for completed instant

2022-08-05 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-4550: - Summary: Investigate why rollback is triggered for completed instant Key: HUDI-4550 URL: https://issues.apache.org/jira/browse/HUDI-4550 Project: Apache Hudi

[GitHub] [hudi] wzx140 commented on a diff in pull request #5629: [HUDI-3384][HUDI-3385] Spark specific file reader/writer.

2022-08-05 Thread GitBox
wzx140 commented on code in PR #5629: URL: https://github.com/apache/hudi/pull/5629#discussion_r938517253 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordMerger.java: ## @@ -30,9 +34,19 @@ * It can implement the merging logic of HoodieRecord of different

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206106447 ## CI report: * 5e75dee8c56cb14110b33548c09aad222adc57d2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6311: [HUDI-4548] Unpack the column max/min to string instead of Utf8 for M…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6311: URL: https://github.com/apache/hudi/pull/6311#issuecomment-1206106505 ## CI report: * 17848d0f924115607c4144b3fa0a218333e89c99 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206109860 ## CI report: * 5e75dee8c56cb14110b33548c09aad222adc57d2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6311: [HUDI-4548] Unpack the column max/min to string instead of Utf8 for M…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6311: URL: https://github.com/apache/hudi/pull/6311#issuecomment-1206109906 ## CI report: * 17848d0f924115607c4144b3fa0a218333e89c99 Azure:

[GitHub] [hudi] yanghua commented on a diff in pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
yanghua commented on code in PR #6144: URL: https://github.com/apache/hudi/pull/6144#discussion_r938516686 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/ScheduleCompactionActionExecutor.java: ## @@ -140,6 +140,17 @@ private Option>

[GitHub] [hudi] yanghua commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
yanghua commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206126780 @fengjian428 check the CI please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codope commented on issue #6224: [SUPPORT] Caused by: java.lang.IllegalArgumentException: Cannot use marker based rollback strategy on completed instant

2022-08-05 Thread GitBox
codope commented on issue #6224: URL: https://github.com/apache/hudi/issues/6224#issuecomment-1206097133 @jtchen-study Ideally, rollback is triggered only for failed writes. As such fallback to listing-based rollback should be safe but we need to understand how rollback got triggered for

[GitHub] [hudi] hudi-bot commented on pull request #6309: [HUDI-4547] Fix SortOperatorGen sort indices

2022-08-05 Thread GitBox
hudi-bot commented on PR #6309: URL: https://github.com/apache/hudi/pull/6309#issuecomment-1206103348 ## CI report: * f6df4432d24639619566565e3fac86cbd855ce9d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6310: [HUDI-4474] Fix inferring props for meta sync

2022-08-05 Thread GitBox
hudi-bot commented on PR #6310: URL: https://github.com/apache/hudi/pull/6310#issuecomment-1206103377 ## CI report: * 366dc59d094ffcdd05ba7cdf905b85cb684a9fa7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-08-05 Thread GitBox
hudi-bot commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1206102844 ## CI report: * 5a6ac9622379715e890f1ec1cd7be9422febeb5c Azure:

[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
Mulavar commented on issue #6169: URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206138784 Thanks, I tried to add dependencies with "provided" scope to classpath and solved the problem, and then ran into another problem: ```bash Exception in thread "main"

[GitHub] [hudi] danny0405 commented on pull request #6306: [HUDI-4545] Do not modify the current record directly for OverwriteNo…

2022-08-05 Thread GitBox
danny0405 commented on PR #6306: URL: https://github.com/apache/hudi/pull/6306#issuecomment-1206083880 The failed it should not be affected by this patch and it succeed i last run:

[GitHub] [hudi] leesf commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
leesf commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206236867 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
Mulavar commented on issue #6169: URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206253584 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] Mulavar closed issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
Mulavar closed issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly? URL: https://github.com/apache/hudi/issues/6169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206282226 ## CI report: * b24d8bf43a4f3e3c9831de0fb356ccf4c946f721 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206282170 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] nsivabalan commented on pull request #6248: [HUDI-4303] Adding 4 to 5 upgrade handler to check for old deprecated "default" partition value

2022-08-05 Thread GitBox
nsivabalan commented on PR #6248: URL: https://github.com/apache/hudi/pull/6248#issuecomment-1206493415 @YuweiXiao : apparently query engines don't like any other default partition other than "__HIVE_DEFAULT_PARTITION__". For eg, if partition path is of type long, during partition pruning,

[GitHub] [hudi] nsivabalan commented on pull request #6248: [HUDI-4303] Adding 4 to 5 upgrade handler to check for old deprecated "default" partition value

2022-08-05 Thread GitBox
nsivabalan commented on PR #6248: URL: https://github.com/apache/hudi/pull/6248#issuecomment-1206493517 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] SteNicholas opened a new pull request, #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
SteNicholas opened a new pull request, #6312: URL: https://github.com/apache/hudi/pull/6312 ## What is the purpose of the pull request The default value of `READ_TASKS`, `WRITE_TASKS`, `CLUSTERING_TASKS` is 4, which could be the parallelism of the execution environment. ##

[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
Mulavar commented on issue #6169: URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206198899 > So could I create a hoodie directory to support the example running properly? If yes, how do I create a hoodir directory. I totally agree with you about improving the

[jira] [Assigned] (HUDI-1161) Support update partial fields for MoR table

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-1161: Assignee: Jian Feng (was: Nicholas Jiang) > Support update partial fields for MoR table >

[GitHub] [hudi] danny0405 commented on pull request #6311: [HUDI-4548] Unpack the column max/min to string instead of Utf8 for M…

2022-08-05 Thread GitBox
danny0405 commented on PR #6311: URL: https://github.com/apache/hudi/pull/6311#issuecomment-1206301821 The failed case is flaky and not caused by this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HUDI-4547) Partition sorting does not take effect when use bucket_insert.

2022-08-05 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4547: - Fix Version/s: 0.12.0 > Partition sorting does not take effect when use bucket_insert. >

[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
xicm commented on issue #6169: URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206189741 You don't neet to create the directory, If the directory exists, HUDi will assume that the table exists but cannot find the.hoodie directory and throw an exception. Maybe we can

[GitHub] [hudi] prasannarajaperumal commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-05 Thread GitBox
prasannarajaperumal commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r938603827 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,27 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206223731 ## CI report: * f21fa7d0bb9b0a56cf91d01151fdd673da59eb09 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206228183 ## CI report: * f21fa7d0bb9b0a56cf91d01151fdd673da59eb09 Azure:

[hudi] branch master updated (fcdd4cf06c -> ae7395df1e)

2022-08-05 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from fcdd4cf06c [HUDI-4544] support retain hour cleaning policy for flink (#6300) add ae7395df1e [HUDI-4547] Fix

[GitHub] [hudi] codope commented on a diff in pull request #6266: [HUDI-4517] If no marker type file, fallback to timeline based marker

2022-08-05 Thread GitBox
codope commented on code in PR #6266: URL: https://github.com/apache/hudi/pull/6266#discussion_r938743512 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/MarkerBasedRollbackUtils.java: ## @@ -56,10 +56,16 @@ public static List

[GitHub] [hudi] SteNicholas commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
SteNicholas commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206367113 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Updated] (HUDI-4510) Config "hive_sync.metastore.uris" in flink sql sync is not effective

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4510: -- Priority: Blocker (was: Major) > Config "hive_sync.metastore.uris" in flink sql sync is not effective

[GitHub] [hudi] prasannarajaperumal commented on pull request #5436: [RFC-51] [HUDI-3478] Hudi to support Change-Data-Capture

2022-08-05 Thread GitBox
prasannarajaperumal commented on PR #5436: URL: https://github.com/apache/hudi/pull/5436#issuecomment-1206165913 We need to explicitly call out how CDC behaves for the following scenarios in the commit range operated 1. Insert and Delete of the same key (multiple times?) -

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206219327 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6311: [HUDI-4548] Unpack the column max/min to string instead of Utf8 for M…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6311: URL: https://github.com/apache/hudi/pull/6311#issuecomment-1206219410 ## CI report: * 04f067fce6df4225c497caeecd63dba7d069ba75 Azure:

[GitHub] [hudi] codope opened a new pull request, #6313: [HUDI-4550] Fallback to listing based rollback for completed instant

2022-08-05 Thread GitBox
codope opened a new pull request, #6313: URL: https://github.com/apache/hudi/pull/6313 ## What is the purpose of the pull request Ideally, rollback is not triggered for completed instants. However, if it gets triggered due to some extraneous condition or forced while rollback

[jira] [Updated] (HUDI-4550) Investigate why rollback is triggered for completed instant

2022-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4550: - Labels: pull-request-available (was: ) > Investigate why rollback is triggered for completed

[GitHub] [hudi] hudi-bot commented on pull request #6313: [HUDI-4550] Fallback to listing based rollback for completed instant

2022-08-05 Thread GitBox
hudi-bot commented on PR #6313: URL: https://github.com/apache/hudi/pull/6313#issuecomment-1206351949 ## CI report: * 2d1931b2f622e2ae524068464c7b5bc4d6bdd5ac Azure:

[jira] [Created] (HUDI-4552) RFC-58: Integrate column stats index with query engines other than spark

2022-08-05 Thread Pratyaksh Sharma (Jira)
Pratyaksh Sharma created HUDI-4552: -- Summary: RFC-58: Integrate column stats index with query engines other than spark Key: HUDI-4552 URL: https://issues.apache.org/jira/browse/HUDI-4552 Project:

[jira] [Assigned] (HUDI-2545) Flink compaction source supports the Source interface based on FLIP-27

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-2545: Assignee: yuemeng (was: Nicholas Jiang) > Flink compaction source supports the Source

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206399981 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6266: [HUDI-4517] If no marker type file, fallback to timeline based marker

2022-08-05 Thread GitBox
hudi-bot commented on PR #6266: URL: https://github.com/apache/hudi/pull/6266#issuecomment-1206399855 ## CI report: * 0d4f1037dad7db5b79d996e0e701400d0c07a2de Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206399650 ## CI report: * 661dbfff72f5f52b637dc4db2b8e89e9a4ab8a20 Azure:

[jira] [Closed] (HUDI-4504) Disable metadata table by default for flink

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4504. - Resolution: Fixed > Disable metadata table by default for flink >

[jira] [Closed] (HUDI-4510) Config "hive_sync.metastore.uris" in flink sql sync is not effective

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4510. - Resolution: Fixed > Config "hive_sync.metastore.uris" in flink sql sync is not effective >

[jira] [Closed] (HUDI-4505) Returns instead of throws if lock file exists for FileSystemBasedLockProvider

2022-08-05 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4505. - Resolution: Fixed > Returns instead of throws if lock file exists for FileSystemBasedLockProvider >

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206169231 ## CI report: * b24d8bf43a4f3e3c9831de0fb356ccf4c946f721 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6309: [HUDI-4547] Fix SortOperatorGen sort indices

2022-08-05 Thread GitBox
hudi-bot commented on PR #6309: URL: https://github.com/apache/hudi/pull/6309#issuecomment-1206228521 ## CI report: * f6df4432d24639619566565e3fac86cbd855ce9d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206228577 ## CI report: * b24d8bf43a4f3e3c9831de0fb356ccf4c946f721 Azure:

[jira] [Assigned] (HUDI-2441) To support partial update function which can move and update the data from the old partition to the new partition , when the data with same key change it's partition

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-2441: Assignee: David_Liang (was: Nicholas Jiang) > To support partial update function which

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206239026 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[jira] (HUDI-4269) Support multiple precombine fields

2022-08-05 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4269 ] Nicholas Jiang deleted comment on HUDI-4269: -- was (Author: nicholasjiang): [~danny0405], I have interest to multiple precombine fields. Could you please assign this ticket to me? > Support

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206336823 ## CI report: * 661dbfff72f5f52b637dc4db2b8e89e9a4ab8a20 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206337162 ## CI report: * 33048bf5a19016121cbe271a9353803d3fe1d261 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206396021 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206395645 ## CI report: * 661dbfff72f5f52b637dc4db2b8e89e9a4ab8a20 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206165027 ## CI report: * b24d8bf43a4f3e3c9831de0fb356ccf4c946f721 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?

2022-08-05 Thread GitBox
xicm commented on issue #6169: URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206215365 You can create the parent path of the base path, or just pass a directory string. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] hudi-bot commented on pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-05 Thread GitBox
hudi-bot commented on PR #6312: URL: https://github.com/apache/hudi/pull/6312#issuecomment-1206224195 ## CI report: * b24d8bf43a4f3e3c9831de0fb356ccf4c946f721 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6309: [HUDI-4547] Fix SortOperatorGen sort indices

2022-08-05 Thread GitBox
hudi-bot commented on PR #6309: URL: https://github.com/apache/hudi/pull/6309#issuecomment-1206224120 ## CI report: * f6df4432d24639619566565e3fac86cbd855ce9d Azure:

[GitHub] [hudi] SteNicholas commented on a diff in pull request #6100: [HUDI-4392]Flink MOR table compaction plan execution sequence should …

2022-08-05 Thread GitBox
SteNicholas commented on code in PR #6100: URL: https://github.com/apache/hudi/pull/6100#discussion_r938661789 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -502,6 +502,12 @@ private FlinkOptions() {

[GitHub] [hudi] hudi-bot commented on pull request #6313: [HUDI-4550] Fallback to listing based rollback for completed instant

2022-08-05 Thread GitBox
hudi-bot commented on PR #6313: URL: https://github.com/apache/hudi/pull/6313#issuecomment-1206286175 ## CI report: * 2d1931b2f622e2ae524068464c7b5bc4d6bdd5ac UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] chenshzh commented on pull request #6121: [HUDI-4406] Support Flink compaction commit write error resolvement to avoid data loss

2022-08-05 Thread GitBox
chenshzh commented on PR #6121: URL: https://github.com/apache/hudi/pull/6121#issuecomment-1206340858 @danny0405 would you pls help see this pr for avoiding data loss during compaction due to some write errors such as non-thrown exceptions ? -- This is an automated message from the

[GitHub] [hudi] hudi-bot commented on pull request #6266: [HUDI-4517] If no marker type file, fallback to timeline based marker

2022-08-05 Thread GitBox
hudi-bot commented on PR #6266: URL: https://github.com/apache/hudi/pull/6266#issuecomment-1206395852 ## CI report: * 0d4f1037dad7db5b79d996e0e701400d0c07a2de Azure:

[jira] [Updated] (HUDI-4552) RFC-58: Integrate column stats index with query engines other than spark

2022-08-05 Thread Pratyaksh Sharma (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-4552: --- Labels: hudi-umbrellas (was: ) > RFC-58: Integrate column stats index with query engines

[jira] [Updated] (HUDI-4552) RFC-58: Integrate column stats index with query engines other than spark

2022-08-05 Thread Pratyaksh Sharma (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-4552: --- Status: In Progress (was: Open) > RFC-58: Integrate column stats index with query engines

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206474881 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6266: [HUDI-4517] If no marker type file, fallback to timeline based marker

2022-08-05 Thread GitBox
hudi-bot commented on PR #6266: URL: https://github.com/apache/hudi/pull/6266#issuecomment-1206479522 ## CI report: * 47f5c73193af282b3a229375be9351a91f613f5c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6307: [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis

2022-08-05 Thread GitBox
hudi-bot commented on PR #6307: URL: https://github.com/apache/hudi/pull/6307#issuecomment-1206479694 ## CI report: * 14aa4355ee414a6cb4814950216fe5ea93ccba16 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6144: [HUDI-4424] add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ…

2022-08-05 Thread GitBox
hudi-bot commented on PR #6144: URL: https://github.com/apache/hudi/pull/6144#issuecomment-1206488706 ## CI report: * 661dbfff72f5f52b637dc4db2b8e89e9a4ab8a20 Azure:

[GitHub] [hudi] fengjian428 commented on issue #6308: [SUPPORT] Spark multi writer failed ! ! !

2022-08-05 Thread GitBox
fengjian428 commented on issue #6308: URL: https://github.com/apache/hudi/issues/6308#issuecomment-1206550539 ![image](https://user-images.githubusercontent.com/4403474/183103858-81383666-30d2-4f08-912c-3bdbc0a6fc6c.png) I also met the second issue, will create a pr to fix it -- This

[GitHub] [hudi] hudi-bot commented on pull request #6248: [HUDI-4303] Adding 4 to 5 upgrade handler to check for old deprecated "default" partition value

2022-08-05 Thread GitBox
hudi-bot commented on PR #6248: URL: https://github.com/apache/hudi/pull/6248#issuecomment-1206550924 ## CI report: * a2da187e83c39188fe8783743e7e3f6f1816691a UNKNOWN * f9a56355c62858901b4ec1f53e31f971eef56f81 UNKNOWN * 7b874997625becffbcad6cba0c51ceca9fd5080c Azure:

[jira] [Created] (HUDI-4554) Investigate hdfs test failures

2022-08-05 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4554: - Summary: Investigate hdfs test failures Key: HUDI-4554 URL: https://issues.apache.org/jira/browse/HUDI-4554 Project: Apache Hudi Issue Type: Test

[GitHub] [hudi] hudi-bot commented on pull request #6310: [HUDI-4474] Fix inferring props for meta sync

2022-08-05 Thread GitBox
hudi-bot commented on PR #6310: URL: https://github.com/apache/hudi/pull/6310#issuecomment-1206612896 ## CI report: * 366dc59d094ffcdd05ba7cdf905b85cb684a9fa7 Azure:

[GitHub] [hudi] 15663671003 commented on issue #6315: [SUPPORT]

2022-08-05 Thread GitBox
15663671003 commented on issue #6315: URL: https://github.com/apache/hudi/issues/6315#issuecomment-1206695542 What is certain is that there are no write transactions running during incremental reads, why do I get such an error? -- This is an automated message from the Apache Git Service.

[jira] [Closed] (HUDI-4553) fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: ALREADY_ACQUIRED

2022-08-05 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-4553. --- Resolution: Duplicate > fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: >

[jira] [Updated] (HUDI-4555) The behavior of "show fsview all" is confusing

2022-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4555: - Labels: pull-request-available (was: ) > The behavior of "show fsview all" is confusing >

[GitHub] [hudi] hudi-bot commented on pull request #6310: [HUDI-4474] Fix inferring props for meta sync

2022-08-05 Thread GitBox
hudi-bot commented on PR #6310: URL: https://github.com/apache/hudi/pull/6310#issuecomment-1206680992 ## CI report: * 88c31fc72be6506b31eda058f6f0d0130a7a9b05 Azure:

[GitHub] [hudi] 15663671003 opened a new issue, #6315: [SUPPORT]

2022-08-05 Thread GitBox
15663671003 opened a new issue, #6315: URL: https://github.com/apache/hudi/issues/6315 **Describe the problem you faced** The problem that the parquet file cannot be found when using spark to incrementally read the MOR table, When reading, no write transactions are executed

[GitHub] [hudi] hudi-bot commented on pull request #6248: [HUDI-4303] Adding 4 to 5 upgrade handler to check for old deprecated "default" partition value

2022-08-05 Thread GitBox
hudi-bot commented on PR #6248: URL: https://github.com/apache/hudi/pull/6248#issuecomment-1206545633 ## CI report: * a2da187e83c39188fe8783743e7e3f6f1816691a UNKNOWN * f9a56355c62858901b4ec1f53e31f971eef56f81 UNKNOWN * 7b874997625becffbcad6cba0c51ceca9fd5080c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6310: [HUDI-4474] Fix inferring props for meta sync

2022-08-05 Thread GitBox
hudi-bot commented on PR #6310: URL: https://github.com/apache/hudi/pull/6310#issuecomment-1206617449 ## CI report: * 366dc59d094ffcdd05ba7cdf905b85cb684a9fa7 Azure:

[jira] [Created] (HUDI-4555) The behavior of "show fsview all" is confusing

2022-08-05 Thread xi chaomin (Jira)
xi chaomin created HUDI-4555: Summary: The behavior of "show fsview all" is confusing Key: HUDI-4555 URL: https://issues.apache.org/jira/browse/HUDI-4555 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] xicm opened a new pull request, #6314: [HUDI-4555] fix bug in cli show fsview all

2022-08-05 Thread GitBox
xicm opened a new pull request, #6314: URL: https://github.com/apache/hudi/pull/6314 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[GitHub] [hudi] hudi-bot commented on pull request #6314: [HUDI-4555] fix bug in cli show fsview all

2022-08-05 Thread GitBox
hudi-bot commented on PR #6314: URL: https://github.com/apache/hudi/pull/6314#issuecomment-1206670194 ## CI report: * 2c061f4259ead8271187b9fd7dd44f86325f1f30 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

  1   2   3   >