[GitHub] [hudi] hudi-bot commented on pull request #6619: [HUDI-4796] MetricsReporter stop bug

2022-09-06 Thread GitBox
hudi-bot commented on PR #6619: URL: https://github.com/apache/hudi/pull/6619#issuecomment-1238940922 ## CI report: * 461a755d6938132f17243987fb7ab5e69a883f1e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2022-09-06 Thread GitBox
hudi-bot commented on PR #6384: URL: https://github.com/apache/hudi/pull/6384#issuecomment-1238940534 ## CI report: * 37785220f2d17a1a04d136521f10c3a0314fe448 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6619: [HUDI-4796] MetricsReporter stop bug

2022-09-06 Thread GitBox
hudi-bot commented on PR #6619: URL: https://github.com/apache/hudi/pull/6619#issuecomment-1238937620 ## CI report: * 461a755d6938132f17243987fb7ab5e69a883f1e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6616: Add Postgres Schema Name to Postgres Debezium Source

2022-09-06 Thread GitBox
hudi-bot commented on PR #6616: URL: https://github.com/apache/hudi/pull/6616#issuecomment-1238934271 ## CI report: * 8176e809b4f329e0cfbff75484b3595c69970207 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6612: [HUDI-4790] a more effective HoodieMergeHandler for COW table with parquet

2022-09-06 Thread GitBox
hudi-bot commented on PR #6612: URL: https://github.com/apache/hudi/pull/6612#issuecomment-1238934239 ## CI report: * a990d7b411e5692568e548f4b31394f1fd051e77 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6574: Keep a clustering running at the same time.#6573

2022-09-06 Thread GitBox
hudi-bot commented on PR #6574: URL: https://github.com/apache/hudi/pull/6574#issuecomment-1238934128 ## CI report: * 277061fa910ff388b9fa580083fd3af406ce3b94 Azure:

[jira] [Updated] (HUDI-4796) Properly release MetricsReporter resources

2022-09-06 Thread Timothy Brown (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Brown updated HUDI-4796: Status: In Progress (was: Open) > Properly release MetricsReporter resources >

[jira] [Updated] (HUDI-4796) Properly release MetricsReporter resources

2022-09-06 Thread Timothy Brown (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Brown updated HUDI-4796: Status: Patch Available (was: In Progress) > Properly release MetricsReporter resources >

[GitHub] [hudi] xushiyan commented on a diff in pull request #4718: [HUDI-3345][RFC-36] Hudi metastore server

2022-09-06 Thread GitBox
xushiyan commented on code in PR #4718: URL: https://github.com/apache/hudi/pull/4718#discussion_r964383930 ## rfc/rfc-36/rfc-36.md: ## @@ -0,0 +1,605 @@ + +# RFC-36: Hudi Metastore Server + +## Proposers + +- @minihippo + +## Approvers + + +## Status + +JIRA:

[jira] [Updated] (HUDI-4796) Properly release MetricsReporter resources

2022-09-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4796: - Labels: pull-request-available (was: ) > Properly release MetricsReporter resources >

[GitHub] [hudi] the-other-tim-brown opened a new pull request, #6619: [HUDI-4796] MetricsReporter stop bug

2022-09-06 Thread GitBox
the-other-tim-brown opened a new pull request, #6619: URL: https://github.com/apache/hudi/pull/6619 ### Change Logs - Removes a confusing method, `getReporter()` in the abstract class MetricsReporter since we want to be calling `stop` the MetricsReporter instances to make sure they

[jira] [Assigned] (HUDI-4796) Properly release MetricsReporter resources

2022-09-06 Thread Timothy Brown (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Brown reassigned HUDI-4796: --- Assignee: Timothy Brown > Properly release MetricsReporter resources >

[jira] [Created] (HUDI-4796) Properly release MetricsReporter resources

2022-09-06 Thread Timothy Brown (Jira)
Timothy Brown created HUDI-4796: --- Summary: Properly release MetricsReporter resources Key: HUDI-4796 URL: https://issues.apache.org/jira/browse/HUDI-4796 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] xushiyan commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-09-06 Thread GitBox
xushiyan commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1238907156 @prasannarajaperumal Other than the RPC protocol consideration as @minihippo mentioned, with Thrift generated models we'll have flexibilities in adapting with different metastores /

[GitHub] [hudi] hudi-bot commented on pull request #6574: Keep a clustering running at the same time.#6573

2022-09-06 Thread GitBox
hudi-bot commented on PR #6574: URL: https://github.com/apache/hudi/pull/6574#issuecomment-1238906690 ## CI report: * 277061fa910ff388b9fa580083fd3af406ce3b94 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238904393 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238902007 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6615: [HUDI-4758] Add validations to java spark examples

2022-09-06 Thread GitBox
hudi-bot commented on PR #6615: URL: https://github.com/apache/hudi/pull/6615#issuecomment-1238899373 ## CI report: * 61214015c3aed029c00882f121e6ec0333767e7f Azure:

[GitHub] [hudi] xushiyan commented on a diff in pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-09-06 Thread GitBox
xushiyan commented on code in PR #5064: URL: https://github.com/apache/hudi/pull/5064#discussion_r964373714 ## hudi-metaserver/src/main/java/org/apache/hudi/common/table/HoodieTableMetaServerClient.java: ## @@ -0,0 +1,158 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table

2022-09-06 Thread GitBox
hudi-bot commented on PR #6016: URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238898719 ## CI report: * 46e53b5182ffdf6fa43b5a93921222e869e4e535 Azure:

[jira] [Closed] (HUDI-4635) Update roadmap page based on H2 2022 plan

2022-09-06 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-4635. --- Resolution: Fixed > Update roadmap page based on H2 2022 plan > - > >

[GitHub] [hudi] nsivabalan commented on issue #6609: hudi upsert occured data duplication by spark streaming (cow table)

2022-09-06 Thread GitBox
nsivabalan commented on issue #6609: URL: https://github.com/apache/hudi/issues/6609#issuecomment-1238884639 we identified an issue w/ spark streaming where duplicate data could sneak into hudi w/ failures. https://github.com/apache/hudi/pull/6098 can you give it a try w/ latest

[GitHub] [hudi] danny0405 commented on pull request #6429: [HUDI-4636] Output preCombine fields of delete records when changelog disabled

2022-09-06 Thread GitBox
danny0405 commented on PR #6429: URL: https://github.com/apache/hudi/pull/6429#issuecomment-1238884544 > We need the preCombine and partition fields also, so pull this request. Can you explain why we need this then, do you want to write to another hudi table using these records ?

[GitHub] [hudi] boneanxs commented on pull request #6046: [HUDI-4363] Support Clustering row writer to improve performance

2022-09-06 Thread GitBox
boneanxs commented on PR #6046: URL: https://github.com/apache/hudi/pull/6046#issuecomment-1238883968 Hey, @alexeykudinkin, addressed all comments, could you plz review again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] danny0405 commented on pull request #6595: [HUDI-4777] Fix flink gen bucket index of mor table not consistent wi…

2022-09-06 Thread GitBox
danny0405 commented on PR #6595: URL: https://github.com/apache/hudi/pull/6595#issuecomment-1238882822 > When spark use loadPartitionBucketIdFileIdMapping of org.apache.hudi.index.bucket.HoodieSimpleBucketIndex, it will not find the bucket num which written by hudi-flink Seems we

[GitHub] [hudi] nsivabalan commented on issue #6606: Observing data duplication with Single Writer

2022-09-06 Thread GitBox
nsivabalan commented on issue #6606: URL: https://github.com/apache/hudi/issues/6606#issuecomment-1238882651 oh, I thought, both jobs are running concurrently? is it not. can you throw some light on exact steps. is it. step1: start job1 in EMR cluster1. which consumes from source X

[GitHub] [hudi] nsivabalan commented on issue #6606: Observing data duplication with Single Writer

2022-09-06 Thread GitBox
nsivabalan commented on issue #6606: URL: https://github.com/apache/hudi/issues/6606#issuecomment-1238880984 unless you configure lock providers, hudi can't guarantee this. I would suggest to add locking for both writers. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] voonhous commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
voonhous commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964360302 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -62,6 +64,8 @@ public class

[GitHub] [hudi] danny0405 commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
danny0405 commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964359395 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -62,6 +64,8 @@ public class

[GitHub] [hudi] nsivabalan commented on issue #6590: [SUPPORT] HoodieDeltaStreamer AWSDmsAvroPayload fails to handle deletes in MySQL

2022-09-06 Thread GitBox
nsivabalan commented on issue #6590: URL: https://github.com/apache/hudi/issues/6590#issuecomment-1238879167 @codope : this is similar to the other issue you were triaging last week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[hudi] branch master updated: [HUDI-4615] Return checkpoint as null for empty data from events queue. (#6387)

2022-09-06 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new d2d1cb8b28 [HUDI-4615] Return checkpoint as

[GitHub] [hudi] nsivabalan merged pull request #6387: [HUDI-4615] Return checkpoint as null for empty data from events queue.

2022-09-06 Thread GitBox
nsivabalan merged PR #6387: URL: https://github.com/apache/hudi/pull/6387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] ymZhao1001 commented on pull request #6366: [HUDI-4794] add an option of the log file block size

2022-09-06 Thread GitBox
ymZhao1001 commented on PR #6366: URL: https://github.com/apache/hudi/pull/6366#issuecomment-1238877502 > @ymZhao1001 Could you follow the process [here](https://hudi.apache.org/contribute/developer-setup#filing-jiras) by filing and claiming a Jira ticket? done

[GitHub] [hudi] voonhous commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
voonhous commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964355713 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -335,5 +391,17 @@ public void

[GitHub] [hudi] voonhous commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
voonhous commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964355713 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -335,5 +391,17 @@ public void

[GitHub] [hudi] danny0405 commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
danny0405 commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964350288 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -335,5 +391,17 @@ public void

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238868528 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6366: [HUDI-4794] add an option of the log file block size

2022-09-06 Thread GitBox
hudi-bot commented on PR #6366: URL: https://github.com/apache/hudi/pull/6366#issuecomment-1238868304 ## CI report: * 41f40900c1a22a49dce612f2684de711c6760199 Azure:

[GitHub] [hudi] voonhous commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
voonhous commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964348442 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -335,5 +391,17 @@ public void

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238865999 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #6387: [HUDI-4615] Return checkpoint as null for empty data from events queue.

2022-09-06 Thread GitBox
hudi-bot commented on PR #6387: URL: https://github.com/apache/hudi/pull/6387#issuecomment-1238865806 ## CI report: * eaf0accd1d182170e591417c2ca1ef832fae5924 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6366: [HUDI-4794] add an option of the log file block size

2022-09-06 Thread GitBox
hudi-bot commented on PR #6366: URL: https://github.com/apache/hudi/pull/6366#issuecomment-1238865778 ## CI report: * 41f40900c1a22a49dce612f2684de711c6760199 Azure:

[GitHub] [hudi] wzx140 commented on pull request #6486: [HUDI-4706] Fix InternalSchemaChangeApplier#applyAddChange error to add nest type

2022-09-06 Thread GitBox
wzx140 commented on PR #6486: URL: https://github.com/apache/hudi/pull/6486#issuecomment-1238863109 @xiarixiaoyao @yihua I found UT not cover InternalSchemaChangeApplier. I will add some tests later. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238863050 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] danny0405 commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
danny0405 commented on code in PR #6566: URL: https://github.com/apache/hudi/pull/6566#discussion_r964343611 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/HoodieFlinkClusteringJob.java: ## @@ -335,5 +391,17 @@ public void

[GitHub] [hudi] danny0405 commented on a diff in pull request #5113: [HUDI-3625] [RFC-60] Optimized storage layout for Cloud Object Stores

2022-09-06 Thread GitBox
danny0405 commented on code in PR #5113: URL: https://github.com/apache/hudi/pull/5113#discussion_r964341113 ## rfc/rfc-56/rfc-56.md: ## @@ -0,0 +1,226 @@ + + +# RFC-56: Federated Storage Layer + +## Proposers +- @umehrot2 + +## Approvers +- @vinoth +- @shivnarayan + +## Status

[GitHub] [hudi] danny0405 commented on a diff in pull request #5113: [HUDI-3625] [RFC-60] Optimized storage layout for Cloud Object Stores

2022-09-06 Thread GitBox
danny0405 commented on code in PR #5113: URL: https://github.com/apache/hudi/pull/5113#discussion_r964340481 ## rfc/rfc-56/rfc-56.md: ## @@ -0,0 +1,226 @@ + + +# RFC-56: Federated Storage Layer + +## Proposers +- @umehrot2 + +## Approvers +- @vinoth +- @shivnarayan + +## Status

[jira] [Updated] (HUDI-4794) add an option of the log file block size

2022-09-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4794: - Labels: pull-request-available (was: ) > add an option of the log file block size >

[GitHub] [hudi] ymZhao1001 commented on a diff in pull request #6366: [HUDI-4794] add an option of the log file block size

2022-09-06 Thread GitBox
ymZhao1001 commented on code in PR #6366: URL: https://github.com/apache/hudi/pull/6366#discussion_r964340073 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFormatWriter.java: ## @@ -230,8 +230,7 @@ private void rollOver() throws IOException { }

[jira] [Updated] (HUDI-4704) bulk insert overwrite table will delete the table and then recreate a table

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4704: -- Sprint: 2022/09/05 > bulk insert overwrite table will delete the table and then

[jira] [Assigned] (HUDI-4704) bulk insert overwrite table will delete the table and then recreate a table

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4704: - Assignee: Raymond Xu > bulk insert overwrite table will delete the table and

[jira] [Updated] (HUDI-4704) bulk insert overwrite table will delete the table and then recreate a table

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4704: -- Fix Version/s: 0.12.1 > bulk insert overwrite table will delete the table and then

[jira] [Updated] (HUDI-4716) Avoid bundle parquet in hadoop-mr

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4716: -- Sprint: 2022/09/19 > Avoid bundle parquet in hadoop-mr >

[jira] [Closed] (HUDI-4720) HoodieInternalRow return wrong num of fields when source not contains meta fields

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4720. - Assignee: sivabalan narayanan Resolution: Fixed > HoodieInternalRow return wrong

[jira] [Updated] (HUDI-4722) Add support for metrics for locking infra

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4722: -- Sprint: 2022/09/05 > Add support for metrics for locking infra >

[jira] [Updated] (HUDI-4720) HoodieInternalRow return wrong num of fields when source not contains meta fields

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4720: -- Fix Version/s: 0.12.1 > HoodieInternalRow return wrong num of fields when source not

[jira] [Updated] (HUDI-4722) Add support for metrics for locking infra

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4722: -- Reviewers: sivabalan narayanan Story Points: 1 > Add support for metrics for

[jira] [Updated] (HUDI-4722) Add support for metrics for locking infra

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4722: -- Fix Version/s: 0.12.1 > Add support for metrics for locking infra >

[jira] [Updated] (HUDI-4722) Add support for metrics for locking infra

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4722: -- Priority: Major (was: Minor) > Add support for metrics for locking infra >

[jira] [Updated] (HUDI-4724) add function of skip the _rt suffix for read snapshot

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4724: -- Sprint: 2022/09/05 > add function of skip the _rt suffix for read snapshot >

[jira] [Updated] (HUDI-4724) add function of skip the _rt suffix for read snapshot

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4724: -- Reviewers: Raymond Xu Story Points: 1 > add function of skip the _rt suffix for

[jira] [Updated] (HUDI-4734) Add table config change validation in deltastreamer

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4734: -- Sprint: 2022/09/05 > Add table config change validation in deltastreamer >

[jira] [Updated] (HUDI-4735) Spark2 bundles made from master after 2022-07-23 failed to stop

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4735: -- Sprint: 2022/09/05 > Spark2 bundles made from master after 2022-07-23 failed to stop >

[jira] [Updated] (HUDI-4735) Spark2 bundles made from master after 2022-07-23 failed to stop

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4735: -- Fix Version/s: 0.12.1 > Spark2 bundles made from master after 2022-07-23 failed to stop

[GitHub] [hudi] Aload opened a new issue, #6618: Caused by: org.apache.http.NoHttpResponseException: xxxxxx:34812 failed to respond[SUPPORT]

2022-09-06 Thread GitBox
Aload opened a new issue, #6618: URL: https://github.com/apache/hudi/issues/6618 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

[jira] [Updated] (HUDI-4762) Hive sync update schema removes columns

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4762: -- Sprint: 2022/09/05 > Hive sync update schema removes columns >

[jira] [Updated] (HUDI-4762) Hive sync update schema removes columns

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4762: -- Component/s: meta-sync > Hive sync update schema removes columns >

[jira] [Updated] (HUDI-4762) Hive sync update schema removes columns

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4762: -- Reviewers: Raymond Xu Story Points: 1 > Hive sync update schema removes columns

[jira] [Updated] (HUDI-4762) Hive sync update schema removes columns

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4762: -- Priority: Critical (was: Major) > Hive sync update schema removes columns >

[jira] [Updated] (HUDI-4762) Hive sync update schema removes columns

2022-09-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4762: -- Fix Version/s: 0.12.1 > Hive sync update schema removes columns >

[GitHub] [hudi] danny0405 commented on a diff in pull request #5716: [HUDI-4167] Remove the timeline refresh with initializing hoodie table

2022-09-06 Thread GitBox
danny0405 commented on code in PR #5716: URL: https://github.com/apache/hudi/pull/5716#discussion_r931721763 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -567,6 +564,16 @@ private synchronized void close(Pair

[GitHub] [hudi] hudi-bot commented on pull request #6574: Keep a clustering running at the same time.#6573

2022-09-06 Thread GitBox
hudi-bot commented on PR #6574: URL: https://github.com/apache/hudi/pull/6574#issuecomment-1238837321 ## CI report: * 277061fa910ff388b9fa580083fd3af406ce3b94 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

2022-09-06 Thread GitBox
hudi-bot commented on PR #6566: URL: https://github.com/apache/hudi/pull/6566#issuecomment-1238837297 ## CI report: * b10c9d062f03c2c2675866c6f4bf6346dc03ea49 UNKNOWN * a2dcd81f74603e88c4db895900d43eee6702a6da UNKNOWN * c404647afc6d26bc0e69a7a8ef93f378b397bb96 UNKNOWN *

[GitHub] [hudi] eric9204 commented on a diff in pull request #6574: Keep a clustering running at the same time.#6573

2022-09-06 Thread GitBox
eric9204 commented on code in PR #6574: URL: https://github.com/apache/hudi/pull/6574#discussion_r963258228 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/ClusteringPlanActionExecutor.java: ## @@ -77,11 +78,14 @@ protected Option

[GitHub] [hudi] hudi-bot commented on pull request #6574: Keep a clustering running at the same time.#6573

2022-09-06 Thread GitBox
hudi-bot commented on PR #6574: URL: https://github.com/apache/hudi/pull/6574#issuecomment-1238834637 ## CI report: * 277061fa910ff388b9fa580083fd3af406ce3b94 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6607: [HUDI-4782] Support TIMESTAMP_LTZ type for flink

2022-09-06 Thread GitBox
hudi-bot commented on PR #6607: URL: https://github.com/apache/hudi/pull/6607#issuecomment-1238831881 ## CI report: * e05038ec2798a39ce3ab7bcbdbcf9c7e009c8188 Azure:

[GitHub] [hudi] danny0405 closed pull request #2419: [HUDI-1421] Improvement of failure recovery for HoodieFlinkStreamer.

2022-09-06 Thread GitBox
danny0405 closed pull request #2419: [HUDI-1421] Improvement of failure recovery for HoodieFlinkStreamer. URL: https://github.com/apache/hudi/pull/2419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] danny0405 closed issue #6540: [SUPPORT]KryoException when bulk insert into hudi with flink

2022-09-06 Thread GitBox
danny0405 closed issue #6540: [SUPPORT]KryoException when bulk insert into hudi with flink URL: https://github.com/apache/hudi/issues/6540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] danny0405 commented on issue #6540: [SUPPORT]KryoException when bulk insert into hudi with flink

2022-09-06 Thread GitBox
danny0405 commented on issue #6540: URL: https://github.com/apache/hudi/issues/6540#issuecomment-1238828954 Thanks, the problem expects to be fixed in #6571, feel free to reopen it if the problem still exists. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot commented on pull request #6607: [HUDI-4782] Support TIMESTAMP_LTZ type for flink

2022-09-06 Thread GitBox
hudi-bot commented on PR #6607: URL: https://github.com/apache/hudi/pull/6607#issuecomment-1238828680 ## CI report: * e05038ec2798a39ce3ab7bcbdbcf9c7e009c8188 Azure:

[jira] [Resolved] (HUDI-4795) Fix KryoException when bulk insert into a not bucket index hudi table

2022-09-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4795. -- > Fix KryoException when bulk insert into a not bucket index hudi table >

[jira] [Commented] (HUDI-4795) Fix KryoException when bulk insert into a not bucket index hudi table

2022-09-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601102#comment-17601102 ] Danny Chen commented on HUDI-4795: -- Fixed via master branch: 27c7efb4efc380360af7a18fc57c0757f852390f >

[hudi] branch master updated (323f19685c -> 27c7efb4ef)

2022-09-06 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 323f19685c [HUDI-4776] Fix merge into use unresolved assignment (#6589) add 27c7efb4ef [HUDI-4795] Fix

[jira] [Updated] (HUDI-4795) Fix KryoException when bulk insert into a not bucket index hudi table

2022-09-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4795: - Labels: pull-request-available (was: ) > Fix KryoException when bulk insert into a not bucket

[jira] [Created] (HUDI-4795) Fix KryoException when bulk insert into a not bucket index hudi table

2022-09-06 Thread Danny Chen (Jira)
Danny Chen created HUDI-4795: Summary: Fix KryoException when bulk insert into a not bucket index hudi table Key: HUDI-4795 URL: https://issues.apache.org/jira/browse/HUDI-4795 Project: Apache Hudi

[GitHub] [hudi] danny0405 merged pull request #6571: [HUDI-4795] Fix KryoException when bulk insert into a not bucket index hudi table

2022-09-06 Thread GitBox
danny0405 merged PR #6571: URL: https://github.com/apache/hudi/pull/6571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #6612: [HUDI-4790] a more effective HoodieMergeHandler for COW table with parquet

2022-09-06 Thread GitBox
danny0405 commented on PR #6612: URL: https://github.com/apache/hudi/pull/6612#issuecomment-1238824801 Overall an interesting idea, let put the details in the document. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] paul8263 commented on pull request #6489: [HUDI-4485] [cli] Bumped spring shell to 2.1.1. Updated the default …

2022-09-06 Thread GitBox
paul8263 commented on PR #6489: URL: https://github.com/apache/hudi/pull/6489#issuecomment-1238819752 Hi @codope and @yihua , Errors of hudi-integ-test are almost cleared. The only one left is:

[GitHub] [hudi] hbgstc123 commented on issue #6540: [SUPPORT]KryoException when bulk insert into hudi with flink

2022-09-06 Thread GitBox
hbgstc123 commented on issue #6540: URL: https://github.com/apache/hudi/issues/6540#issuecomment-1238816669 > Do you use streaming mode? When I use streaming mode and use bounded source, this error will be reproduced stably. Both streaming mode and batch mode can reproduce stably

[GitHub] [hudi] yihua closed pull request #3010: Improving Hudi CLI tool docs

2022-09-06 Thread GitBox
yihua closed pull request #3010: Improving Hudi CLI tool docs URL: https://github.com/apache/hudi/pull/3010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hudi] yihua commented on pull request #3010: Improving Hudi CLI tool docs

2022-09-06 Thread GitBox
yihua commented on PR #3010: URL: https://github.com/apache/hudi/pull/3010#issuecomment-1238815146 Closing this as the current [OSS Hudi CLI guide](https://hudi.apache.org/docs/cli) and [EMR Hudi CLI guide](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hudi-cli.html) are

[GitHub] [hudi] yihua commented on pull request #2963: [HUDI-1904] Introduce SchemaProviderInterface to make SchemaProvider unified

2022-09-06 Thread GitBox
yihua commented on PR #2963: URL: https://github.com/apache/hudi/pull/2963#issuecomment-1238811686 @wangxianghu do we still need this or can we close it now, given schema on read / evolution is supported in Spark? -- This is an automated message from the Apache Git Service. To respond to

[jira] [Created] (HUDI-4794) add an option of the log file block size

2022-09-06 Thread zhaoyangming (Jira)
zhaoyangming created HUDI-4794: -- Summary: add an option of the log file block size Key: HUDI-4794 URL: https://issues.apache.org/jira/browse/HUDI-4794 Project: Apache Hudi Issue Type:

[GitHub] [hudi] yihua commented on pull request #2751: [HUDI-1748] Read operation will possibly fail on mor table rt view when a write operations is concurrency running

2022-09-06 Thread GitBox
yihua commented on PR #2751: URL: https://github.com/apache/hudi/pull/2751#issuecomment-1238805411 @alexeykudinkin Is this already fixed by the new input format classes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] yihua commented on pull request #2701: [HUDI 1623] New Hoodie Instant on disk format with end time and milliseconds granularity

2022-09-06 Thread GitBox
yihua commented on PR #2701: URL: https://github.com/apache/hudi/pull/2701#issuecomment-1238803297 The code base has changed a lot and millisecond instant time has already been supported. @n3nash Do you want to close this PR and open a new one by cleaning up the changes, instead of

[GitHub] [hudi] yihua commented on pull request #2607: [HUDI-1643] Hudi observability - framework to report stats from execu…

2022-09-06 Thread GitBox
yihua commented on PR #2607: URL: https://github.com/apache/hudi/pull/2607#issuecomment-1238801513 @nbalajee @prashantwason is this PR still being worked on? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] yihua commented on pull request #2519: [HUDI-1573] Spark Sql Writer support Multi preCmp Field

2022-09-06 Thread GitBox
yihua commented on PR #2519: URL: https://github.com/apache/hudi/pull/2519#issuecomment-1238800315 @nsivabalan if this something we can tackle given that a couple of users ask for support of multiple pre-combine fields? -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] alexeykudinkin commented on pull request #6616: Add Postgres Schema Name to Postgres Debezium Source

2022-09-06 Thread GitBox
alexeykudinkin commented on PR #6616: URL: https://github.com/apache/hudi/pull/6616#issuecomment-1238800247 @modi95 LGTM @rmahindra123 can you PTAL as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] hudi-bot commented on pull request #6617: [HUDI-4793] Fixing ScalaTest tests to properly respect Log4j2 configs

2022-09-06 Thread GitBox
hudi-bot commented on PR #6617: URL: https://github.com/apache/hudi/pull/6617#issuecomment-1238800074 ## CI report: * 6245a9ffa1b721f6c0640c3e9b819f0d258f02f2 Azure:

[GitHub] [hudi] yihua commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2022-09-06 Thread GitBox
yihua commented on PR #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-1238798240 @vinothchandar @nsivabalan @xushiyan if I understand correctly based on the discussion, this PR is ready to land after fixing the tests. The performance test on S3 is a plus. Or could we

  1   2   3   4   >