[GitHub] [hudi] sam-wmt opened a new issue #1782: Merge-On-Read performance degrades for single partition table [SUPPORT]

sam-wmt opened a new issue #1782: URL: https://github.com/apache/hudi/issues/1782 **Describe the problem you faced** Currently we are streaming data, upserting into a Merge-On-Read table. The total table will contain 350M entities bounded, and we expect the approximate table size to

[hudi] branch asf-site updated: Travis CI build asf-site

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new efa8532 Travis CI build asf-site efa8532 is

[GitHub] [hudi] leesf commented on pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

leesf commented on pull request #1509: URL: https://github.com/apache/hudi/pull/1509#issuecomment-652781339 @n3nash @bvaradar kindly remind to land this PR? This is an automated message from the Apache Git Service. To

[GitHub] [hudi] Trevor-zhang edited a comment on pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

Trevor-zhang edited a comment on pull request #1779: URL: https://github.com/apache/hudi/pull/1779#issuecomment-652779036 > @Trevor-zhang pls check the reason why Travis is red. Will merge it, after it turns into green. @yanghua The test case not passed, i will fix the problem ASAP.

[GitHub] [hudi] Trevor-zhang commented on pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

Trevor-zhang commented on pull request #1779: URL: https://github.com/apache/hudi/pull/1779#issuecomment-652779036 > @Trevor-zhang pls check the reason why Travis is red. Will merge it, after it turns into green. The test case not passed, i will fix the problem ASAP.

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448745913 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/prometheus/PushGatewayReporter.java ## @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448745428 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/prometheus/PushGatewayReporter.java ## @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448745372 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/prometheus/PrometheusReporter.java ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448745067 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/prometheus/PrometheusReporter.java ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448744264 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/JmxMetricsReporter.java ## @@ -126,4 +126,5 @@ private JmxReporterServer

[GitHub] [hudi] leesf commented on a change in pull request #1726: [HUDI-210]Hudi support prometheus

leesf commented on a change in pull request #1726: URL: https://github.com/apache/hudi/pull/1726#discussion_r448744092 ## File path: hudi-client/pom.xml ## @@ -114,8 +114,20 @@ - io.dropwizard.metrics - metrics-core Review comment: why

[hudi] branch asf-site updated: [MINOR] Add the users@ mailing list to the community page (#1778)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 7462373 [MINOR] Add the users@ mailing list

[GitHub] [hudi] leesf merged pull request #1778: [MINOR] Add the users@ mailing list to the community page

leesf merged pull request #1778: URL: https://github.com/apache/hudi/pull/1778 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] yanghua commented on pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

yanghua commented on pull request #1779: URL: https://github.com/apache/hudi/pull/1779#issuecomment-652767915 @Trevor-zhang pls check the reason why Travis is red. Will merge it, after it turns into green. This is an

[GitHub] [hudi] yanghua commented on a change in pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

yanghua commented on a change in pull request #1779: URL: https://github.com/apache/hudi/pull/1779#discussion_r448734416 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -202,9 +202,14 @@ public

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #326

See Changes: -- [...truncated 2.30 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[GitHub] [hudi] wangxianghu commented on a change in pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

wangxianghu commented on a change in pull request #1779: URL: https://github.com/apache/hudi/pull/1779#discussion_r448728114 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -202,9 +202,14 @@ public

[GitHub] [hudi] yanghua commented on a change in pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

yanghua commented on a change in pull request #1779: URL: https://github.com/apache/hudi/pull/1779#discussion_r448723699 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -202,9 +202,14 @@ public

[GitHub] [hudi] codecov-commenter commented on pull request #1149: [WIP] [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

codecov-commenter commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-652734921 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1149?src=pr=h1) Report > Merging [#1149](https://codecov.io/gh/apache/hudi/pull/1149?src=pr=desc) into

[GitHub] [hudi] clocklear commented on pull request #1781: [MINOR] Relocate jetty during shading/packaging

clocklear commented on pull request #1781: URL: https://github.com/apache/hudi/pull/1781#issuecomment-652605410 cc @vinothchandar , apologies for directly involving you, but not sure how to get this item noticed. This is an

[GitHub] [hudi] prashanthpdesai commented on issue #1775: INCREMETNAL QUERY-Null value Exception

prashanthpdesai commented on issue #1775: URL: https://github.com/apache/hudi/issues/1775#issuecomment-652599980 Sure @bhasudha 1. please find the spark spark-shell --master yarn --queue queue_q1 --jars

[GitHub] [hudi] clocklear opened a new pull request #1781: [MINOR] Relocate jetty during shading/packaging

clocklear opened a new pull request #1781: URL: https://github.com/apache/hudi/pull/1781 ## What is the purpose of the pull request This pull-request relocates jetty during the packaging of `hudi-spark-bundle`. I'm testing Hudi in Databricks and found there is a transitive

[GitHub] [hudi] WaterKnight1998 commented on issue #1777: [SUPPORT] org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were

WaterKnight1998 commented on issue #1777: URL: https://github.com/apache/hudi/issues/1777#issuecomment-652597165 > Ah okay, I think these are default values for the configs. You would need configure each of them based on table schema. Here is the config session that has explanation of

[GitHub] [hudi] bhasudha commented on issue #1777: [SUPPORT] org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were

bhasudha commented on issue #1777: URL: https://github.com/apache/hudi/issues/1777#issuecomment-652590917 Ah okay, I think these are default values for the configs. You would need configure each of them based on table schema. Here is the config session that has explanation of these

[GitHub] [hudi] bhasudha commented on issue #1775: INCREMETNAL QUERY-Null value Exception

bhasudha commented on issue #1775: URL: https://github.com/apache/hudi/issues/1775#issuecomment-652586696 @prashanthpdesai thanks for the update. Just wanted to point you to a NOTE on Fetch task in Hive here - https://hudi.apache.org/docs/querying_data.html#hive So IIUC, there

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1558: [HUDI-796]: added deduping logic for upserts case

pratyakshsharma commented on a change in pull request #1558: URL: https://github.com/apache/hudi/pull/1558#discussion_r448523173 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/RepairsCommand.java ## @@ -77,16 +77,21 @@ public String deduplicate(

[GitHub] [hudi] zuyanton opened a new issue #1780: [SUPPORT]IllegalStateException: Hudi File Id has more than 1 pending compactions. MoR. Compaction inline.

zuyanton opened a new issue #1780: URL: https://github.com/apache/hudi/issues/1780 We are having an issue when running simple count query on our hudi table via hive. the error is Hudi File Id has more then one pending compactions. The table is MoR , compaction gets executed in line,

[GitHub] [hudi] pratyakshsharma commented on pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on pull request #1433: URL: https://github.com/apache/hudi/pull/1433#issuecomment-652558032 > LGTM. can you squash all commits and let me know I have already squashed @nsivabalan :) This is

[GitHub] [hudi] lw309637554 commented on pull request #1756: [HUDI-839] Adding unit test for MarkerFiles,RollbackUtils, RollbackActionExecutor for markers and filelisting

lw309637554 commented on pull request #1756: URL: https://github.com/apache/hudi/pull/1756#issuecomment-652546754 > > for rollback successful commit, in HoodieWriteClient.java i remove the deleteMarkerDir() in postcommit when is in usingmarkers mode. But it will double the file numbers in

[GitHub] [hudi] WaterKnight1998 edited a comment on issue #1777: [SUPPORT] org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were

WaterKnight1998 edited a comment on issue #1777: URL: https://github.com/apache/hudi/issues/1777#issuecomment-652446917 > @WaterKnight1998 It looks like this field `ts` is not there in the record. Could you print the table schema ? Also which version of Hudi are you using ?

[GitHub] [hudi] prashanthpdesai edited a comment on issue #1775: INCREMETNAL QUERY-Null value Exception

prashanthpdesai edited a comment on issue #1775: URL: https://github.com/apache/hudi/issues/1775#issuecomment-652208979 @bhasudha : we are using master jar . Just an added information vinoth @vinothchandar wanted to share information here ,have created external table manually on

[GitHub] [hudi] WaterKnight1998 commented on issue #1777: [SUPPORT] org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were

WaterKnight1998 commented on issue #1777: URL: https://github.com/apache/hudi/issues/1777#issuecomment-652446917 > @WaterKnight1998 It looks like this field `ts` is not there in the record. Could you print the table schema ? Also which version of Hudi are you using ? @bhasudha here

[GitHub] [hudi] leesf commented on pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on pull request #1761: URL: https://github.com/apache/hudi/pull/1761#issuecomment-652406919 @afeldman1 would you please address the comments and push again, we will be home. Thanks This is an automated

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r448345964 ## File path: docs/_docs/1_1_quick_start_guide.md ## @@ -117,7 +117,7 @@ df.write.format("hudi"). options(getQuickstartWriteConfigs).

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r448345964 ## File path: docs/_docs/1_1_quick_start_guide.md ## @@ -117,7 +117,7 @@ df.write.format("hudi"). options(getQuickstartWriteConfigs).

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r448345240 ## File path: docs/_docs/2_2_writing_data.md ## @@ -176,15 +176,49 @@ In some cases, you may want to migrate your existing table into Hudi beforehand. ##

[GitHub] [hudi] pengzhiwei2018 commented on pull request #1104: [HUDI-404] fix the error of compiling project.

pengzhiwei2018 commented on pull request #1104: URL: https://github.com/apache/hudi/pull/1104#issuecomment-652274417 Hi @wojustme ,I meet the same problem with you. I have to add "jsr305" dependency alone to solve this problem. Can you share you solution? THK

[jira] [Comment Edited] (HUDI-480) Support a querying delete data methond in incremental view

[ https://issues.apache.org/jira/browse/HUDI-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149186#comment-17149186 ] vinoyang edited comment on HUDI-480 at 7/1/20, 7:58 AM: [~vinoth] For the hard

[jira] [Commented] (HUDI-480) Support a querying delete data methond in incremental view

[ https://issues.apache.org/jira/browse/HUDI-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149186#comment-17149186 ] vinoyang commented on HUDI-480: --- For the hard deletion, can we log the row key list as the metadata of a

[jira] [Comment Edited] (HUDI-480) Support a querying delete data methond in incremental view

[ https://issues.apache.org/jira/browse/HUDI-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149186#comment-17149186 ] vinoyang edited comment on HUDI-480 at 7/1/20, 7:28 AM: [~vinoth] For the hard

[jira] [Commented] (HUDI-480) Support a querying delete data methond in incremental view

[ https://issues.apache.org/jira/browse/HUDI-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149182#comment-17149182 ] vinoyang commented on HUDI-480: --- [~chenxiang] What's the progress? > Support a querying delete data methond

[jira] [Updated] (HUDI-1062) Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

[ https://issues.apache.org/jira/browse/HUDI-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1062: - Labels: pull-request-available (was: ) > Remove unnecessary maxEvent check and add some log in

[GitHub] [hudi] Trevor-zhang opened a new pull request #1779: [HUDI-1062]Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

Trevor-zhang opened a new pull request #1779: URL: https://github.com/apache/hudi/pull/1779 KafkaOffsetGen ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ##

[GitHub] [hudi] prashanthpdesai edited a comment on issue #1775: INCREMETNAL QUERY-Null value Exception

prashanthpdesai edited a comment on issue #1775: URL: https://github.com/apache/hudi/issues/1775#issuecomment-652208979 @bhasudha : we are using master jar . This is an automated message from the Apache Git Service. To