[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable

pratyakshsharma commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable URL: https://github.com/apache/incubator-hudi/pull/1039#discussion_r350581065 ## File path:

[jira] [Assigned] (HUDI-209) Implement JMX metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-209: - Assignee: (was: vinoyang) > Implement JMX metrics reporter > -- > >

[jira] [Commented] (HUDI-118) Provide CLI Option for passing properties to Compactor, Cleaner and ParquetImporter

[ https://issues.apache.org/jira/browse/HUDI-118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982116#comment-16982116 ] Vinoth Chandar commented on HUDI-118: - Hmmm. I dont have context into this myself. Have to kick this

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #110

See Changes: -- [...truncated 2.18 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/bin: m2.conf mvn mvn.cmd mvnDebug mvnDebug.cmd mvnyjp

[jira] [Commented] (HUDI-184) Integrate Hudi with Apache Flink

[ https://issues.apache.org/jira/browse/HUDI-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982107#comment-16982107 ] Vinoth Chandar commented on HUDI-184: - Okay cool then. On using Flink’s state as index, if you take a

[jira] [Commented] (HUDI-118) Provide CLI Option for passing properties to Compactor, Cleaner and ParquetImporter

[ https://issues.apache.org/jira/browse/HUDI-118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982085#comment-16982085 ] vinoyang commented on HUDI-118: --- [~Pratyaksh] I have nothing to supply. Let [~vinoth] make the decision. >

[jira] [Commented] (HUDI-358) Add Java-doc and importOrder checkstyle rule

[ https://issues.apache.org/jira/browse/HUDI-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982083#comment-16982083 ] lamber-ken commented on HUDI-358: - Hi, [~yanghua], I also can't assign this to myself. [~vinoth] can you

[jira] [Commented] (HUDI-184) Integrate Hudi with Apache Flink

[ https://issues.apache.org/jira/browse/HUDI-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982081#comment-16982081 ] vinoyang commented on HUDI-184: --- {quote}In spark, we can run two spark jobs  (i.e the jobs tab you see in

[jira] [Commented] (HUDI-358) Add Java-doc and importOrder checkstyle rule

[ https://issues.apache.org/jira/browse/HUDI-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982080#comment-16982080 ] vinoyang commented on HUDI-358: --- [~lamber-ken] I can not assign this ticket to you. You can specify to

[jira] [Updated] (HUDI-367) Implement Prometheus metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken updated HUDI-367: Labels: WIP (was: ) > Implement Prometheus metrics reporter > - > >

[jira] [Created] (HUDI-367) Implement Prometheus metrics reporter

lamber-ken created HUDI-367: --- Summary: Implement Prometheus metrics reporter Key: HUDI-367 URL: https://issues.apache.org/jira/browse/HUDI-367 Project: Apache Hudi (incubating) Issue Type: New

[jira] [Updated] (HUDI-367) Implement Prometheus metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken updated HUDI-367: Description: Implement PrometheusGateway metrics reporter, it is a push model metrics system.

[jira] [Created] (HUDI-366) Refactor hudi-hadoop-mr based on new ImportOrder code style rule

lamber-ken created HUDI-366: --- Summary: Refactor hudi-hadoop-mr based on new ImportOrder code style rule Key: HUDI-366 URL: https://issues.apache.org/jira/browse/HUDI-366 Project: Apache Hudi (incubating)

[jira] [Updated] (HUDI-364) Refactor hudi-hive based on new ImportOrder code style rule

[ https://issues.apache.org/jira/browse/HUDI-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-364: Labels: pull-request-available (was: ) > Refactor hudi-hive based on new ImportOrder code style

[GitHub] [incubator-hudi] lamber-ken opened a new pull request #1048: [HUDI-364] Refactor hudi-hive based on new ImportOrder code style rule

lamber-ken opened a new pull request #1048: [HUDI-364] Refactor hudi-hive based on new ImportOrder code style rule URL: https://github.com/apache/incubator-hudi/pull/1048 ## What is the purpose of the pull request Refactor hudi-hive based on new ImportOrder code style rule ##

[jira] [Created] (HUDI-365) Refactor hudi-cli based on new ImportOrder code style rule

Gurudatt Kulkarni created HUDI-365: -- Summary: Refactor hudi-cli based on new ImportOrder code style rule Key: HUDI-365 URL: https://issues.apache.org/jira/browse/HUDI-365 Project: Apache Hudi

[jira] [Comment Edited] (HUDI-354) Introduce stricter comment and code style validation rules

[ https://issues.apache.org/jira/browse/HUDI-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982033#comment-16982033 ] lamber-ken edited comment on HUDI-354 at 11/26/19 1:53 AM: --- Because of it's a big

[jira] [Commented] (HUDI-354) Introduce stricter comment and code style validation rules

[ https://issues.apache.org/jira/browse/HUDI-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982050#comment-16982050 ] Gurudatt Kulkarni commented on HUDI-354: [~lamber-ken] Awesome. Will rebase to master. Thanks. >

[jira] [Assigned] (HUDI-349) Make cleaner retention based on time period to account for higher deviations in ingestion runs

[ https://issues.apache.org/jira/browse/HUDI-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-349: --- Assignee: Aravind Suresh (was: Balaji Varadarajan) > Make cleaner retention based on

[jira] [Assigned] (HUDI-349) Make cleaner retention based on time period to account for higher deviations in ingestion runs

[ https://issues.apache.org/jira/browse/HUDI-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-349: --- Assignee: Balaji Varadarajan > Make cleaner retention based on time period to account

[jira] [Created] (HUDI-364) Refactor hudi-hive based on new ImportOrder code style rule

lamber-ken created HUDI-364: --- Summary: Refactor hudi-hive based on new ImportOrder code style rule Key: HUDI-364 URL: https://issues.apache.org/jira/browse/HUDI-364 Project: Apache Hudi (incubating)

[jira] [Updated] (HUDI-363) Refactor codes based on ImportOrder code style rule

[ https://issues.apache.org/jira/browse/HUDI-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken updated HUDI-363: Description: Refactor codes based on ImportOrder code style rules. Manay places need to refactor, so this

[jira] [Updated] (HUDI-363) Refactor codes based on ImportOrder code style rule

[ https://issues.apache.org/jira/browse/HUDI-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken updated HUDI-363: Description: Refactor codes based on ImportOrder code style rules. Manay places need to refactor, so this

[jira] [Commented] (HUDI-354) Introduce stricter comment and code style validation rules

[ https://issues.apache.org/jira/browse/HUDI-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982033#comment-16982033 ] lamber-ken commented on HUDI-354: - Because of it's a big work to do for each new checkstyle rule, so I

[jira] [Created] (HUDI-363) Refactor codes based on ImportOrder code style rule

lamber-ken created HUDI-363: --- Summary: Refactor codes based on ImportOrder code style rule Key: HUDI-363 URL: https://issues.apache.org/jira/browse/HUDI-363 Project: Apache Hudi (incubating) Issue

[jira] [Commented] (HUDI-354) Introduce stricter comment and code style validation rules

[ https://issues.apache.org/jira/browse/HUDI-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982020#comment-16982020 ] lamber-ken commented on HUDI-354: - hi, [~gurudatt], you can rebase the master branch, and you will see the

[GitHub] [incubator-hudi] bschell commented on issue #1040: [HUDI-327] Add null/empty checks to key generators

bschell commented on issue #1040: [HUDI-327] Add null/empty checks to key generators URL: https://github.com/apache/incubator-hudi/pull/1040#issuecomment-558409453 oops, my mistake. Should be good now. This is an automated

[jira] [Commented] (HUDI-184) Integrate Hudi with Apache Flink

[ https://issues.apache.org/jira/browse/HUDI-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981964#comment-16981964 ] Vinoth Chandar commented on HUDI-184: - >> Yes, It seems the ingestion and compaction steps are

[GitHub] [incubator-hudi] leesf commented on issue #1039: [HUDI-340]: made max events to read from kafka source configurable

leesf commented on issue #1039: [HUDI-340]: made max events to read from kafka source configurable URL: https://github.com/apache/incubator-hudi/pull/1039#issuecomment-558364384 @pratyakshsharma Thanks for your work. It's almost ready.

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable

leesf commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable URL: https://github.com/apache/incubator-hudi/pull/1039#discussion_r350450697 ## File path:

[jira] [Closed] (HUDI-362) Adds a check for the existence of field

[ https://issues.apache.org/jira/browse/HUDI-362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-362. -- Fix Version/s: 0.5.1 Resolution: Fixed Fixed via master: 44823041a37601fed8163502272a8fcb7a5be45d > Adds a check

[jira] [Closed] (HUDI-358) Add Java-doc and importOrder checkstyle rule

[ https://issues.apache.org/jira/browse/HUDI-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-358. -- Fix Version/s: 0.5.1 Resolution: Fixed Fixed via master: 212282c8aaf623f451e3f72674ed4d3ed550602d > Add Java-doc

[jira] [Closed] (HUDI-359) Add hudi-env for hudi-cli module

[ https://issues.apache.org/jira/browse/HUDI-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-359. -- Fix Version/s: 0.5.1 Resolution: Fixed Fixed via master: a7e07cd910425b5cfe9886677e780bfb2ae96c52 > Add hudi-env

[GitHub] [incubator-hudi] vinothchandar merged pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module

vinothchandar merged pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module URL: https://github.com/apache/incubator-hudi/pull/1042 This is an automated message from the Apache Git Service. To respond to the

[incubator-hudi] branch master updated: [HUDI-359] Add hudi-env for hudi-cli module (#1042)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new a7e07cd [HUDI-359] Add hudi-env for

[GitHub] [incubator-hudi] vinothchandar commented on issue #1040: [HUDI-327] Add null/empty checks to key generators

vinothchandar commented on issue #1040: [HUDI-327] Add null/empty checks to key generators URL: https://github.com/apache/incubator-hudi/pull/1040#issuecomment-558341088 I think its there is an error ``` [ERROR]

[GitHub] [incubator-hudi] bschell commented on issue #1040: [HUDI-327] Add null/empty checks to key generators

bschell commented on issue #1040: [HUDI-327] Add null/empty checks to key generators URL: https://github.com/apache/incubator-hudi/pull/1040#issuecomment-558313317 done This is an automated message from the Apache Git

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1001: [HUDI-325] Fix Hive partition error for updated HDFS Hudi table

vinothchandar commented on a change in pull request #1001: [HUDI-325] Fix Hive partition error for updated HDFS Hudi table URL: https://github.com/apache/incubator-hudi/pull/1001#discussion_r350386205 ## File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1001: [HUDI-325] Fix Hive partition error for updated HDFS Hudi table

vinothchandar commented on a change in pull request #1001: [HUDI-325] Fix Hive partition error for updated HDFS Hudi table URL: https://github.com/apache/incubator-hudi/pull/1001#discussion_r350386470 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/FSUtils.java

[jira] [Commented] (HUDI-76) CSV Source support for Hudi Delta Streamer

[ https://issues.apache.org/jira/browse/HUDI-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981839#comment-16981839 ] Vinoth Chandar commented on HUDI-76: > , can we assume that the CSV files sit on DFS so we only consider

[GitHub] [incubator-hudi] vinothchandar commented on issue #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule

vinothchandar commented on issue #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule URL: https://github.com/apache/incubator-hudi/pull/1043#issuecomment-558307683 Looks like a good rule to have. Just trying to understand implications.. Merging.

[incubator-hudi] branch master updated: [HUDI-358] Add Java-doc and importOrder checkstyle rule (#1043)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 212282c [HUDI-358] Add Java-doc and

[GitHub] [incubator-hudi] vinothchandar merged pull request #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule

vinothchandar merged pull request #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule URL: https://github.com/apache/incubator-hudi/pull/1043 This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-hudi] vinothchandar commented on issue #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule

vinothchandar commented on issue #1043: [HUDI-358] Add Java-doc and importOrder checkstyle rule URL: https://github.com/apache/incubator-hudi/pull/1043#issuecomment-558307083 @leesf @lamber-ken how does this apply to all the existing formatting already in code? do people hit this as they

[GitHub] [incubator-hudi] vinothchandar merged pull request #1047: [HUDI-362] Adds a check for the existence of field

vinothchandar merged pull request #1047: [HUDI-362] Adds a check for the existence of field URL: https://github.com/apache/incubator-hudi/pull/1047 This is an automated message from the Apache Git Service. To respond to the

[incubator-hudi] branch master updated: [HUDI-362] Adds a check for the existence of field (#1047)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 4482304 [HUDI-362] Adds a check for

[GitHub] [incubator-hudi] vinothchandar commented on issue #1047: [HUDI-362] Adds a check for the existence of field

vinothchandar commented on issue #1047: [HUDI-362] Adds a check for the existence of field URL: https://github.com/apache/incubator-hudi/pull/1047#issuecomment-558305590 Thanks @hddong ! This is an automated message from the

[GitHub] [incubator-hudi] vinothchandar commented on issue #1021: how can i deal this problem when partition's value changed with the same row_key?

vinothchandar commented on issue #1021: how can i deal this problem when partition's value changed with the same row_key? URL: https://github.com/apache/incubator-hudi/issues/1021#issuecomment-558302481 @nsivabalan Could you debug why the second write it not getting tagged to the first

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350199901 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/JmxMetricsReporter.java ## @@ -0,0

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350198461 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/MetricsReporter.java ## @@ -19,6

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350198077 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/MetricsReporter.java ## @@ -28,7

[jira] [Commented] (HUDI-118) Provide CLI Option for passing properties to Compactor, Cleaner and ParquetImporter

[ https://issues.apache.org/jira/browse/HUDI-118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981557#comment-16981557 ] Pratyaksh Sharma commented on HUDI-118: --- [~yanghua] I guess it is already done. The option is already

[GitHub] [incubator-hudi] leesf commented on issue #1040: [HUDI-327] Add null/empty checks to key generators

leesf commented on issue #1040: [HUDI-327] Add null/empty checks to key generators URL: https://github.com/apache/incubator-hudi/pull/1040#issuecomment-558118734 Hi @bschell Would you please rebase against master to solve the conflicts.

[GitHub] [incubator-hudi] leesf commented on issue #1044: [HUDI-361] Implement CSV metrics reporter

leesf commented on issue #1044: [HUDI-361] Implement CSV metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1044#issuecomment-558114628 Hi @XuQianJin-Stars I am also wondering whether this kind of reporter is easy and common to used?

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350127951 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/MetricsReporter.java ## @@ -28,7

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350128094 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/MetricsReporter.java ## @@ -19,6

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350127640 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/JmxMetricsReporter.java ## @@ -0,0

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter

leesf commented on a change in pull request #1045: [HUDI-209] Implement JMX metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1045#discussion_r350126884 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/JmxMetricsReporter.java ## @@ -0,0

[incubator-hudi] branch master updated (c335510 -> 845a050)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from c335510 [HUDI-328] Adding delete api to HoodieWriteClient (#1004) add 845a050 [MINOR] Some minor

[GitHub] [incubator-hudi] leesf merged pull request #1046: Use commitimestamp2 instead of commitimestamp1 in HoodieJavaStreamingApp

leesf merged pull request #1046: Use commitimestamp2 instead of commitimestamp1 in HoodieJavaStreamingApp URL: https://github.com/apache/incubator-hudi/pull/1046 This is an automated message from the Apache Git Service. To

[GitHub] [incubator-hudi] hddong commented on a change in pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module

hddong commented on a change in pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module URL: https://github.com/apache/incubator-hudi/pull/1042#discussion_r350072902 ## File path: hudi-cli/hudi-cli.sh ## @@ -20,15 +20,12 @@ DIR="$( cd "$( dirname

[GitHub] [incubator-hudi] hddong commented on a change in pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module

hddong commented on a change in pull request #1042: [HUDI-359] Add hudi-env for hudi-cli module URL: https://github.com/apache/incubator-hudi/pull/1042#discussion_r350072902 ## File path: hudi-cli/hudi-cli.sh ## @@ -20,15 +20,12 @@ DIR="$( cd "$( dirname

[GitHub] [incubator-hudi] hddong removed a comment on issue #1042: [HUDI-359] Add hudi-env for hudi-cli module

hddong removed a comment on issue #1042: [HUDI-359] Add hudi-env for hudi-cli module URL: https://github.com/apache/incubator-hudi/pull/1042#issuecomment-557976681 @vinothchandar , checks always have failed, could you have any suggestion and help.

[GitHub] [incubator-hudi] fbalicchia commented on issue #1046: Use commitimestamp2 instead of commitimestamp1 in HoodieJavaStreamingApp

fbalicchia commented on issue #1046: Use commitimestamp2 instead of commitimestamp1 in HoodieJavaStreamingApp URL: https://github.com/apache/incubator-hudi/pull/1046#issuecomment-558044299 > Please remove `unused import - java.util.concurrent.Callable`, and you would clikc the details

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable

leesf commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable URL: https://github.com/apache/incubator-hudi/pull/1039#discussion_r350031504 ## File path: