Build failed in Jenkins: hudi-snapshot-deployment-0.5 #282

See Changes: -- [...truncated 2.38 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[jira] [Commented] (HUDI-648) Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes

[ https://issues.apache.org/jira/browse/HUDI-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110824#comment-17110824 ] liujinhui commented on HUDI-648: Hello, are there any good ideas and design suggestions for this proposal?

[GitHub] [incubator-hudi] codecov-commenter edited a comment on pull request #1640: [WIP] travis jvm fork issue

codecov-commenter edited a comment on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630496124 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] codecov-commenter edited a comment on pull request #1640: [WIP] travis jvm fork issue

codecov-commenter edited a comment on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630496124 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[incubator-hudi] branch master updated: [HUDI-858] Allow multiple operations to be executed within a single commit (#1633)

This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new e6f3bf1 [HUDI-858] Allow multiple

[GitHub] [incubator-hudi] bvaradar merged pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

bvaradar merged pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-hudi] codecov-commenter edited a comment on pull request #1640: [WIP] travis jvm fork issue

codecov-commenter edited a comment on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630496124 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] codecov-commenter edited a comment on pull request #1640: [WIP] travis jvm fork issue

codecov-commenter edited a comment on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630496124 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] codecov-commenter edited a comment on pull request #1640: [WIP] travis jvm fork issue

codecov-commenter edited a comment on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630496124 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] codecov-io commented on pull request #1640: [WIP] travis jvm fork issue

codecov-io commented on pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640#issuecomment-630495938 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1640?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] bvaradar commented on pull request #1634: [HUDI-846][HUDI-848] Enable Incremental cleaning and embedded timeline-server by default

bvaradar commented on pull request #1634: URL: https://github.com/apache/incubator-hudi/pull/1634#issuecomment-630393889 @vinothchandar : Addressed review comment. This is an automated message from the Apache Git Service. To

[GitHub] [incubator-hudi] bvaradar commented on pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

bvaradar commented on pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633#issuecomment-630392894 @leesf: the code coverage looks good now. This is an automated message from the Apache Git

[GitHub] [incubator-hudi] xushiyan opened a new pull request #1640: [WIP] remove -Xms

xushiyan opened a new pull request #1640: URL: https://github.com/apache/incubator-hudi/pull/1640 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[incubator-hudi] branch master updated (459356e -> 2600d2d)

This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from 459356e [HUDI-863] get decimal properties from derived spark DataType (#1596) add 2600d2d [MINOR]

[GitHub] [incubator-hudi] bvaradar merged pull request #1639: [MINOR] Fix apache-rat violations

bvaradar merged pull request #1639: URL: https://github.com/apache/incubator-hudi/pull/1639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-hudi] bvaradar commented on pull request #1639: [MINOR] Fix apache-rat violations

bvaradar commented on pull request #1639: URL: https://github.com/apache/incubator-hudi/pull/1639#issuecomment-630352568 @vinothchandar : Added rat check to hudi-integ-test. This is an automated message from the Apache Git

[GitHub] [incubator-hudi] bvaradar commented on pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

bvaradar commented on pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633#issuecomment-630316425 @leesf : Looking at the code changes, I dont think it should cause such a loss. I had tried force-pushing to trigger another rounds of testing but I dont see

[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

codecov-io edited a comment on pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633#issuecomment-629848790 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1633?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] pratyakshsharma commented on pull request #1558: [HUDI-796]: added deduping logic for upserts case

pratyakshsharma commented on pull request #1558: URL: https://github.com/apache/incubator-hudi/pull/1558#issuecomment-630299786 @hddong shared the logs with you over slack. This is an automated message from the Apache Git

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on a change in pull request #1433: URL: https://github.com/apache/incubator-hudi/pull/1433#discussion_r426735110 ## File path: hudi-spark/src/main/java/org/apache/hudi/keygen/TimestampBasedKeyGenerator.java ## @@ -88,6 +86,13 @@ public

[GitHub] [incubator-hudi] pratyakshsharma commented on pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on pull request #1433: URL: https://github.com/apache/incubator-hudi/pull/1433#issuecomment-630278010 > Can you point me to exact commits where you addressed my last set of comments and commits where you pulled in the other PR? Sure. I addressed your last

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on a change in pull request #1433: URL: https://github.com/apache/incubator-hudi/pull/1433#discussion_r426730912 ## File path: hudi-spark/src/main/java/org/apache/hudi/keygen/CustomKeyGenerator.java ## @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on a change in pull request #1433: URL: https://github.com/apache/incubator-hudi/pull/1433#discussion_r426724593 ## File path: hudi-spark/src/main/java/org/apache/hudi/keygen/CustomKeyGenerator.java ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1433: [HUDI-728]: Implement custom key generator

pratyakshsharma commented on a change in pull request #1433: URL: https://github.com/apache/incubator-hudi/pull/1433#discussion_r426715335 ## File path: hudi-spark/src/main/java/org/apache/hudi/keygen/TimestampBasedKeyGenerator.java ## @@ -88,6 +86,13 @@ public

[jira] [Updated] (HUDI-913) Introduce HoodieEngineContext for hudi write client

[ https://issues.apache.org/jira/browse/HUDI-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-913: - Summary: Introduce HoodieEngineContext for hudi write client (was: Introduce HoodieEngineContext) >

[jira] [Updated] (HUDI-911) Introduce HoodieWriteOutput for hudi write client

[ https://issues.apache.org/jira/browse/HUDI-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-911: - Summary: Introduce HoodieWriteOutput for hudi write client (was: Introduce HoodieWriteOutput) >

[jira] [Updated] (HUDI-912) Introduce HoodieWriteKey for hudi write client

[ https://issues.apache.org/jira/browse/HUDI-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-912: - Summary: Introduce HoodieWriteKey for hudi write client (was: Introduce HoodieWriteKey) > Introduce

[jira] [Updated] (HUDI-910) Introduce HoodieWriteInput for hudi write client

[ https://issues.apache.org/jira/browse/HUDI-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-910: - Summary: Introduce HoodieWriteInput for hudi write client (was: Introduce HoodieWriteInput) > Introduce

[jira] [Assigned] (HUDI-913) Introduce HoodieEngineContext

[ https://issues.apache.org/jira/browse/HUDI-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-913: Assignee: wangxianghu > Introduce HoodieEngineContext > - > >

[jira] [Assigned] (HUDI-910) Introduce HoodieWriteInput

[ https://issues.apache.org/jira/browse/HUDI-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-910: Assignee: wangxianghu > Introduce HoodieWriteInput > -- > >

[jira] [Updated] (HUDI-912) Introduce HoodieWriteKey

[ https://issues.apache.org/jira/browse/HUDI-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-912: - Summary: Introduce HoodieWriteKey (was: Introduce HoodieWriteInput) > Introduce HoodieWriteKey >

[jira] [Assigned] (HUDI-912) Introduce HoodieWriteKey

[ https://issues.apache.org/jira/browse/HUDI-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-912: Assignee: wangxianghu > Introduce HoodieWriteKey > > >

[jira] [Updated] (HUDI-910) Introduce HoodieWriteInput

[ https://issues.apache.org/jira/browse/HUDI-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-910: - Summary: Introduce HoodieWriteInput (was: Introduce unified input format of Hudi) > Introduce

[jira] [Updated] (HUDI-912) Introduce HoodieWriteInput

[ https://issues.apache.org/jira/browse/HUDI-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-912: - Summary: Introduce HoodieWriteInput (was: Introduce unified Hudi key format) > Introduce

[jira] [Updated] (HUDI-913) Introduce HoodieEngineContext

[ https://issues.apache.org/jira/browse/HUDI-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-913: - Summary: Introduce HoodieEngineContext (was: Introduce HoodieEngineContext.java) > Introduce

[jira] [Updated] (HUDI-911) Introduce HoodieWriteOutput

[ https://issues.apache.org/jira/browse/HUDI-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-911: - Summary: Introduce HoodieWriteOutput (was: Introduce unified output format of Hudi) > Introduce

[jira] [Assigned] (HUDI-911) Introduce unified output format of Hudi

[ https://issues.apache.org/jira/browse/HUDI-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-911: Assignee: wangxianghu > Introduce unified output format of Hudi >

[jira] [Created] (HUDI-913) Introduce HoodieEngineContext.java

wangxianghu created HUDI-913: Summary: Introduce HoodieEngineContext.java Key: HUDI-913 URL: https://issues.apache.org/jira/browse/HUDI-913 Project: Apache Hudi (incubating) Issue Type: Sub-task

[jira] [Created] (HUDI-912) Introduce unified Hudi key format

wangxianghu created HUDI-912: Summary: Introduce unified Hudi key format Key: HUDI-912 URL: https://issues.apache.org/jira/browse/HUDI-912 Project: Apache Hudi (incubating) Issue Type: Sub-task

[jira] [Created] (HUDI-911) Introduce unified output format of Hudi

wangxianghu created HUDI-911: Summary: Introduce unified output format of Hudi Key: HUDI-911 URL: https://issues.apache.org/jira/browse/HUDI-911 Project: Apache Hudi (incubating) Issue Type:

[jira] [Created] (HUDI-910) Introduce unified input format of Hudi

wangxianghu created HUDI-910: Summary: Introduce unified input format of Hudi Key: HUDI-910 URL: https://issues.apache.org/jira/browse/HUDI-910 Project: Apache Hudi (incubating) Issue Type:

[jira] [Assigned] (HUDI-4) Support for writing to EMRFS

[ https://issues.apache.org/jira/browse/HUDI-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-4: --- Assignee: liujinhui (was: vinoyang) > Support for writing to EMRFS > > >

[jira] [Created] (HUDI-909) Introduce high level abstract of hudi write client

wangxianghu created HUDI-909: Summary: Introduce high level abstract of hudi write client Key: HUDI-909 URL: https://issues.apache.org/jira/browse/HUDI-909 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1572: [HUDI-836] Implement datadog metrics reporter

xushiyan commented on a change in pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572#discussion_r426634201 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/datadog/DatadogReporter.java ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] hddong commented on pull request #1558: [HUDI-796]: added deduping logic for upserts case

hddong commented on pull request #1558: URL: https://github.com/apache/incubator-hudi/pull/1558#issuecomment-630131586 @prashantwason ： Had send you message in slack, can I hava a look of your log? This is an automated

[GitHub] [incubator-hudi] nsivabalan commented on pull request #1602: [HUDI-494] fix incorrect record size estimation

nsivabalan commented on pull request #1602: URL: https://github.com/apache/incubator-hudi/pull/1602#issuecomment-630126880 I have started going thru the code base, but a naive question in the mean time. If we could log bytesOccupied by actual records separately and then bytes including

[GitHub] [incubator-hudi] vinothchandar merged pull request #1596: [HUDI-863] get decimal properties from derived spark DataType

vinothchandar merged pull request #1596: URL: https://github.com/apache/incubator-hudi/pull/1596 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[incubator-hudi] branch master updated: [HUDI-863] get decimal properties from derived spark DataType (#1596)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 459356e [HUDI-863] get decimal

[GitHub] [incubator-hudi] vinothchandar commented on pull request #1596: [HUDI-863] get decimal properties from derived spark DataType

vinothchandar commented on pull request #1596: URL: https://github.com/apache/incubator-hudi/pull/1596#issuecomment-630118280 Spent some time on this. It's note straight forward for a few reasons. the datasource tests convert avro to json back to DF.. I managed to add a decimal type at

[jira] [Commented] (HUDI-908) Consistent test data generation with data type coverage

[ https://issues.apache.org/jira/browse/HUDI-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110150#comment-17110150 ] Vinoth Chandar commented on HUDI-908: - Should include support for decimals, all the spark types at the

[jira] [Created] (HUDI-908) Consistent test data generation with data type coverage

Vinoth Chandar created HUDI-908: --- Summary: Consistent test data generation with data type coverage Key: HUDI-908 URL: https://issues.apache.org/jira/browse/HUDI-908 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1634: [WIP] [HUDI-846][HUDI-848] Enable Incremental cleaning and embedded timeline-server by default

vinothchandar commented on a change in pull request #1634: URL: https://github.com/apache/incubator-hudi/pull/1634#discussion_r426503762 ## File path: hudi-utilities/pom.xml ## @@ -82,6 +82,8 @@ org.eclipse.jetty.aggregate jetty-all +

[GitHub] [incubator-hudi] vinothchandar commented on pull request #1639: [MINOR] Fix apache-rat violations

vinothchandar commented on pull request #1639: URL: https://github.com/apache/incubator-hudi/pull/1639#issuecomment-630072442 @bvaradar `hudi-integ-test` also does not list the rat plugin.. Might be good to add these to bundles as well, just in case. (our release script will catch all

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

vinothchandar commented on a change in pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633#discussion_r426492576 ## File path: hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java ## @@ -99,6 +99,20 @@ private static final String

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1572: [HUDI-836] Implement datadog metrics reporter

yanghua commented on a change in pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572#discussion_r426435865 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/datadog/DatadogReporter.java ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] zhedoubushishi edited a comment on pull request #1638: HUDI-515 Resolve API conflict for Hive 2 & Hive 3

zhedoubushishi edited a comment on pull request #1638: URL: https://github.com/apache/incubator-hudi/pull/1638#issuecomment-629990534 > @zhedoubushishi what does it take to support Hive 3.x for Hudi with a mvn flag ? If we cannot support hive 3.x, what is the intention of this PR ? I'm

[GitHub] [incubator-hudi] zhedoubushishi commented on pull request #1638: HUDI-515 Resolve API conflict for Hive 2 & Hive 3

zhedoubushishi commented on pull request #1638: URL: https://github.com/apache/incubator-hudi/pull/1638#issuecomment-629990534 > @zhedoubushishi what does it take to support Hive 3.x for Hudi with a mvn flag ? If we cannot support hive 3.x, what is the intention of this PR ? I'm not very

[GitHub] [incubator-hudi] reste85 closed issue #1598: [SUPPORT] Slow upsert time reading from Kafka

reste85 closed issue #1598: URL: https://github.com/apache/incubator-hudi/issues/1598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [incubator-hudi] reste85 edited a comment on issue #1598: [SUPPORT] Slow upsert time reading from Kafka

reste85 edited a comment on issue #1598: URL: https://github.com/apache/incubator-hudi/issues/1598#issuecomment-629988384 Hi guys, As stated in chat, this is not related to Hudi in general. At first sight we thought that the problem was due this:

[GitHub] [incubator-hudi] reste85 commented on issue #1598: [SUPPORT] Slow upsert time reading from Kafka

reste85 commented on issue #1598: URL: https://github.com/apache/incubator-hudi/issues/1598#issuecomment-629988384 Hi guys, As stated in chat, this is not related to Hudi in general. At first sight we thought that the problem was due this:

[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1633: [HUDI-858] Allow multiple operations to be executed within a single commit

codecov-io edited a comment on pull request #1633: URL: https://github.com/apache/incubator-hudi/pull/1633#issuecomment-629848790 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1633?src=pr=h1) Report > Merging

[jira] [Updated] (HUDI-907) Test Presto mor query support changes in HDFS Env

[ https://issues.apache.org/jira/browse/HUDI-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha updated HUDI-907: --- Status: Open (was: New) > Test Presto mor query support changes in HDFS Env >

[jira] [Created] (HUDI-907) Test Presto mor query support changes in HDFS Env

Bhavani Sudha created HUDI-907: -- Summary: Test Presto mor query support changes in HDFS Env Key: HUDI-907 URL: https://issues.apache.org/jira/browse/HUDI-907 Project: Apache Hudi (incubating)

[jira] [Updated] (HUDI-305) Presto MOR "_rt" queries only reads base parquet file

[ https://issues.apache.org/jira/browse/HUDI-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha updated HUDI-305: --- Fix Version/s: 0.5.3 0.6.0 > Presto MOR "_rt" queries only reads base parquet file

[jira] [Updated] (HUDI-705) Add unit test for RollbacksCommand

[ https://issues.apache.org/jira/browse/HUDI-705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-705: -- Status: Open (was: New) > Add unit test for RollbacksCommand > -- > >

[incubator-hudi] branch master updated: [HUDI-705] Add unit test for RollbacksCommand (#1611)

This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 57132f7 [HUDI-705] Add unit test for