Build failed in Jenkins: hudi-snapshot-deployment-0.5 #315

See Changes: -- [...truncated 2.38 KB...] settings.xml toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging: simplelogger.properties

[GitHub] [hudi] vinothchandar commented on pull request #1100: [HUDI-289] Implement a test suite to support long running test for Hudi writing and querying end-end

vinothchandar commented on pull request #1100: URL: https://github.com/apache/hudi/pull/1100#issuecomment-647073091 > The plan is to setup the nightly performance build by @yanghua and a bunch of follow up items that probably captures the essence of what you have in mind for th As

[GitHub] [hudi] vinothchandar commented on pull request #1100: [HUDI-289] Implement a test suite to support long running test for Hudi writing and querying end-end

vinothchandar commented on pull request #1100: URL: https://github.com/apache/hudi/pull/1100#issuecomment-647073129 I can take a final pass, make any small edits myself and merge once you do the renaming This is an

[GitHub] [hudi] vinothchandar commented on pull request #1100: [HUDI-289] Implement a test suite to support long running test for Hudi writing and querying end-end

vinothchandar commented on pull request #1100: URL: https://github.com/apache/hudi/pull/1100#issuecomment-647072976 @n3nash let’s please call this integration test to be consistent with the direction @xushiyan has been pushing towards. Unit, integration, functional are standard test

[GitHub] [hudi] vinothchandar commented on issue #1747: [SUPPORT] HiveSynctool syncs wrong location

vinothchandar commented on issue #1747: URL: https://github.com/apache/hudi/issues/1747#issuecomment-647072617 @bhasudha can you please take a pass This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] vinothchandar commented on issue #933: Support for multiple level partitioning in Hudi

vinothchandar commented on issue #933: URL: https://github.com/apache/hudi/issues/933#issuecomment-647072567 @afeldman1 http://hudi.apache.org/contributing#website sorry if this was a bit obscure :) site content lives in a `asf-site` branch

[GitHub] [hudi] vinothchandar commented on a change in pull request #1748: [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

vinothchandar commented on a change in pull request #1748: URL: https://github.com/apache/hudi/pull/1748#discussion_r443176993 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java ## @@ -27,10 +27,10 @@ import

[jira] [Commented] (HUDI-1015) Audit all getAllPartitionPaths() calls and keep em out of fast path

[ https://issues.apache.org/jira/browse/HUDI-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141271#comment-17141271 ] Vinoth Chandar commented on HUDI-1015: -- Hi renyi , goal here is to audit all these calls that may be

[GitHub] [hudi] xushiyan closed pull request #1619: [HUDI-896] Parallelize CI tests by modules

xushiyan closed pull request #1619: URL: https://github.com/apache/hudi/pull/1619 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] xushiyan commented on pull request #1619: [HUDI-896] Parallelize CI tests by modules

xushiyan commented on pull request #1619: URL: https://github.com/apache/hudi/pull/1619#issuecomment-647068902 Thank you all for the feedbacks..will look into codecov.. closing this for now This is an automated message from

[GitHub] [hudi] xushiyan commented on pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on pull request #1746: URL: https://github.com/apache/hudi/pull/1746#issuecomment-647062786 > @xushiyan codecov seems happy? @vinothchandar When we split some tests from unit tests to another travis job, we lost some coverage reporting in the final report, due to

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443170684 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/SharedResources.java ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443169927 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/SharedResources.java ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443169749 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/DFSProvider.java ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443169804 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/SharedResources.java ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite for hudi-utilities

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443169701 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/checkpointing/TestKafkaConnectHdfsProvider.java ## @@ -19,37 +19,39 @@ package

[jira] [Created] (HUDI-1034) Document info about test structure and guide

Raymond Xu created HUDI-1034: Summary: Document info about test structure and guide Key: HUDI-1034 URL: https://issues.apache.org/jira/browse/HUDI-1034 Project: Apache Hudi Issue Type:

[GitHub] [hudi] xushiyan commented on a change in pull request #1746: [HUDI-996] Add functional test suite

xushiyan commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443169647 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] xushiyan opened a new pull request #1753: [WIP] [HUDI-896] Report test coverages by modules

xushiyan opened a new pull request #1753: URL: https://github.com/apache/hudi/pull/1753 wip ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary

[GitHub] [hudi] bvaradar commented on a change in pull request #1690: [HUDI-908] Add some data types to HoodieTestDataGenerator and fix some bugs

bvaradar commented on a change in pull request #1690: URL: https://github.com/apache/hudi/pull/1690#discussion_r443164942 ## File path: hudi-utilities/src/test/resources/delta-streamer-config/source.avsc ## @@ -43,8 +43,41 @@ }, { "name" : "end_lon", "type" :

[GitHub] [hudi] bvaradar commented on a change in pull request #1752: [WIP] [HUDI-575] Support Async Compaction for spark streaming writes to hudi table

bvaradar commented on a change in pull request #1752: URL: https://github.com/apache/hudi/pull/1752#discussion_r443162781 ## File path: hudi-client/src/main/java/org/apache/hudi/async/AsyncCompactService.java ## @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (HUDI-575) Support Async Compaction for spark streaming writes to hudi table

[ https://issues.apache.org/jira/browse/HUDI-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-575: Labels: pull-request-available (was: ) > Support Async Compaction for spark streaming writes to

[GitHub] [hudi] bvaradar opened a new pull request #1752: [WIP] [HUDI-575] Support Async Compaction for spark streaming writes to hudi table

bvaradar opened a new pull request #1752: URL: https://github.com/apache/hudi/pull/1752 This PR is dependent on #1577 It has 2 commits: The first commit corresponds to #1577 and the second commit is for this PR. Contains: 1. Structured Streaming Async Compaction Support

[jira] [Updated] (HUDI-1033) Remove redundant CLI tests

[ https://issues.apache.org/jira/browse/HUDI-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1033: - Status: Open (was: New) > Remove redundant CLI tests > --- > >

[GitHub] [hudi] bvaradar commented on pull request #1577: [HUDI-855] Run Auto Cleaner in parallel with ingestion

bvaradar commented on pull request #1577: URL: https://github.com/apache/hudi/pull/1577#issuecomment-647048778 @vinothchandar : Ready for review. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] bvaradar commented on a change in pull request #1577: [HUDI-855] Run Auto Cleaner in parallel with ingestion

bvaradar commented on a change in pull request #1577: URL: https://github.com/apache/hudi/pull/1577#discussion_r443162134 ## File path: hudi-cli/src/test/java/org/apache/hudi/cli/integ/ITTestCleansCommand.java ## @@ -1,106 +0,0 @@ -/* Review comment: @yanghua : I went

[jira] [Assigned] (HUDI-1033) Remove redundant CLI tests

[ https://issues.apache.org/jira/browse/HUDI-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-1033: Assignee: vinoyang > Remove redundant CLI tests > --- > >

[jira] [Created] (HUDI-1033) Remove redundant CLI tests

Balaji Varadarajan created HUDI-1033: Summary: Remove redundant CLI tests Key: HUDI-1033 URL: https://issues.apache.org/jira/browse/HUDI-1033 Project: Apache Hudi Issue Type: Task

[GitHub] [hudi] lyogev opened a new issue #1751: [SUPPORT] Hudi not working with Spark 3.0.0

lyogev opened a new issue #1751: URL: https://github.com/apache/hudi/issues/1751 **Describe the problem you faced** Trying to run hudi with spark 3.0.0, and getting an error **To Reproduce** Steps to reproduce the behavior: **Expected behavior**

[GitHub] [hudi] bvaradar commented on a change in pull request #1577: [WIP] [HUDI-855] Run Auto Cleaner in parallel with ingestion

bvaradar commented on a change in pull request #1577: URL: https://github.com/apache/hudi/pull/1577#discussion_r443157306 ## File path: hudi-common/src/main/java/org/apache/hudi/common/async/AbstractAsyncService.java ## @@ -16,7 +16,7 @@ * limitations under the License.

[GitHub] [hudi] bvaradar commented on a change in pull request #1577: [WIP] [HUDI-855] Run Auto Cleaner in parallel with ingestion

bvaradar commented on a change in pull request #1577: URL: https://github.com/apache/hudi/pull/1577#discussion_r443157280 ## File path: hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java ## @@ -677,4 +714,27 @@ private void rollbackPendingCommits() {

[GitHub] [hudi] bvaradar commented on a change in pull request #1577: [WIP] [HUDI-855] Run Auto Cleaner in parallel with ingestion

bvaradar commented on a change in pull request #1577: URL: https://github.com/apache/hudi/pull/1577#discussion_r443157218 ## File path: hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java ## @@ -339,8 +352,13 @@ protected void

[GitHub] [hudi] wangxianghu commented on a change in pull request #1744: [HUDI-1027] Introduce TimestampBasedComplexKeyGenerator to support ti…

wangxianghu commented on a change in pull request #1744: URL: https://github.com/apache/hudi/pull/1744#discussion_r442913719 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/keygen/TestTimestampBasedKeyGenerator.java ## @@ -91,8 +94,9 @@ public void

[jira] [Resolved] (HUDI-1023) Add validation error messages in delta sync

[ https://issues.apache.org/jira/browse/HUDI-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1023. - Resolution: Fixed Fixed via master: 8a9fdd603e3e532ea5252b98205acfb8aa648795 > Add validation error messages in

[jira] [Closed] (HUDI-1023) Add validation error messages in delta sync

[ https://issues.apache.org/jira/browse/HUDI-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1023. --- > Add validation error messages in delta sync > --- > > Key:

[jira] [Resolved] (HUDI-696) Add unit test for CommitsCommand

[ https://issues.apache.org/jira/browse/HUDI-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-696. Fix Version/s: 0.6.0 Resolution: Fixed Fixed via master: f3a701757b9fb838acc4fb2975f378009d71f104 > Add unit

[jira] [Closed] (HUDI-696) Add unit test for CommitsCommand

[ https://issues.apache.org/jira/browse/HUDI-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-696. -- > Add unit test for CommitsCommand > > > Key: HUDI-696 >

[jira] [Updated] (HUDI-696) Add unit test for CommitsCommand

[ https://issues.apache.org/jira/browse/HUDI-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-696: --- Status: Open (was: New) > Add unit test for CommitsCommand > > >

[GitHub] [hudi] vinothchandar commented on a change in pull request #1746: [HUDI-996] Add functional test suite

vinothchandar commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443131872 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/checkpointing/TestKafkaConnectHdfsProvider.java ## @@ -19,37 +19,39 @@ package

[jira] [Updated] (HUDI-1032) Remove unused classes in HoodieCopyOnWriteTable and code clean

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Status: In Progress (was: Open) > Remove unused classes in HoodieCopyOnWriteTable and code clean >

[jira] [Updated] (HUDI-1032) Remove unused classes in HoodieCopyOnWriteTable and code clean

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Status: Open (was: New) > Remove unused classes in HoodieCopyOnWriteTable and code clean >

[jira] [Updated] (HUDI-1032) Remove unused classes in HoodieCopyOnWriteTable and code clean

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Summary: Remove unused classes in HoodieCopyOnWriteTable and code clean (was: Remove unused code in

[GitHub] [hudi] vinothchandar commented on pull request #1746: [HUDI-996] Add functional test suite

vinothchandar commented on pull request #1746: URL: https://github.com/apache/hudi/pull/1746#issuecomment-646993804 @xushiyan codecov seems happy? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] wangxianghu commented on pull request #1750: [HUDI-1032]Remove unused code in HoodieCopyOnWriteTable and code clean

wangxianghu commented on pull request #1750: URL: https://github.com/apache/hudi/pull/1750#issuecomment-646993606 Hi @yanghua, please take a look when free This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] vinothchandar commented on a change in pull request #1746: [HUDI-996] Add functional test suite

vinothchandar commented on a change in pull request #1746: URL: https://github.com/apache/hudi/pull/1746#discussion_r443130343 ## File path: scripts/run_travis_tests.sh ## @@ -20,12 +20,13 @@ mode=$1 sparkVersion=2.4.4 hadoopVersion=2.7 -if [ "$mode" = "unit" ]; -then +if

[GitHub] [hudi] vinothchandar commented on issue #1741: How to ignore the null columns in upsert on MoR tables? - Spark streaming

vinothchandar commented on issue #1741: URL: https://github.com/apache/hudi/issues/1741#issuecomment-646993572 @harishchanderramesh if you are looking for the specific config, it's https://hudi.apache.org/docs/configurations.html#PAYLOAD_CLASS_OPT_KEY if you are already deploying

[jira] [Updated] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable and code clean

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1032: - Labels: pull-request-available (was: ) > Remove unused code in HoodieCopyOnWriteTable and code

[GitHub] [hudi] wangxianghu opened a new pull request #1750: [HUDI-1032]Remove unused code in HoodieCopyOnWriteTable and code clean

wangxianghu opened a new pull request #1750: URL: https://github.com/apache/hudi/pull/1750 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[jira] [Updated] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable and code clean

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Summary: Remove unused code in HoodieCopyOnWriteTable and code clean (was: Remove unused code in

[jira] [Updated] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Description: Currently, SmallFile, InsertBucket, BucketType and BucketInfo had been introduced as 

[jira] [Updated] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1032: -- Description: Currently, SmallFile, InsertBucket, and BucketInfo had been introduced as independent

[jira] [Assigned] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable

[ https://issues.apache.org/jira/browse/HUDI-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-1032: - Assignee: wangxianghu > Remove unused code in HoodieCopyOnWriteTable >

[jira] [Created] (HUDI-1032) Remove unused code in HoodieCopyOnWriteTable

wangxianghu created HUDI-1032: - Summary: Remove unused code in HoodieCopyOnWriteTable Key: HUDI-1032 URL: https://issues.apache.org/jira/browse/HUDI-1032 Project: Apache Hudi Issue Type: Task

[jira] [Comment Edited] (HUDI-340) Increase Default max events to read from kafka source

[ https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141019#comment-17141019 ] wangxianghu edited comment on HUDI-340 at 6/20/20, 12:04 PM: - Hi [~Pratyaksh],

[jira] [Assigned] (HUDI-396) Provide an documentation to describe how to use test suite

[ https://issues.apache.org/jira/browse/HUDI-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu reassigned HUDI-396: Assignee: Trevorzhang (was: wangxianghu) > Provide an documentation to describe how to use test

[GitHub] [hudi] shenh062326 commented on pull request #1690: [HUDI-908] Add some data types to HoodieTestDataGenerator and fix some bugs

shenh062326 commented on pull request #1690: URL: https://github.com/apache/hudi/pull/1690#issuecomment-646971258 @bvaradar I have add all the missing types and fix some bugs. This is an automated message from the Apache

[jira] [Comment Edited] (HUDI-340) Increase Default max events to read from kafka source

[ https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141019#comment-17141019 ] wangxianghu edited comment on HUDI-340 at 6/20/20, 9:18 AM: Hi [~Pratyaksh],

[jira] [Comment Edited] (HUDI-340) Increase Default max events to read from kafka source

[ https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141019#comment-17141019 ] wangxianghu edited comment on HUDI-340 at 6/20/20, 9:17 AM: Hi [~Pratyaksh],

[jira] [Commented] (HUDI-340) Increase Default max events to read from kafka source

[ https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141019#comment-17141019 ] wangxianghu commented on HUDI-340: -- Hi [~Pratyaksh], thanks for the feedback! yes, the user will never

[jira] [Commented] (HUDI-340) Increase Default max events to read from kafka source

[ https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140999#comment-17140999 ] Pratyaksh Sharma commented on HUDI-340: --- Hi [~wangxianghu], the idea behind having these checks is