[jira] [Created] (HUDI-402) Code clean up in DataSourceUtils class

Pratyaksh Sharma created HUDI-402: - Summary: Code clean up in DataSourceUtils class Key: HUDI-402 URL: https://issues.apache.org/jira/browse/HUDI-402 Project: Apache Hudi (incubating) Issue

[GitHub] [incubator-hudi] hddong commented on issue #1096: [HUDI-398]Add set env for spark launcher

hddong commented on issue #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#issuecomment-564881150 > If have a command like `show env` will be better, just from my side.  Thanks, added `show env`

[jira] [Updated] (HUDI-288) Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

[ https://issues.apache.org/jira/browse/HUDI-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-288: -- Fix Version/s: 0.5.1 > Add support for ingesting multiple kafka streams in a single

[GitHub] [incubator-hudi] hddong opened a new pull request #1098: [HUDI-401]Remove unnecessary use of spark in savepoint timeline

hddong opened a new pull request #1098: [HUDI-401]Remove unnecessary use of spark in savepoint timeline URL: https://github.com/apache/incubator-hudi/pull/1098 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review

[jira] [Updated] (HUDI-401) Remove unnecessary use of spark in savepoint timeline

[ https://issues.apache.org/jira/browse/HUDI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-401: Labels: pull-request-available (was: ) > Remove unnecessary use of spark in savepoint timeline >

[jira] [Created] (HUDI-401) Remove unnecessary use of spark in savepoint timeline

hong dongdong created HUDI-401: -- Summary: Remove unnecessary use of spark in savepoint timeline Key: HUDI-401 URL: https://issues.apache.org/jira/browse/HUDI-401 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] yanghua commented on issue #626: Adding documentation for hudi test suite

yanghua commented on issue #626: Adding documentation for hudi test suite URL: https://github.com/apache/incubator-hudi/pull/626#issuecomment-564862931 IMO, it should be mapped with [HUDI-396](https://issues.apache.org/jira/browse/HUDI-396) and merged into `hudi_test_suite_refactor`

[jira] [Commented] (HUDI-334) Clean up terminologies in code/docs

[ https://issues.apache.org/jira/browse/HUDI-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994250#comment-16994250 ] Shahida Khan commented on HUDI-334: --- Fine by me. :) > Clean up terminologies in code/docs >

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect

lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-564689247 @imperio-wxm, you need to set the value of `DataSourceWriteOptions.HIVE_ASSUME_DATE_PARTITION_OPT_KEY()` to `false`.

[jira] [Commented] (HUDI-331) Fix java docs for all public apis (HoodieWriteClient)

[ https://issues.apache.org/jira/browse/HUDI-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994165#comment-16994165 ] sivabalan narayanan commented on HUDI-331: -- Nope. I have 3 pending work items that I intend to

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #126

See Changes: -- [...truncated 2.17 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/bin: m2.conf mvn mvn.cmd mvnDebug mvnDebug.cmd mvnyjp

[GitHub] [incubator-hudi] lamber-ken commented on issue #1096: [HUDI-398]Add set env for spark launcher

lamber-ken commented on issue #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#issuecomment-564837868 If have a command like `show env` will be better, just from my side. 

[jira] [Assigned] (HUDI-343) Create a DOAP File for Hudi

[ https://issues.apache.org/jira/browse/HUDI-343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf reassigned HUDI-343: -- Assignee: leesf > Create a DOAP File for Hudi > --- > > Key: HUDI-343 >

[GitHub] [incubator-hudi] leesf merged pull request #1093: [MINOR] replace scala map add operator

leesf merged pull request #1093: [MINOR] replace scala map add operator URL: https://github.com/apache/incubator-hudi/pull/1093 This is an automated message from the Apache Git Service. To respond to the message, please log

[incubator-hudi] branch master updated (ba514cf -> a405d38)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from ba514cf [MINOR] Remove redundant plus operator (#1097) add a405d38 [MINOR] replace scala map add

[GitHub] [incubator-hudi] lamber-ken commented on issue #1093: [MINOR] replace scala map add operator

lamber-ken commented on issue #1093: [MINOR] replace scala map add operator URL: https://github.com/apache/incubator-hudi/pull/1093#issuecomment-564832734 > IIRC the difference is in how the return list is typed? In this case, anyway both are Map[String,String] ? So it does not really

[GitHub] [incubator-hudi] vinothchandar commented on issue #626: Adding documentation for hudi test suite

vinothchandar commented on issue #626: Adding documentation for hudi test suite URL: https://github.com/apache/incubator-hudi/pull/626#issuecomment-564830667 @n3nash Can we move close this PR and move to cWiki. I feel this is purely a developer facing feature.. cc @yanghua

[GitHub] [incubator-hudi] vinothchandar closed pull request #657: [HUDI-57] support orc file

vinothchandar closed pull request #657: [HUDI-57] support orc file URL: https://github.com/apache/incubator-hudi/pull/657 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Updated] (HUDI-398) Add set env for spark launcher

[ https://issues.apache.org/jira/browse/HUDI-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-398: Status: Patch Available (was: In Progress) > Add set env for spark launcher >

[jira] [Updated] (HUDI-398) Add set env for spark launcher

[ https://issues.apache.org/jira/browse/HUDI-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-398: Fix Version/s: 0.5.1 > Add set env for spark launcher > -- > >

[GitHub] [incubator-hudi] vinothchandar commented on issue #1096: [HUDI-398]Add set env for spark launcher

vinothchandar commented on issue #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#issuecomment-564830351 LGTM overall. Over to you @leesf This is an automated message

[jira] [Updated] (HUDI-398) Add set env for spark launcher

[ https://issues.apache.org/jira/browse/HUDI-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-398: Status: In Progress (was: Open) > Add set env for spark launcher > -- >

[GitHub] [incubator-hudi] vinothchandar commented on issue #1093: [MINOR] replace scala map add operator

vinothchandar commented on issue #1093: [MINOR] replace scala map add operator URL: https://github.com/apache/incubator-hudi/pull/1093#issuecomment-564829804 IIRC the difference is in how the return list is typed? In this case, anyway both are Map[String,String] ? So it does not really

[jira] [Commented] (HUDI-343) Create a DOAP File for Hudi

[ https://issues.apache.org/jira/browse/HUDI-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994116#comment-16994116 ] Vinoth Chandar commented on HUDI-343: - Feel free to grab the ticket..  > Create a DOAP File for Hudi >

[jira] [Commented] (HUDI-343) Create a DOAP File for Hudi

[ https://issues.apache.org/jira/browse/HUDI-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994115#comment-16994115 ] Vinoth Chandar commented on HUDI-343: - Scoping what needs to be done is actually the first step :)  >

[jira] [Commented] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994110#comment-16994110 ] Vinoth Chandar commented on HUDI-251: - Let's may be wait one more day and reassign.. [~taherk77] Either

[jira] [Commented] (HUDI-331) Fix java docs for all public apis (HoodieWriteClient)

[ https://issues.apache.org/jira/browse/HUDI-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994059#comment-16994059 ] leesf commented on HUDI-331: Hi [~shivnarayan], do you have interest working on it? > Fix java docs for all

[jira] [Commented] (HUDI-343) Create a DOAP File for Hudi

[ https://issues.apache.org/jira/browse/HUDI-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994056#comment-16994056 ] leesf commented on HUDI-343: [~vinoth] I can help to work on it together with you. > Create a DOAP File for

[jira] [Commented] (HUDI-248) CLI doesn't allow rolling back a Delta commit

[ https://issues.apache.org/jira/browse/HUDI-248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994055#comment-16994055 ] leesf commented on HUDI-248: hi [~rbhartia], do you have interest in working it? > CLI doesn't allow rolling

[jira] [Commented] (HUDI-304) Bring back spotless plugin

[ https://issues.apache.org/jira/browse/HUDI-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994051#comment-16994051 ] leesf commented on HUDI-304: [~vbalaji] Sorry for lately response maybe for the missing of notification. Will

[jira] [Assigned] (HUDI-304) Bring back spotless plugin

[ https://issues.apache.org/jira/browse/HUDI-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf reassigned HUDI-304: -- Assignee: leesf > Bring back spotless plugin > --- > > Key: HUDI-304 >

[jira] [Commented] (HUDI-376) AWS Glue dependency issue for EMR 5.28.0

[ https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994048#comment-16994048 ] leesf commented on HUDI-376: [~XingXPan] Would you please send a PR to fix it? > AWS Glue dependency issue for

[GitHub] [incubator-hudi] lamber-ken commented on issue #827: java.lang.ClassNotFoundException: com.uber.hoodie.hadoop.HoodieInputFormat

lamber-ken commented on issue #827: java.lang.ClassNotFoundException: com.uber.hoodie.hadoop.HoodieInputFormat URL: https://github.com/apache/incubator-hudi/issues/827#issuecomment-564806845 > @n3nash > > I didn't know why `spark2-submit --jars basePath/hoodie-hadoop-mr-0.4.7.jar`

[GitHub] [incubator-hudi] hddong commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher

hddong commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#discussion_r356914212 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SetSparkEnvCommand.java ## @@ -0,0

[jira] [Created] (HUDI-400) Add more checks to TestCompactionUtils#testUpgradeDowngrade

leesf created HUDI-400: -- Summary: Add more checks to TestCompactionUtils#testUpgradeDowngrade Key: HUDI-400 URL: https://issues.apache.org/jira/browse/HUDI-400 Project: Apache Hudi (incubating) Issue

[jira] [Commented] (HUDI-322) DeltaSteamer should pick checkpoints off only deltacommits for MOR tables

[ https://issues.apache.org/jira/browse/HUDI-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993922#comment-16993922 ] leesf commented on HUDI-322: You are welcome.:) just do it. > DeltaSteamer should pick checkpoints off only

[GitHub] [incubator-hudi] leesf commented on issue #1093: [MINOR] replace scala map add operator

leesf commented on issue #1093: [MINOR] replace scala map add operator URL: https://github.com/apache/incubator-hudi/pull/1093#issuecomment-564749151 @vinothchandar WDYT? This is an automated message from the Apache Git

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher

leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#discussion_r356851835 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SetSparkEnvCommand.java ## @@ -0,0

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher

leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#discussion_r356851835 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SetSparkEnvCommand.java ## @@ -0,0

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher

leesf commented on a change in pull request #1096: [HUDI-398]Add set env for spark launcher URL: https://github.com/apache/incubator-hudi/pull/1096#discussion_r356850671 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SetSparkEnvCommand.java ## @@ -0,0

[incubator-hudi] branch master updated: [MINOR] Remove redundant plus operator (#1097)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new ba514cf [MINOR] Remove redundant plus

[GitHub] [incubator-hudi] leesf merged pull request #1097: [MINOR] Remove redundant plus operator

leesf merged pull request #1097: [MINOR] Remove redundant plus operator URL: https://github.com/apache/incubator-hudi/pull/1097 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-hudi] n3nash commented on issue #1057: Hudi Test Suite

n3nash commented on issue #1057: Hudi Test Suite URL: https://github.com/apache/incubator-hudi/pull/1057#issuecomment-564716209 @yanghua sounds good. This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect

lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-564689247 @imperio-wxm, you need to set the value of `DataSourceWriteOptions.HIVE_ASSUME_DATE_PARTITION_OPT_KEY()` to `false`.

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect

lamber-ken edited a comment on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-564689247 @imperio-wxm, you need to set the value of `DataSourceWriteOptions.HIVE_ASSUME_DATE_PARTITION_OPT_KEY()` to `false`.

[GitHub] [incubator-hudi] lamber-ken commented on issue #828: Synchronizing to hive partition is incorrect

lamber-ken commented on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-564689247 @imperio-wxm, you need to set the value of `DataSourceWriteOptions.HIVE_ASSUME_DATE_PARTITION_OPT_KEY()` to `false`. ###

[jira] [Comment Edited] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993751#comment-16993751 ] Purushotham Pushpavanthar edited comment on HUDI-251 at 12/11/19 5:44 PM:

[jira] [Commented] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993751#comment-16993751 ] Purushotham Pushpavanthar commented on HUDI-251: Hi [~taherk77], are you still working on

[jira] [Commented] (HUDI-334) Clean up terminologies in code/docs

[ https://issues.apache.org/jira/browse/HUDI-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993627#comment-16993627 ] Vinoth Chandar commented on HUDI-334: - This one is related to the thread discussion above. It could be

[jira] [Commented] (HUDI-322) DeltaSteamer should pick checkpoints off only deltacommits for MOR tables

[ https://issues.apache.org/jira/browse/HUDI-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993625#comment-16993625 ] Vinoth Chandar commented on HUDI-322: - You have it..  :) Please let us know if you have questions along

[jira] [Updated] (HUDI-322) DeltaSteamer should pick checkpoints off only deltacommits for MOR tables

[ https://issues.apache.org/jira/browse/HUDI-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-322: Description: When using DeltaStreamer with MOR, the checkpoints would be written out to .deltacommit

[jira] [Assigned] (HUDI-322) DeltaSteamer should pick checkpoints off only deltacommits for MOR tables

[ https://issues.apache.org/jira/browse/HUDI-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-322: --- Assignee: Shahida Khan > DeltaSteamer should pick checkpoints off only deltacommits for MOR

[GitHub] [incubator-hudi] vinothchandar commented on issue #969: [HUDI-251] JDBC incremental load to HUDI DeltaStreamer

vinothchandar commented on issue #969: [HUDI-251] JDBC incremental load to HUDI DeltaStreamer URL: https://github.com/apache/incubator-hudi/pull/969#issuecomment-564593845 @pushpavanthar Was out for a conference.. So picking this back up.. I am also pretty interested in this feature. I

[jira] [Assigned] (HUDI-399) Fix conflicts of test suite basic implementation based on master branch

[ https://issues.apache.org/jira/browse/HUDI-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-399: - Assignee: vinoyang > Fix conflicts of test suite basic implementation based on master branch >

[jira] [Commented] (HUDI-399) Fix conflicts of test suite basic implementation based on master branch

[ https://issues.apache.org/jira/browse/HUDI-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993474#comment-16993474 ] vinoyang commented on HUDI-399: --- [~vinoth] [~nishith29] FYI. Will try to fix these conflicts. > Fix

[jira] [Created] (HUDI-399) Fix conflicts of test suite basic implementation based on master branch

vinoyang created HUDI-399: - Summary: Fix conflicts of test suite basic implementation based on master branch Key: HUDI-399 URL: https://issues.apache.org/jira/browse/HUDI-399 Project: Apache Hudi

[jira] [Resolved] (HUDI-394) Provide a basic implementation of test suite

[ https://issues.apache.org/jira/browse/HUDI-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang resolved HUDI-394. --- Resolution: Implemented Implemented via hudi_test_suite_refactor branch: 

[GitHub] [incubator-hudi] XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter

XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095#issuecomment-564471341 > hi, Thanks for opening the PR @XuQianJin-Stars, this test case is useless. Because PushGateway use

[GitHub] [incubator-hudi] XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter

XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095#issuecomment-564460334 hi @lamber-ken Thank you very much, Let me change it.

[GitHub] [incubator-hudi] lamber-ken opened a new pull request #1097: [MINOR] Remove redundant plus operator

lamber-ken opened a new pull request #1097: [MINOR] Remove redundant plus operator URL: https://github.com/apache/incubator-hudi/pull/1097 ## What is the purpose of the pull request Remove redundant plus operator ## Brief change log - Remove redundant plus operator