[jira] [Updated] (HUDI-335) Improvements to DiskBasedMap

[ https://issues.apache.org/jira/browse/HUDI-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-335: --- Fix Version/s: (was: 0.5.2) 0.5.1 > Improvements to DiskBasedMap >

[jira] [Resolved] (HUDI-526) inline compact not work

[ https://issues.apache.org/jira/browse/HUDI-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-526. Fix Version/s: 0.5.1 Resolution: Fixed Fixed via master: c1f8acab344fa632f1cce6268d2fc765c45e8b22 > inline

[GitHub] [incubator-hudi] yanghua commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter

yanghua commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095#issuecomment-575052312 @XuQianJin-Stars What's the status of this PR? This is an

[jira] [Created] (HUDI-539) No FileSystem for scheme: abfss

Sam Somuah created HUDI-539: --- Summary: No FileSystem for scheme: abfss Key: HUDI-539 URL: https://issues.apache.org/jira/browse/HUDI-539 Project: Apache Hudi (incubating) Issue Type: Bug

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367492682 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM

xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM URL: https://github.com/apache/incubator-hudi/pull/1187#discussion_r367508976 ## File path: hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1229: [HUDI-535] Ensure Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1 reader-writer compatibility

vinothchandar commented on a change in pull request #1229: [HUDI-535] Ensure Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1 reader-writer compatibility issues URL: https://github.com/apache/incubator-hudi/pull/1229#discussion_r367471314 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM

xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM URL: https://github.com/apache/incubator-hudi/pull/1187#discussion_r367490392 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM

xushiyan commented on a change in pull request #1187: [HUDI-499] Allow update partition path with GLOBAL_BLOOM URL: https://github.com/apache/incubator-hudi/pull/1187#discussion_r367508270 ## File path: hudi-client/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java

[GitHub] [incubator-hudi] ssomuah commented on issue #1228: No FileSystem for scheme: abfss

ssomuah commented on issue #1228: No FileSystem for scheme: abfss URL: https://github.com/apache/incubator-hudi/issues/1228#issuecomment-575163833 The scheme I'm trying to use is supported (abfss). The problem is a blank Hadoop Configuration is passed in in `HoodieROTablePathFilter` so it

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

vinothchandar commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367472578 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367491239 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

vinothchandar commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#issuecomment-575197796 @n3nash Addressed comments.. Responded to the renaming issue. Let me know what you think and we can hopefully

[GitHub] [incubator-hudi] lamber-ken opened a new pull request #1236: [MINOR] Fix missing @Override annotation on BufferedRandomAccessFilemethod

lamber-ken opened a new pull request #1236: [MINOR] Fix missing @Override annotation on BufferedRandomAccessFilemethod URL: https://github.com/apache/incubator-hudi/pull/1236 ## What is the purpose of the pull request Fix missing @Override annotation on

[GitHub] [incubator-hudi] zhedoubushishi commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12

zhedoubushishi commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12 URL: https://github.com/apache/incubator-hudi/pull/1226#issuecomment-575296997 > @leesf : FYI : This diff will have implication on how we release packages. We will be changing the names of packages :

[incubator-hudi] branch master updated (b39458b -> 8a3a503)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from b39458b [MINOR] Make constant fields final in HoodieTestDataGenerator (#1234) add 8a3a503 [MINOR]

[GitHub] [incubator-hudi] vinothchandar merged pull request #1236: [MINOR] Fix missing @Override annotation on BufferedRandomAccessFilemethod

vinothchandar merged pull request #1236: [MINOR] Fix missing @Override annotation on BufferedRandomAccessFilemethod URL: https://github.com/apache/incubator-hudi/pull/1236 This is an automated message from the Apache Git

[jira] [Created] (HUDI-540) Incorrect archive directory path in show archived commits cli

Venkatesh created HUDI-540: -- Summary: Incorrect archive directory path in show archived commits cli Key: HUDI-540 URL: https://issues.apache.org/jira/browse/HUDI-540 Project: Apache Hudi (incubating)

[jira] [Updated] (HUDI-528) Incremental Pull fails when latest commit is empty

[ https://issues.apache.org/jira/browse/HUDI-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Javier Vega updated HUDI-528: - Description: When trying to create an incremental view of a dataset, an exception is thrown when the

[GitHub] [incubator-hudi] pratyakshsharma commented on issue #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on issue #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#issuecomment-575461848 @bvaradar Have asked for clarification on 2-3 comments. Otherwise give

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017739#comment-17017739 ] Vinoth Chandar commented on HUDI-538: - I think the first step here is to pull spark facing stuff into a

[jira] [Assigned] (HUDI-510) Update site documentation in sync with cWiki

[ https://issues.apache.org/jira/browse/HUDI-510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha reassigned HUDI-510: -- Assignee: Bhavani Sudha (was: Vinoth Chandar) > Update site documentation in sync with cWiki >

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017674#comment-17017674 ] Zijie Lu commented on HUDI-538: --- [~yanghua],I am also interested in this issue and want to contribute in

[jira] [Updated] (HUDI-536) Update release notes to include KeyGenerator package changes

[ https://issues.apache.org/jira/browse/HUDI-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha updated HUDI-536: --- Status: Open (was: New) > Update release notes to include KeyGenerator package changes >

[jira] [Updated] (HUDI-29) Patch to Hive-sync to enable stats on Hive tables #393

[ https://issues.apache.org/jira/browse/HUDI-29?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha updated HUDI-29: -- Fix Version/s: (was: 0.5.1) 0.6.0 > Patch to Hive-sync to enable stats on Hive

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017682#comment-17017682 ] vinoyang commented on HUDI-538: --- [~alfredlu] (y) > Restructuring hudi client module for multi engine support

[GitHub] [incubator-hudi] bvaradar commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12

bvaradar commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12 URL: https://github.com/apache/incubator-hudi/pull/1226#issuecomment-575469290 > Also there is another way to do this: I can rename artifact Id, e.g.`hudi-spark` to `hudi-spark_${scala.binary.version}`, and thus

[GitHub] [incubator-hudi] melin opened a new issue #1240: future support for multi-client concurrent write?

melin opened a new issue #1240: future support for multi-client concurrent write? URL: https://github.com/apache/incubator-hudi/issues/1240 future support for multi-client concurrent write? This is an automated message from

[GitHub] [incubator-hudi] zhedoubushishi commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12

zhedoubushishi commented on issue #1226: [HUDI-238] Make Hudi support Scala 2.12 URL: https://github.com/apache/incubator-hudi/pull/1226#issuecomment-575513618 > > Also there is another way to do this: I can rename artifact Id, e.g.`hudi-spark` to `hudi-spark_${scala.binary.version}`, and

[GitHub] [incubator-hudi] yanghua commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes

yanghua commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes URL: https://github.com/apache/incubator-hudi/pull/1237#issuecomment-575513806 @vinothchandar OK, will review this PR. This is

[jira] [Assigned] (HUDI-541) Replace variables/comments named "data files" to "base file"

[ https://issues.apache.org/jira/browse/HUDI-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha reassigned HUDI-541: -- Assignee: Bhavani Sudha > Replace variables/comments named "data files" to "base file" >

[GitHub] [incubator-hudi] vinothchandar commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

vinothchandar commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#issuecomment-575512172 @bhasudha good catch. I cleaned up what you pointed out.. but there are still tons of variables named `dataFile`

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #161

See Changes: -- [...truncated 2.21 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/bin: m2.conf mvn mvn.cmd mvnDebug mvnDebug.cmd mvnyjp

[GitHub] [incubator-hudi] vinothchandar commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes

vinothchandar commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes URL: https://github.com/apache/incubator-hudi/pull/1237#issuecomment-575491467 @yanghua Are you able to do a quick review of this? I am dealing with some other release blockers..

[GitHub] [incubator-hudi] vinothchandar commented on issue #1229: [HUDI-535] Ensure Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1 reader-writer compatibility issues

vinothchandar commented on issue #1229: [HUDI-535] Ensure Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1 reader-writer compatibility issues URL: https://github.com/apache/incubator-hudi/pull/1229#issuecomment-575491975 @bvaradar any updates?

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017750#comment-17017750 ] Zijie Lu commented on HUDI-538: --- [~vinoth] And any plan for integrating hudi with flink? > Restructuring

[GitHub] [incubator-hudi] yihua opened a new pull request #1239: [MINOR] Abstract a test case class for DFS Source to make it extensible

yihua opened a new pull request #1239: [MINOR] Abstract a test case class for DFS Source to make it extensible URL: https://github.com/apache/incubator-hudi/pull/1239 ## What is the purpose of the pull request `TestDFSSource` class contains redundant code of the test logic for

[jira] [Updated] (HUDI-541) Replace variables/comments named "data files" to "base file"

[ https://issues.apache.org/jira/browse/HUDI-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-541: Issue Type: Improvement (was: Bug) > Replace variables/comments named "data files" to "base file" >

[jira] [Created] (HUDI-541) Replace variables/comments named "data files" to "base file"

Vinoth Chandar created HUDI-541: --- Summary: Replace variables/comments named "data files" to "base file" Key: HUDI-541 URL: https://issues.apache.org/jira/browse/HUDI-541 Project: Apache Hudi

[GitHub] [incubator-hudi] vinothchandar merged pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

vinothchandar merged pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212 This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-hudi] bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367679877 ## File path:

[GitHub] [incubator-hudi] hmatu commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes

hmatu commented on issue #1237: [MINOR] Code Cleanup, remove redundant code, and other changes URL: https://github.com/apache/incubator-hudi/pull/1237#issuecomment-575414268 Big thanks for your work on this, but it's hard to review like https://github.com/apache/incubator-hudi/pull/1159.

[jira] [Commented] (HUDI-540) Incorrect archive directory path in show archived commits cli

[ https://issues.apache.org/jira/browse/HUDI-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017639#comment-17017639 ] hong dongdong commented on HUDI-540: [~venkee14] It‘s need  {code:java} @CliOption(key =

[jira] [Assigned] (HUDI-540) Incorrect archive directory path in show archived commits cli

[ https://issues.apache.org/jira/browse/HUDI-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hong dongdong reassigned HUDI-540: -- Assignee: hong dongdong > Incorrect archive directory path in show archived commits cli >

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367687172 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367687094 ## File path:

[GitHub] [incubator-hudi] bhasudha commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

bhasudha commented on issue #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#issuecomment-575380133 LGTM overall. Left minor comments. Was just thinking about them. Feel free to ignore if its too much refactoring.

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

n3nash commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367687060 ## File path:

[GitHub] [incubator-hudi] melin opened a new issue #1238: planned hudi-client not dependent on spark api?

melin opened a new issue #1238: planned hudi-client not dependent on spark api? URL: https://github.com/apache/incubator-hudi/issues/1238 Hudi-client relies heavily on spark api and can be replaced? This is an automated

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017648#comment-17017648 ] hong dongdong commented on HUDI-538: [~yanghua] I am interesting in this work, please @ me if any need.

[jira] [Commented] (HUDI-539) No FileSystem for scheme: abfss

[ https://issues.apache.org/jira/browse/HUDI-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017644#comment-17017644 ] hong dongdong commented on HUDI-539: [~ssomuah] As far as I know, ABFS was support after hadoop-3.2.0,

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017670#comment-17017670 ] vinoyang commented on HUDI-538: --- [~hongdongdong] Glad to hear this news. Welcome! > Restructuring hudi

[GitHub] [incubator-hudi] bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367678245 ## File path:

[GitHub] [incubator-hudi] bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring

bhasudha commented on a change in pull request #1212: [HUDI-509] Renaming code in sync with cWiki restructuring URL: https://github.com/apache/incubator-hudi/pull/1212#discussion_r367678068 ## File path:

[GitHub] [incubator-hudi] n3nash commented on issue #1115: [HUDI-392] Introduce DIstributedTestDataSource to generate test data

n3nash commented on issue #1115: [HUDI-392] Introduce DIstributedTestDataSource to generate test data URL: https://github.com/apache/incubator-hudi/pull/1115#issuecomment-575381731 @yanghua Are we close to calling this as a first version of the test suite ?

[GitHub] [incubator-hudi] smarthi opened a new pull request #1237: [MINOR] Code Cleanup, remove redundant code, and other changes

smarthi opened a new pull request #1237: [MINOR] Code Cleanup, remove redundant code, and other changes URL: https://github.com/apache/incubator-hudi/pull/1237 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367363989 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367366905 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367371389 ## File path:

[GitHub] [incubator-hudi] XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter

XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095#issuecomment-575115811 hi @leesf @lamber-ken Hive any time to continue review this PR?

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r367363989 ## File path: