[GitHub] [incubator-hudi] garyli1019 commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
garyli1019 commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#discussion_r359196312 ## File path: hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@

[GitHub] [incubator-hudi] zhedoubushishi commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12

2019-12-17 Thread GitBox
zhedoubushishi commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12 URL: https://github.com/apache/incubator-hudi/pull/1109#issuecomment-566914616 > hi @zhedoubushishi , > > 1. I didn't know that I can import it through maven, would have saved me quite some time... Will do

[GitHub] [incubator-hudi] ezhux commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12

2019-12-17 Thread GitBox
ezhux commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12 URL: https://github.com/apache/incubator-hudi/pull/1109#issuecomment-566913999 hi @zhedoubushishi , 1. I didn't know that I can import it through maven, would have saved me quite some time... Will do this 2. thanks

[jira] [Commented] (HUDI-425) Implement support for bootstrapping in HoodieDeltaStreamer

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998879#comment-16998879 ] Balaji Varadarajan commented on HUDI-425: - [~nicholasjiang]: I haveĀ  a basic implementation along

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#discussion_r359189109 ## File path: hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359174660 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/HoodieCLI.java ## @@ -40,7 +40,7 @@

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359186653 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/DatasetsCommand.java ## @@

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359175830 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ArchivedCommitsCommand.java

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359188519 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SavepointsCommand.java ##

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359178974 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/CompactionCommand.java ##

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359183312 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/HoodieSyncCommand.java ##

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1110: [HUDI-416] improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#discussion_r359185857 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/CommitsCommand.java ## @@

[jira] [Commented] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998868#comment-16998868 ] Balaji Varadarajan commented on HUDI-417: - [~nicholasjiang]: I may not get a chance to work on it

[GitHub] [incubator-hudi] garyli1019 commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
garyli1019 commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#discussion_r359180192 ## File path: hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@

[GitHub] [incubator-hudi] nsivabalan commented on issue #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
nsivabalan commented on issue #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#issuecomment-566891974 Looks good. Will wait for vinoth to comment. This is an automated

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#discussion_r359173720 ## File path: hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@

[jira] [Commented] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

2019-12-17 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998847#comment-16998847 ] Nicholas Jiang commented on HUDI-417: - [~vbalaji]Do you have free time to move above commit

[jira] [Commented] (HUDI-425) Implement support for bootstrapping in HoodieDeltaStreamer

2019-12-17 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998843#comment-16998843 ] Nicholas Jiang commented on HUDI-425: - [~vbalaji]I have interest in the implement for bootstrapping in

[GitHub] [incubator-hudi] vinothchandar closed pull request #626: Adding documentation for hudi test suite

2019-12-17 Thread GitBox
vinothchandar closed pull request #626: Adding documentation for hudi test suite URL: https://github.com/apache/incubator-hudi/pull/626 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-hudi] vinothchandar commented on issue #1110: [HUDI-416]improve hint information for cli

2019-12-17 Thread GitBox
vinothchandar commented on issue #1110: [HUDI-416]improve hint information for cli URL: https://github.com/apache/incubator-hudi/pull/1110#issuecomment-566871837 @nsivabalan Please review this when you get a chance This is

[GitHub] [incubator-hudi] vinothchandar commented on issue #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
vinothchandar commented on issue #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113#issuecomment-566871335 @nsivabalan Please review this when you get a chance This is

[jira] [Assigned] (HUDI-127) [Good to do] Tidy up cWiki

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-127: --- Assignee: Vinoth Chandar > [Good to do] Tidy up cWiki > -- > >

[jira] [Updated] (HUDI-332) Add operation type (insert/upsert/bulkinsert/delete) to HoodieCommitMetadata

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-332: Fix Version/s: (was: 0.5.1) > Add operation type (insert/upsert/bulkinsert/delete) to

[GitHub] [incubator-hudi] lamber-ken commented on issue #1105: [HUDI-405] Fix sync no hive partition at first time

2019-12-17 Thread GitBox
lamber-ken commented on issue #1105: [HUDI-405] Fix sync no hive partition at first time URL: https://github.com/apache/incubator-hudi/pull/1105#issuecomment-566858402 hi, @vinothchandar. We also can change `assumeDatePartitioning` to `!assumeDatePartitioning` at

[GitHub] [incubator-hudi] lamber-ken commented on issue #828: Synchronizing to hive partition is incorrect

2019-12-17 Thread GitBox
lamber-ken commented on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-566858078 Right, we also can change `assumeDatePartitioning` to `!assumeDatePartitioning` at `FSUtils#getAllPartitionPaths` to fix this

[GitHub] [incubator-hudi] vinothchandar commented on issue #828: Synchronizing to hive partition is incorrect

2019-12-17 Thread GitBox
vinothchandar commented on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-566855042 @lamber-ken get the full context now.. `DEFAULT_HIVE_ASSUME_DATE_PARTITION_OPT_VAL` is `false` in fact.

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #132

2019-12-17 Thread Apache Jenkins Server
See Changes: -- [...truncated 2.18 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/bin: m2.conf mvn mvn.cmd mvnDebug mvnDebug.cmd mvnyjp

[jira] [Commented] (HUDI-259) Hadoop 3 support for Hudi writing

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998780#comment-16998780 ] Vinoth Chandar commented on HUDI-259: - Can we do hadoop 3 i.e make the project compile and run with

[jira] [Commented] (HUDI-233) Redo log statements using SLF4J

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998779#comment-16998779 ] Vinoth Chandar commented on HUDI-233: - We did this before and it had its own issues... I prefer a low

[jira] [Updated] (HUDI-233) Redo log statements using SLF4J

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-233: Fix Version/s: 0.5.2 > Redo log statements using SLF4J > > >

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1105: [HUDI-405] Fix sync no hive partition at first time

2019-12-17 Thread GitBox
vinothchandar commented on a change in pull request #1105: [HUDI-405] Fix sync no hive partition at first time URL: https://github.com/apache/incubator-hudi/pull/1105#discussion_r359135764 ## File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java ##

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1105: [HUDI-405] Fix sync no hive partition at first time

2019-12-17 Thread GitBox
vinothchandar commented on a change in pull request #1105: [HUDI-405] Fix sync no hive partition at first time URL: https://github.com/apache/incubator-hudi/pull/1105#discussion_r359136457 ## File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java ##

[GitHub] [incubator-hudi] vinothchandar commented on issue #1105: [HUDI-405] Fix sync no hive partition at first time

2019-12-17 Thread GitBox
vinothchandar commented on issue #1105: [HUDI-405] Fix sync no hive partition at first time URL: https://github.com/apache/incubator-hudi/pull/1105#issuecomment-566848062 ` If HIVE_ASSUME_DATE_PARTITION_OPT_KEY is set true` is pretty much a historical config, that could solely affect old

[jira] [Updated] (HUDI-106) Dynamically tune bloom filter entries

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-106: Status: Closed (was: Patch Available) > Dynamically tune bloom filter entries >

[jira] [Updated] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-251: Status: Closed (was: Patch Available) > JDBC incremental load to HUDI with DeltaStreamer >

[jira] [Reopened] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reopened HUDI-251: - > JDBC incremental load to HUDI with DeltaStreamer >

[jira] [Commented] (HUDI-251) JDBC incremental load to HUDI with DeltaStreamer

2019-12-17 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998769#comment-16998769 ] Vinoth Chandar commented on HUDI-251: - hi all, the process is documented hereĀ 

[GitHub] [incubator-hudi] vinothchandar merged pull request #976: [HUDI-106] Adding support for DynamicBloomFilter

2019-12-17 Thread GitBox
vinothchandar merged pull request #976: [HUDI-106] Adding support for DynamicBloomFilter URL: https://github.com/apache/incubator-hudi/pull/976 This is an automated message from the Apache Git Service. To respond to the

[incubator-hudi] branch master updated (7498ca7 -> 14881e9)

2019-12-17 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from 7498ca7 [MINOR] Add slack invite icon in README (#1108) add 14881e9 [HUDI-106] Adding support for

[GitHub] [incubator-hudi] hddong commented on issue #1111: [HUDI-331]Fix java docs for all public apis in HoodieWriteClient

2019-12-17 Thread GitBox
hddong commented on issue #: [HUDI-331]Fix java docs for all public apis in HoodieWriteClient URL: https://github.com/apache/incubator-hudi/pull/#issuecomment-566837130 @leesf all be fixed This is an automated

[jira] [Created] (HUDI-437) Support user-defined index

2019-12-17 Thread leesf (Jira)
leesf created HUDI-437: -- Summary: Support user-defined index Key: HUDI-437 URL: https://issues.apache.org/jira/browse/HUDI-437 Project: Apache Hudi (incubating) Issue Type: Improvement

[GitHub] [incubator-hudi] leesf commented on issue #689: [HUDI-25] Optimize HoodieInputFormat.listStatus for faster Hive Incremental queries

2019-12-17 Thread GitBox
leesf commented on issue #689: [HUDI-25] Optimize HoodieInputFormat.listStatus for faster Hive Incremental queries URL: https://github.com/apache/incubator-hudi/pull/689#issuecomment-566799788 @bhasudha Could you please rebase to master and merge it as it is ready?

[jira] [Created] (HUDI-436) Integrate HFile and Compaction

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-436: Summary: Integrate HFile and Compaction Key: HUDI-436 URL: https://issues.apache.org/jira/browse/HUDI-436 Project: Apache Hudi (incubating) Issue

[jira] [Created] (HUDI-435) Make async compaction extensible to be available in other components.

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-435: Summary: Make async compaction extensible to be available in other components. Key: HUDI-435 URL: https://issues.apache.org/jira/browse/HUDI-435 Project:

[jira] [Created] (HUDI-434) Design and develop HFile based Index using InlineFS

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-434: Summary: Design and develop HFile based Index using InlineFS Key: HUDI-434 URL: https://issues.apache.org/jira/browse/HUDI-434 Project: Apache Hudi

[jira] [Updated] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2019-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-415: Labels: pull-request-available (was: ) > HoodieSparkSqlWriter Commit time not representing the

[GitHub] [incubator-hudi] garyli1019 opened a new pull request #1113: [HUDI-415] Get commit time when Spark start

2019-12-17 Thread GitBox
garyli1019 opened a new pull request #1113: [HUDI-415] Get commit time when Spark start URL: https://github.com/apache/incubator-hudi/pull/1113 ## Why do I create this PR The current `HoodieSparkSqlWriter` record the `commitTime` after checking the input `hoodieRecords` is not

[jira] [Created] (HUDI-433) Fix hoodie hash magic for Log block separators

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-433: Summary: Fix hoodie hash magic for Log block separators Key: HUDI-433 URL: https://issues.apache.org/jira/browse/HUDI-433 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-432) Benchmark Inline Parquet Logging

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-432: Summary: Benchmark Inline Parquet Logging Key: HUDI-432 URL: https://issues.apache.org/jira/browse/HUDI-432 Project: Apache Hudi (incubating) Issue

[jira] [Created] (HUDI-431) Design and develop parquet logging in Log file

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-431: Summary: Design and develop parquet logging in Log file Key: HUDI-431 URL: https://issues.apache.org/jira/browse/HUDI-431 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-430) Design Inline FileSystem which supports embedding any file format (parquet/avro/etc)

2019-12-17 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-430: Summary: Design Inline FileSystem which supports embedding any file format (parquet/avro/etc) Key: HUDI-430 URL: https://issues.apache.org/jira/browse/HUDI-430

[GitHub] [incubator-hudi] jwo-salesforce commented on issue #143: Tracking ticket for folks to be added to slack group

2019-12-17 Thread GitBox
jwo-salesforce commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-56670 please add j...@salesforce.com thanks! This is an

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358964106 ## File path:

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358963953 ## File path:

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358964106 ## File path:

[jira] [Commented] (HUDI-259) Hadoop 3 support for Hudi writing

2019-12-17 Thread Wenning Ding (Jira)
[ https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998480#comment-16998480 ] Wenning Ding commented on HUDI-259: --- Hey [~Pratyaksh], I am also working on hadoop 3 support for Hudi.

[GitHub] [incubator-hudi] XuQianJin-Stars opened a new pull request #1095: [HUDI-210] Implement prometheus metrics reporter

2019-12-17 Thread GitBox
XuQianJin-Stars opened a new pull request #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095 ## What is the purpose of the pull request Since Prometheus is a very popular monitoring system and time series database, it

[GitHub] [incubator-hudi] XuQianJin-Stars closed pull request #1095: [HUDI-210] Implement prometheus metrics reporter

2019-12-17 Thread GitBox
XuQianJin-Stars closed pull request #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-hudi] XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter

2019-12-17 Thread GitBox
XuQianJin-Stars commented on issue #1095: [HUDI-210] Implement prometheus metrics reporter URL: https://github.com/apache/incubator-hudi/pull/1095#issuecomment-566574817 > > @lamber-ken @XuQianJin-Stars. Is it ready? > > Not yet.

[jira] [Updated] (HUDI-419) Basic Implementation for verifying if bootstrapping works end to end

2019-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-419: Labels: pull-request-available (was: ) > Basic Implementation for verifying if bootstrapping works

[GitHub] [incubator-hudi] bvaradar opened a new pull request #1112: [WIP] [HUDI-419] Basic Implementation of bootstrapping

2019-12-17 Thread GitBox
bvaradar opened a new pull request #1112: [WIP] [HUDI-419] Basic Implementation of bootstrapping URL: https://github.com/apache/incubator-hudi/pull/1112 ## What is the purpose of the pull request This is a temporary PR. It contains basic changes needed for

[jira] [Created] (HUDI-429) Long Running Testing to certify Bootstrapping

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-429: --- Summary: Long Running Testing to certify Bootstrapping Key: HUDI-429 URL: https://issues.apache.org/jira/browse/HUDI-429 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-428) Web documentation for explaining how to bootstrap

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-428: --- Summary: Web documentation for explaining how to bootstrap Key: HUDI-428 URL: https://issues.apache.org/jira/browse/HUDI-428 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-427) Implement CLI support for performing bootstrap

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-427: --- Summary: Implement CLI support for performing bootstrap Key: HUDI-427 URL: https://issues.apache.org/jira/browse/HUDI-427 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-426) Implement Spark DataSource Support for querying bootstrapped tables

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-426: --- Summary: Implement Spark DataSource Support for querying bootstrapped tables Key: HUDI-426 URL: https://issues.apache.org/jira/browse/HUDI-426 Project: Apache

[jira] [Created] (HUDI-425) Implement support for bootstrapping in HoodieDeltaStreamer

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-425: --- Summary: Implement support for bootstrapping in HoodieDeltaStreamer Key: HUDI-425 URL: https://issues.apache.org/jira/browse/HUDI-425 Project: Apache Hudi

[jira] [Assigned] (HUDI-425) Implement support for bootstrapping in HoodieDeltaStreamer

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-425: --- Assignee: Balaji Varadarajan > Implement support for bootstrapping in

[jira] [Assigned] (HUDI-424) Implement Query Side Integration for querying tables containing bootstrap file slices

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-424: --- Assignee: Balaji Varadarajan > Implement Query Side Integration for querying tables

[jira] [Created] (HUDI-423) Implement upsert functionality for handling updates to these bootstrap file slices

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-423: --- Summary: Implement upsert functionality for handling updates to these bootstrap file slices Key: HUDI-423 URL: https://issues.apache.org/jira/browse/HUDI-423

[jira] [Assigned] (HUDI-423) Implement upsert functionality for handling updates to these bootstrap file slices

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-423: --- Assignee: Balaji Varadarajan > Implement upsert functionality for handling updates to

[jira] [Assigned] (HUDI-422) Cleanup bootstrap code and create HoodieBootstrapClient for supporting bootstrap

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-422: --- Assignee: Balaji Varadarajan > Cleanup bootstrap code and create

[jira] [Created] (HUDI-422) Cleanup bootstrap code and create HoodieBootstrapClient for supporting bootstrap

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-422: --- Summary: Cleanup bootstrap code and create HoodieBootstrapClient for supporting bootstrap Key: HUDI-422 URL: https://issues.apache.org/jira/browse/HUDI-422

[jira] [Assigned] (HUDI-421) Cleanup bootstrap code and create PR for FileStystemView changes

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-421: --- Assignee: Balaji Varadarajan > Cleanup bootstrap code and create PR for

[jira] [Created] (HUDI-421) Cleanup bootstrap code and create PR for FileStystemView changes

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-421: --- Summary: Cleanup bootstrap code and create PR for FileStystemView changes Key: HUDI-421 URL: https://issues.apache.org/jira/browse/HUDI-421 Project: Apache

[jira] [Created] (HUDI-420) Automated end to end Integration Test

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-420: --- Summary: Automated end to end Integration Test Key: HUDI-420 URL: https://issues.apache.org/jira/browse/HUDI-420 Project: Apache Hudi (incubating)

[jira] [Commented] (HUDI-259) Hadoop 3 support for Hudi writing

2019-12-17 Thread Pratyaksh Sharma (Jira)
[ https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998238#comment-16998238 ] Pratyaksh Sharma commented on HUDI-259: --- Yes, this way you can build your jars for deployment

[jira] [Assigned] (HUDI-419) Basic Implementation for verifying if bootstrapping works end to end

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-419: --- Assignee: Balaji Varadarajan > Basic Implementation for verifying if bootstrapping

[jira] [Created] (HUDI-419) Basic Implementation for verifying if bootstrapping works end to end

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-419: --- Summary: Basic Implementation for verifying if bootstrapping works end to end Key: HUDI-419 URL: https://issues.apache.org/jira/browse/HUDI-419 Project: Apache

[jira] [Created] (HUDI-418) Bootstrap Index - Implementation

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-418: --- Summary: Bootstrap Index - Implementation Key: HUDI-418 URL: https://issues.apache.org/jira/browse/HUDI-418 Project: Apache Hudi (incubating) Issue

[jira] [Assigned] (HUDI-418) Bootstrap Index - Implementation

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-418: --- Assignee: Balaji Varadarajan > Bootstrap Index - Implementation >

[jira] [Assigned] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-417: --- Assignee: (was: Balaji Varadarajan) > Refactor HoodieWriteClient so that commit

[jira] [Created] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

2019-12-17 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-417: --- Summary: Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations Key: HUDI-417 URL:

[jira] [Assigned] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

2019-12-17 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-417: --- Assignee: Balaji Varadarajan > Refactor HoodieWriteClient so that commit logic can be

[jira] [Updated] (HUDI-331) Fix java docs for all public apis (HoodieWriteClient)

2019-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-331: Labels: newbie pull-request-available (was: newbie) > Fix java docs for all public apis

[GitHub] [incubator-hudi] hddong opened a new pull request #1111: [HUDI-331]Fix java docs for all public apis in HoodieWriteClient

2019-12-17 Thread GitBox
hddong opened a new pull request #: [HUDI-331]Fix java docs for all public apis in HoodieWriteClient URL: https://github.com/apache/incubator-hudi/pull/ ## What is the purpose of the pull request *Some public apis in HoodieWriteClient need to be fixed with sufficient info.

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358661320 ## File path:

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358654092 ## File path:

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358645468 ## File path:

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom

2019-12-17 Thread GitBox
leesf commented on a change in pull request #1091: [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358645528 ## File path: