[GitHub] [incubator-hudi] n3nash commented on issue #624: (WIP) Test suite documentation

2019-04-01 Thread GitBox
n3nash commented on issue #624: (WIP) Test suite documentation URL: https://github.com/apache/incubator-hudi/pull/624#issuecomment-478822267 Closing this PR since it was opened against master branch This is an automated

[GitHub] [incubator-hudi] n3nash closed pull request #624: (WIP) Test suite documentation

2019-04-01 Thread GitBox
n3nash closed pull request #624: (WIP) Test suite documentation URL: https://github.com/apache/incubator-hudi/pull/624 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-hudi] arunlakshman commented on issue #622: [HUDI-61] Remove payload class dependency in SpillableMap

2019-04-02 Thread GitBox
arunlakshman commented on issue #622: [HUDI-61] Remove payload class dependency in SpillableMap URL: https://github.com/apache/incubator-hudi/pull/622#issuecomment-478884206 I just saw this on JIRA board and started working on this - https://issues.apache.org/jira/browse/HUDI-61 -

[GitHub] [incubator-hudi] SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

2019-04-02 Thread GitBox
SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724 Do we want to consider what this means more exactly from perspective of

[GitHub] [incubator-hudi] SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

2019-04-02 Thread GitBox
SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724 Do we want to consider what this means more exactly from perspective of

[GitHub] [incubator-hudi] n3nash opened a new pull request #626: Adding documentation for hudi test suite

2019-04-01 Thread GitBox
n3nash opened a new pull request #626: Adding documentation for hudi test suite URL: https://github.com/apache/incubator-hudi/pull/626 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-hudi] SemanticBeeng commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

2019-04-02 Thread GitBox
SemanticBeeng commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724 Do we want to consider what this means more exactly from perspective of

[GitHub] [incubator-hudi] milantracy opened a new pull request #627: [HUDI-66] FSUtils.getRelativePartitionPath does not handle repeated f…

2019-04-02 Thread GitBox
milantracy opened a new pull request #627: [HUDI-66] FSUtils.getRelativePartitionPath does not handle repeated f… URL: https://github.com/apache/incubator-hudi/pull/627 …older names This is an automated message from the

[GitHub] [incubator-hudi] vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark

2019-03-29 Thread GitBox
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-478077857 @leilinen did you have the same problem with this patch applied as well? Some background. FileSystem

[GitHub] [incubator-hudi] vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

2019-04-02 Thread GitBox
vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134137 @SemanticBeeng this is orthogonal to connection management I believe.. This

[GitHub] [incubator-hudi] lyogev commented on issue #498: Is there any record delete examples?

2019-04-02 Thread GitBox
lyogev commented on issue #498: Is there any record delete examples? URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479127330 Does anyone have an example of how to hard delete in spark SQL? I can't seem to find a way to do this.

[GitHub] [incubator-hudi] vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

2019-04-02 Thread GitBox
vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134748 Once we approach beam, we can consider ProcessContext. for now, I think

[GitHub] [incubator-hudi] lyogev commented on issue #498: Is there any record delete examples?

2019-04-02 Thread GitBox
lyogev commented on issue #498: Is there any record delete examples? URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479134812 Sorry figured it out. Created a class: ``` class EmptyHoodieRecordPayload extends HoodieRecordPayload[EmptyHoodieRecordPayload] {

[GitHub] [incubator-hudi] n3nash commented on issue #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close()

2019-04-02 Thread GitBox
n3nash commented on issue #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close() URL: https://github.com/apache/incubator-hudi/pull/621#issuecomment-479254921 Okay, fair enough. This is an automated

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #599: Fixing small file handling, inline compaction defaults

2019-04-02 Thread GitBox
n3nash commented on a change in pull request #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#discussion_r271531565 ## File path: hoodie-spark/src/test/scala/DataSourceTest.scala ## @@ -131,6 +131,7 @@

[GitHub] [incubator-hudi] thesuperzapper opened a new issue #628: Replace Databricks spark-avro with native spark-avro

2019-04-02 Thread GitBox
thesuperzapper opened a new issue #628: Replace Databricks spark-avro with native spark-avro URL: https://github.com/apache/incubator-hudi/issues/628 I believe that we should bump to Spark 2.4 minimum for the native spark-avro package. Its just going to become problematic the longer we

[GitHub] [incubator-hudi] vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults

2019-04-02 Thread GitBox
vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479337017 I think repush got rid of it.. fished it out of email notification > I see, put a sleep(1s) for now ?

[GitHub] [incubator-hudi] vinothchandar commented on issue #628: Replace Databricks spark-avro with native spark-avro

2019-04-02 Thread GitBox
vinothchandar commented on issue #628: Replace Databricks spark-avro with native spark-avro URL: https://github.com/apache/incubator-hudi/issues/628#issuecomment-479336469 Agree.. IIRC someone was working on spark 2.4 upgrade. Mind joining the mailing list and kick off a thread there?

[GitHub] [incubator-hudi] vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults

2019-04-02 Thread GitBox
vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479336163 I think I resolved both your comments? @n3nash did I miss something? I added the comment to denoet the actual MB

[GitHub] [incubator-hudi] vinothchandar merged pull request #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close()

2019-04-02 Thread GitBox
vinothchandar merged pull request #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close() URL: https://github.com/apache/incubator-hudi/pull/621 This is an automated message from the Apache Git

[GitHub] [incubator-hudi] BigdataGit commented on issue #607: The difference between `_hoodie_record_key` and `_row_key`?

2019-03-20 Thread GitBox
BigdataGit commented on issue #607: The difference between `_hoodie_record_key` and `_row_key`? URL: https://github.com/apache/incubator-hudi/issues/607#issuecomment-475094145 > yes. to add, all merges/compaction etc happens based on _hoodie_record_key once its extracted out from the

[GitHub] [incubator-hudi] ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code

2019-03-25 Thread GitBox
ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476446712 > @ambition119 squashing is different.. I think those are for rebasing (original error you reported).. > >

[GitHub] [incubator-hudi] n3nash commented on issue #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
n3nash commented on issue #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476390837 @bvaradar Fixed the code and added a small test case as well, please take a look. @vinothchandar please review

[GitHub] [incubator-hudi] vinothchandar closed pull request #535: Build with Error Prone

2019-03-25 Thread GitBox
vinothchandar closed pull request #535: Build with Error Prone URL: https://github.com/apache/incubator-hudi/pull/535 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-hudi] vinothchandar commented on issue #535: Build with Error Prone

2019-03-25 Thread GitBox
vinothchandar commented on issue #535: Build with Error Prone URL: https://github.com/apache/incubator-hudi/pull/535#issuecomment-476417963 closing due to inactivity This is an automated message from the Apache Git Service.

[GitHub] [incubator-hudi] n3nash edited a comment on issue #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
n3nash edited a comment on issue #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476390837 @bvaradar Updated and added a small test case as well, please take a look. @vinothchandar please review

[GitHub] [incubator-hudi] vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex code

2019-03-25 Thread GitBox
vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex code URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476408168 @ambition119 squashing is different.. I think those are for rebasing (original error you reported)..

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268887126 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r26636 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268886753 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r26490 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-25 Thread GitBox
vinothchandar commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268887669 ## File path:

[GitHub] [incubator-hudi] leilinen commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation

2019-03-27 Thread GitBox
leilinen commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477158609 I noticed that the dstream consumed from kafka topic was empty, for dstream.foreachRDD resulted nothing. But the

[GitHub] [incubator-hudi] leilinen opened a new issue #615: how change HoodieDeltaStreamer with Real-time calculation

2019-03-26 Thread GitBox
leilinen opened a new issue #615: how change HoodieDeltaStreamer with Real-time calculation URL: https://github.com/apache/incubator-hudi/issues/615 Hi, In my project, I want to consume data from kafka topic and upsert data with hoodie. In hoodie, the HoodieDeltaStreamer is

[GitHub] [incubator-hudi] cdmikechen commented on issue #547: The question about Hudi transform timestamp and decimal's type

2019-03-26 Thread GitBox
cdmikechen commented on issue #547: The question about Hudi transform timestamp and decimal's type URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476621181 @vinothchandar Sorry for delay. I've changed my job in February, and my new job works for an automobile

[GitHub] [incubator-hudi] vinothchandar commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation

2019-03-27 Thread GitBox
vinothchandar commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477248903 lets continue on the mailing list.. ! This

[GitHub] [incubator-hudi] vinothchandar commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation

2019-03-27 Thread GitBox
vinothchandar commented on issue #615: how change HoodieDeltaStreamer with Real-time calculation URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477248778 @leilinen please post on the mailing list for faster response. https://hudi.apache.org/community.html

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #617: Fixes HUDI-9 : Check precondition minInstantsToKeep > cleanerCommitsR…

2019-03-27 Thread GitBox
vinothchandar commented on a change in pull request #617: Fixes HUDI-9 : Check precondition minInstantsToKeep > cleanerCommitsR… URL: https://github.com/apache/incubator-hudi/pull/617#discussion_r269683323 ## File path:

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #616: Fixes HUDI-38: Reduce memory overhead of WriteStatus

2019-03-27 Thread GitBox
bvaradar commented on a change in pull request #616: Fixes HUDI-38: Reduce memory overhead of WriteStatus URL: https://github.com/apache/incubator-hudi/pull/616#discussion_r269691293 ## File path: hoodie-client/src/main/java/com/uber/hoodie/WriteStatus.java ## @@ -48,6

[GitHub] [incubator-hudi] vinothchandar commented on issue #616: Fixes HUDI-38: Reduce memory overhead of WriteStatus

2019-03-27 Thread GitBox
vinothchandar commented on issue #616: Fixes HUDI-38: Reduce memory overhead of WriteStatus URL: https://github.com/apache/incubator-hudi/pull/616#issuecomment-477284353 yes @ovj @n3nash the custom WriteStatus need to change post release. its okay for you to ignore the args though.. Once

[GitHub] [incubator-hudi] vinothchandar merged pull request #617: Fixes HUDI-9 : Check precondition minInstantsToKeep > cleanerCommitsR…

2019-03-27 Thread GitBox
vinothchandar merged pull request #617: Fixes HUDI-9 : Check precondition minInstantsToKeep > cleanerCommitsR… URL: https://github.com/apache/incubator-hudi/pull/617 This is an automated message from the Apache Git Service.

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236754 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java

[GitHub] [incubator-hudi] bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat

2019-03-26 Thread GitBox
bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476779092 @Achuth17 : This was done on purpose. There are lot of different hadoop/hive environments out there. With Hudi having to interoperate

[GitHub] [incubator-hudi] bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat

2019-03-26 Thread GitBox
bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476779549 @daikon12 : Is run_hive_sync tool working fine now ? This is an

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269237813 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269237737 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236754 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236502 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieCreateHandle.java

[GitHub] [incubator-hudi] n3nash commented on issue #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on issue #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476771521 @vinothchandar Addressed and replied to comments.

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads

2019-03-26 Thread GitBox
n3nash commented on a change in pull request #612: Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269238293 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type

2019-03-26 Thread GitBox
vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476752594 Congrats on the new gig! :) This is an

[GitHub] [incubator-hudi] vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type

2019-03-26 Thread GitBox
vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476752493 @cdmikechen no worries.. it would help if you can file a JIRA on hive/hadoop version gaps preventing you

[GitHub] [incubator-hudi] vinothchandar merged pull request #608: [HUDI-63] Removed unused BucketedIndex code

2019-03-26 Thread GitBox
vinothchandar merged pull request #608: [HUDI-63] Removed unused BucketedIndex code URL: https://github.com/apache/incubator-hudi/pull/608 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-hudi] vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex code

2019-03-26 Thread GitBox
vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex code URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476753401 awesome .. @ambition119 . congrats on your first PR :) This is

[GitHub] [incubator-hudi] Zhujun-Vungle commented on issue #143: Tracking ticket for folks to be added to slack group

2019-03-21 Thread GitBox
Zhujun-Vungle commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-475150950 Hi would you add me to slack please~ jun@vungle.com

[GitHub] [incubator-hudi] vinothchandar closed issue #580: Change serializer for DiskBasedMap to handle non serializable java objects

2019-03-22 Thread GitBox
vinothchandar closed issue #580: Change serializer for DiskBasedMap to handle non serializable java objects URL: https://github.com/apache/incubator-hudi/issues/580 This is an automated message from the Apache Git Service.

[GitHub] [incubator-hudi] vinothchandar commented on issue #596: Unable to find previous checkpoint. Please double check if this table was indeed built via delta streamer

2019-03-22 Thread GitBox
vinothchandar commented on issue #596: Unable to find previous checkpoint. Please double check if this table was indeed built via delta streamer URL: https://github.com/apache/incubator-hudi/issues/596#issuecomment-475820514 Closing this due to inactivity

[GitHub] [incubator-hudi] vinothchandar closed issue #596: Unable to find previous checkpoint. Please double check if this table was indeed built via delta streamer

2019-03-22 Thread GitBox
vinothchandar closed issue #596: Unable to find previous checkpoint. Please double check if this table was indeed built via delta streamer URL: https://github.com/apache/incubator-hudi/issues/596 This is an automated

[GitHub] [incubator-hudi] vinothchandar opened a new pull request #611: Refreshing site content

2019-03-22 Thread GitBox
vinothchandar opened a new pull request #611: Refreshing site content URL: https://github.com/apache/incubator-hudi/pull/611 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-hudi] n3nash opened a new pull request #612: (WIP) Fixing source schema and writer schema distinction in payloads

2019-03-22 Thread GitBox
n3nash opened a new pull request #612: (WIP) Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612 This is an automated message from the Apache Git Service. To

[GitHub] [incubator-hudi] vinothchandar commented on issue #493: Hive partition column defaults to String

2019-03-22 Thread GitBox
vinothchandar commented on issue #493: Hive partition column defaults to String URL: https://github.com/apache/incubator-hudi/issues/493#issuecomment-475820882 @AndrewKL non partitioned tables must be supported now.. (@bvaradar fyi).. close this?

[GitHub] [incubator-hudi] vinothchandar closed issue #540: `COMBINE_BEFORE_INSERT_PROP` and `COMBINE_BEFORE_UPSERT_PROP` default values

2019-03-22 Thread GitBox
vinothchandar closed issue #540: `COMBINE_BEFORE_INSERT_PROP` and `COMBINE_BEFORE_UPSERT_PROP` default values URL: https://github.com/apache/incubator-hudi/issues/540 This is an automated message from the Apache Git

[GitHub] [incubator-hudi] vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type

2019-03-22 Thread GitBox
vinothchandar commented on issue #547: The question about Hudi transform timestamp and decimal's type URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-475820698 @cdmikechen there is a thread on this on the mailing list. Are you still working/hitting this? If so, could

[GitHub] [incubator-hudi] vinothchandar commented on issue #540: `COMBINE_BEFORE_INSERT_PROP` and `COMBINE_BEFORE_UPSERT_PROP` default values

2019-03-22 Thread GitBox
vinothchandar commented on issue #540: `COMBINE_BEFORE_INSERT_PROP` and `COMBINE_BEFORE_UPSERT_PROP` default values URL: https://github.com/apache/incubator-hudi/issues/540#issuecomment-475820737 Closing this due to inactivity

[GitHub] [incubator-hudi] vinothchandar merged pull request #611: Refreshing site content

2019-03-22 Thread GitBox
vinothchandar merged pull request #611: Refreshing site content URL: https://github.com/apache/incubator-hudi/pull/611 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #612: (WIP) Fixing source schema and writer schema distinction in payloads

2019-03-22 Thread GitBox
bvaradar commented on a change in pull request #612: (WIP) Fixing source schema and writer schema distinction in payloads URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268368530 ## File path:

[GitHub] [incubator-hudi] Achuth17 commented on issue #581: ClassNotFoundException:HoodieInputFormat

2019-03-24 Thread GitBox
Achuth17 commented on issue #581: ClassNotFoundException:HoodieInputFormat URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476058892 I faced the antlr recognition exception as well, Fixed it by adding a few jars (like the antlr jar) manually. Just thinking out loud,

[GitHub] [incubator-hudi] vinothchandar commented on issue #613: Too many S3 Connections being opened while df.write

2019-03-24 Thread GitBox
vinothchandar commented on issue #613: Too many S3 Connections being opened while df.write URL: https://github.com/apache/incubator-hudi/issues/613#issuecomment-476034123 https://issues.apache.org/jira/browse/HUDI-80 tracks a potential fix for this..

[GitHub] [incubator-hudi] smdahmed opened a new issue #613: Too many S3 Connections being opened while df.write

2019-03-23 Thread GitBox
smdahmed opened a new issue #613: Too many S3 Connections being opened while df.write URL: https://github.com/apache/incubator-hudi/issues/613 1. Table clean is called for every dataframe write

[GitHub] [incubator-hudi] ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code

2019-03-23 Thread GitBox
ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-475876976 > awesome.. can you please squash the commits again? :) I use these command : ```shell $ git remote add apache

[GitHub] [incubator-hudi] vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark

2019-04-03 Thread GitBox
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-479576736 Also lets take this discussion to the mailing list, so others also benefit :)

[GitHub] [incubator-hudi] vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark

2019-04-03 Thread GitBox
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for writing through spark URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-479576565 @leilinen so may be with increased parallelism, hitting the race condition becomes more probable.

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905565 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905880 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271912046 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieWriteClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271915076 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieAppendHandle.java ##

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271922431 ## File path: hoodie-client/src/test/java/com/uber/hoodie/io/strategy/TestHoodieDataFile.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271927078 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/TableFileSystemView.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271931617 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905131 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271914607 ## File path: hoodie-client/src/main/java/com/uber/hoodie/func/EmbeddedTimelineServer.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925460 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271926261 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieView.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925979 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieView.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271918483 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271928501 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271906279 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271911256 ## File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieWriteClient.java ## @@

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271913114 ## File path: hoodie-client/src/main/java/com/uber/hoodie/func/EmbeddedTimelineServer.java

[GitHub] [incubator-hudi] n3nash opened a new pull request #629: 1. Minor changes to fix compaction 2. Adding 2 compaction policies 3. Adding a Hbase index property

2019-04-03 Thread GitBox
n3nash opened a new pull request #629: 1. Minor changes to fix compaction 2. Adding 2 compaction policies 3. Adding a Hbase index property URL: https://github.com/apache/incubator-hudi/pull/629 This is an automated message

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271917051 ## File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieCleanHelper.java ##

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271918483 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271923395 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieDataFile.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271923395 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieDataFile.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271924558 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieWriteStat.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271928350 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271948421 ## File path:

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271922608 ## File path: hoodie-client/src/test/java/com/uber/hoodie/table/TestMergeOnReadTable.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925460 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support

2019-04-03 Thread GitBox
n3nash commented on a change in pull request #600: Timeline Service with Incremental View Syncing support URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925335 ## File path: hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java

  1   2   3   4   5   6   7   8   9   10   >