[GitHub] [incubator-hudi] amaranathv edited a comment on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
amaranathv edited a comment on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510581260 I am getting same error. ``` scala>

[GitHub] [incubator-hudi] vinothchandar commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
vinothchandar commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510606829 @amaranathv just to confirm you are using `NonpartitionedKeyGenerator` as the key

[GitHub] [incubator-hudi] amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510609856 yes This is an automated

[GitHub] [incubator-hudi] amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510609967 DataSourceWriteOptions.KEYGENERATOR_CLASS_OPT_KEY->

[GitHub] [incubator-hudi] vinothchandar commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI

2019-07-11 Thread GitBox
vinothchandar commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714#issuecomment-510636174 One thing I notice is about 10 mins is spent in retrying the stages . This is usually indicatve that the

[GitHub] [incubator-hudi] vinothchandar commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
vinothchandar commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510644864 @amaranathv again the issue from this line `.onParentPath(new

[GitHub] [incubator-hudi] thesuperzapper commented on issue #638: [HUDI-12] Upgrade to Spark 2.4, Avro 1.8.2, Parquet 1.10.0...

2019-07-11 Thread GitBox
thesuperzapper commented on issue #638: [HUDI-12] Upgrade to Spark 2.4, Avro 1.8.2, Parquet 1.10.0... URL: https://github.com/apache/incubator-hudi/pull/638#issuecomment-510649936 We are working on #780 first This is an

[GitHub] [incubator-hudi] vinothchandar commented on issue #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on issue #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#issuecomment-510538727 @thesuperzapper Thanks! This is great.. let me review these. I have also been working on #751 around the same thing.. Ideally, if we can merge

[GitHub] [incubator-hudi] vinothchandar commented on issue #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on issue #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#issuecomment-510547665 @thesuperzapper as for testing, if you can run the demo steps once and confirm there are no NoClassDefFound errors and such, it would be a good

[GitHub] [incubator-hudi] vinothchandar commented on issue #700: HUDI-138 - Meta Files handling also need to support consistency guard

2019-07-11 Thread GitBox
vinothchandar commented on issue #700: HUDI-138 - Meta Files handling also need to support consistency guard URL: https://github.com/apache/incubator-hudi/pull/700#issuecomment-510562983 @bvaradar is this ready for merging?

[GitHub] [incubator-hudi] NetsanetGeb opened a new issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI

2019-07-11 Thread GitBox
NetsanetGeb opened a new issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714 I am trying to ingest data using the DatasourceAPI and HoodieDeltaStreamer. The ingestion latency of using DataSource API with

[GitHub] [incubator-hudi] NetsanetGeb edited a comment on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI

2019-07-11 Thread GitBox
NetsanetGeb edited a comment on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714#issuecomment-510569606 **Benchmarking Hudi Upsert** I am trying to bench mark Hudi upsert operation and the latency

[GitHub] [incubator-hudi] NetsanetGeb commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI

2019-07-11 Thread GitBox
NetsanetGeb commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714#issuecomment-510569606 **### Bench marking Hudi Upsert** I am trying to bench mark Hudi upsert operation and the latency of

[GitHub] [incubator-hudi] vinothchandar commented on issue #735: Failed to instantiate SLF4J LoggerFactory

2019-07-11 Thread GitBox
vinothchandar commented on issue #735: Failed to instantiate SLF4J LoggerFactory URL: https://github.com/apache/incubator-hudi/issues/735#issuecomment-510579271 @Gowthamsb12 #751 is redoing the bundles. but for this issue, I think you should be able to find log4J if the classpath is set

[GitHub] [incubator-hudi] tooptoop4 commented on issue #638: [HUDI-12] Upgrade to Spark 2.4, Avro 1.8.2, Parquet 1.10.0...

2019-07-11 Thread GitBox
tooptoop4 commented on issue #638: [HUDI-12] Upgrade to Spark 2.4, Avro 1.8.2, Parquet 1.10.0... URL: https://github.com/apache/incubator-hudi/pull/638#issuecomment-510575991 @thesuperzapper @bvaradar gentle ping This is an

[GitHub] [incubator-hudi] vinothchandar closed issue #705: hadoop 2.8.x miss RecoveryInProgressException class

2019-07-11 Thread GitBox
vinothchandar closed issue #705: hadoop 2.8.x miss RecoveryInProgressException class URL: https://github.com/apache/incubator-hudi/issues/705 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-hudi] amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present

2019-07-11 Thread GitBox
amaranathv commented on issue #764: Hoodie 0.4.7: Error upserting bucketType UPDATE for partition #, No value present URL: https://github.com/apache/incubator-hudi/issues/764#issuecomment-510581260 I am getting same error. scala>

[GitHub] [incubator-hudi] vinothchandar commented on issue #736: hoodie-hive-hundle don't have hive jars

2019-07-11 Thread GitBox
vinothchandar commented on issue #736: hoodie-hive-hundle don't have hive jars URL: https://github.com/apache/incubator-hudi/issues/736#issuecomment-510581625 @cdmikechen these jars are in the hive installation, thats why we don't bundle them. ``` ls hive/lib/datanucleus-*

[GitHub] [incubator-hudi] vinothchandar commented on issue #550: NoSuchMethodError: org.eclipse.jetty.servlet.ServletMapping.setDefault(Z)V

2019-07-11 Thread GitBox
vinothchandar commented on issue #550: NoSuchMethodError: org.eclipse.jetty.servlet.ServletMapping.setDefault(Z)V URL: https://github.com/apache/incubator-hudi/issues/550#issuecomment-510590195 In #751 , I actually found that the jetty jars are needed by the new embedded timeline server..

[GitHub] [incubator-hudi] yihua opened a new pull request #781: [HUDI-161] Remove --key-generator-class CLI arg in HoodieDeltaStreamer and use key generator class specified in datasource properties

2019-07-11 Thread GitBox
yihua opened a new pull request #781: [HUDI-161] Remove --key-generator-class CLI arg in HoodieDeltaStreamer and use key generator class specified in datasource properties URL: https://github.com/apache/incubator-hudi/pull/781 1. Removing --key-generator-class CLI arg in

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#discussion_r302762792 ## File path: hoodie-cli/pom.xml ## @@ -159,67 +176,51 @@ spark-sql_2.11 + -

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#discussion_r302762037 ## File path: hoodie-cli/pom.xml ## @@ -29,8 +29,6 @@ 1.2.0.RELEASE

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#discussion_r302763536 ## File path: hoodie-client/pom.xml ## @@ -78,6 +79,71 @@ hoodie-timeline-service

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#discussion_r302762896 ## File path: hoodie-cli/pom.xml ## @@ -159,67 +176,51 @@ spark-sql_2.11 + -

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath

2019-07-11 Thread GitBox
vinothchandar commented on a change in pull request #780: Cleanup Maven POM/Classpath URL: https://github.com/apache/incubator-hudi/pull/780#discussion_r302763907 ## File path: hoodie-client/pom.xml ## @@ -78,6 +79,71 @@ hoodie-timeline-service