[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #728: adding support for complex keys

vinothchandar commented on a change in pull request #728: adding support for complex keys URL: https://github.com/apache/incubator-hudi/pull/728#discussion_r296105100 ## File path: hoodie-spark/src/main/java/com/uber/hoodie/ComplexKeyGenerator.java ## @@ -0,0 +1,81 @@ +/*

[GitHub] [incubator-hudi] vinothchandar merged pull request #750: Adding support for optional skipping single archiving failures

vinothchandar merged pull request #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750 This is an automated message from the Apache Git Service. To

[incubator-hudi] branch master updated: Adding support for optional skipping single archiving failures

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 1b61eb4 Adding support for optional

[GitHub] [incubator-hudi] vinothchandar commented on issue #750: Adding support for optional skipping single archiving failures

vinothchandar commented on issue #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750#issuecomment-504300018 Thanks for the fix and the note @RonBarabash !

[incubator-hudi] branch master updated: Reword confusing message and reducing the severity level

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 66c7fa2 Reword confusing message and

[GitHub] [incubator-hudi] vinothchandar merged pull request #752: Reword confusing message and reducing the severity level

vinothchandar merged pull request #752: Reword confusing message and reducing the severity level URL: https://github.com/apache/incubator-hudi/pull/752 This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-hudi] vinothchandar merged pull request #756: Add maprfs to storage schemes

vinothchandar merged pull request #756: Add maprfs to storage schemes URL: https://github.com/apache/incubator-hudi/pull/756 This is an automated message from the Apache Git Service. To respond to the message, please log on

[incubator-hudi] branch master updated: Add maprfs to storage schemes

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 8223127 Add maprfs to storage schemes

[GitHub] [incubator-hudi] cdmikechen commented on issue #754: InvalidSchemaException

cdmikechen commented on issue #754: InvalidSchemaException URL: https://github.com/apache/incubator-hudi/issues/754#issuecomment-504298032 When I moved some hive2.* jars to SPARK_HOME/jars/ and then wrote some data to parquet, it reported error like that. Were you doing something like

[GitHub] [incubator-hudi] xuFabius commented on issue #754: InvalidSchemaException

xuFabius commented on issue #754: InvalidSchemaException URL: https://github.com/apache/incubator-hudi/issues/754#issuecomment-504261071 Thanks for cdmikechen's comment. Data schema is 'com.uber.hoodie' . run_hoodie_app.sh is is hudi demo script in hoodie-spark project. My command

[GitHub] [incubator-hudi] cdmikechen commented on issue #754: InvalidSchemaException

cdmikechen commented on issue #754: InvalidSchemaException URL: https://github.com/apache/incubator-hudi/issues/754#issuecomment-504253795 Can you show your schema and write data? In spark2.3, spark use avro1.7, if you want to use avro1.8, you need to replace your SPARK_HOME/jars/avro*

[GitHub] [incubator-hudi] cdmikechen commented on issue #753: when column type is decimal, should add precision and scale

cdmikechen commented on issue #753: when column type is decimal, should add precision and scale URL: https://github.com/apache/incubator-hudi/pull/753#issuecomment-504250543 I found this problem when I revised a change about not using `com.databricks:spark-avro_2.11` and letting hoodie

[GitHub] [incubator-hudi] cdmikechen commented on issue #753: when column type is decimal, should add precision and scale

cdmikechen commented on issue #753: when column type is decimal, should add precision and scale URL: https://github.com/apache/incubator-hudi/pull/753#issuecomment-504249248 @vinothchandar yes. I have a hive table with decimal column. When I used `run_sync_tool.sh` to sync a hoodie

[GitHub] [incubator-hudi] bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504213734 Awesome. Closing this issue. This

[GitHub] [incubator-hudi] bvaradar closed issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar closed issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741 This is an automated message from the Apache Git Service. To

[GitHub] [incubator-hudi] bvaradar commented on issue #756: Add maprfs to storage schemes

bvaradar commented on issue #756: Add maprfs to storage schemes URL: https://github.com/apache/incubator-hudi/pull/756#issuecomment-504213478 @n3nash @vinothchandar : @amaranathv confirmed this patch works with MapR ecosystem

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504209411 Hive sync is also complete and i can query the table as well.. Thanks! for your quick

[incubator-hudi] branch master updated: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration

This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 2c40e84 Ensure TableMetaClient and

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504180560 I am part of mailing list.I will ad to slack channel soon. It is working now.I can see the output

[GitHub] [incubator-hudi] RonBarabash commented on a change in pull request #750: Adding support for optional skipping single archiving failures

RonBarabash commented on a change in pull request #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750#discussion_r295981703 ## File path:

[GitHub] [incubator-hudi] RonBarabash commented on a change in pull request #750: Adding support for optional skipping single archiving failures

RonBarabash commented on a change in pull request #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750#discussion_r295979325 ## File path:

[GitHub] [incubator-hudi] RonBarabash commented on issue #661: Tracking ticket for reporting Hudi usages from the community

RonBarabash commented on issue #661: Tracking ticket for reporting Hudi usages from the community URL: https://github.com/apache/incubator-hudi/issues/661#issuecomment-504168231 Hey, We are using Hudi at Yotpo for several usages. Firstly, we integrated Hudi as a writer in our open

[GitHub] [incubator-hudi] bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504161194 @amaranathv : Thanks. Also join the mailing list and slack channel if you have not

[GitHub] [incubator-hudi] bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504155185 @amaranathv : Can you rebuild Hudi with this patch https://github.com/apache/incubator-hudi/pull/756

[GitHub] [incubator-hudi] bvaradar opened a new pull request #756: [WIP] Add maprfs to storage schemes

bvaradar opened a new pull request #756: [WIP] Add maprfs to storage schemes URL: https://github.com/apache/incubator-hudi/pull/756 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504147822 Please bear with me for too many questions..If it is cloudera distribution i shouldn't been asking

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504147148

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504145084 Is it because of MapR Hadoop distribution?

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504144982 I am getting below error. Caused by: com.uber.hoodie.exception.HoodieException:

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504137883 Thanks! I will check and revert back...

[GitHub] [incubator-hudi] vinothchandar commented on issue #739: Support building Hoodie with Hadoop 2.8.x

vinothchandar commented on issue #739: Support building Hoodie with Hadoop 2.8.x URL: https://github.com/apache/incubator-hudi/pull/739#issuecomment-504130581 yes.. we cleaned up a bunch of thigns in the pom here. https://github.com/apache/incubator-hudi/tree/hackathon-0619 do you

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #750: Adding support for optional skipping single archiving failures

vinothchandar commented on a change in pull request #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750#discussion_r295934013 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #753: when column type is decimal, should add precision and scale

vinothchandar commented on issue #753: when column type is decimal, should add precision and scale URL: https://github.com/apache/incubator-hudi/pull/753#issuecomment-504127589 @cdmikechen thanks for the fix. does this happen when hive syncing a DECIMAL column?

[GitHub] [incubator-hudi] bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504127186 @amaranathv : From the commit, it looks like you are writing to local file system

[GitHub] [incubator-hudi] vinothchandar commented on issue #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration

vinothchandar commented on issue #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration URL: https://github.com/apache/incubator-hudi/pull/755#issuecomment-504125823 LGTM.. please merge once CI passes

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504123629

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504122108 $ cat 20190618132401.commit { "partitionToWriteStats" : { "2018/08/31" : [ {

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504121832 sorry..i have .hoodie This is

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504121478 No This is an automated message

[GitHub] [incubator-hudi] bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

bvaradar commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504121247 @amaranathv : No .hoodie ? Should be /mapr/user/avenka23/delta-streamer/output8/.hoodie

[GitHub] [incubator-hudi] amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException

amaranathv commented on issue #741: java.lang.NoClassDefFoundError: com/uber/hoodie/exception/HoodieException URL: https://github.com/apache/incubator-hudi/issues/741#issuecomment-504120148 avenka23@dbsld0031:/mapr/user/avenka23/delta-streamer/output8/2018/08/31 $ ls -al total 2

[GitHub] [incubator-hudi] bvaradar commented on issue #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration

bvaradar commented on issue #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration URL: https://github.com/apache/incubator-hudi/pull/755#issuecomment-504099767 @n3nash @vinothchandar : This is to fix ConcurrentModification Exception seen when using

[GitHub] [incubator-hudi] bvaradar opened a new pull request #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration

bvaradar opened a new pull request #755: Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration URL: https://github.com/apache/incubator-hudi/pull/755 Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration

[GitHub] [incubator-hudi] xuFabius opened a new issue #754: InvalidSchemaException

xuFabius opened a new issue #754: InvalidSchemaException URL: https://github.com/apache/incubator-hudi/issues/754 When I run ./run_hoodie_app.sh. The program ended with InvalidSchemaException. ` Exception in thread "main" java.lang.ExceptionInInitializerError at

[GitHub] [incubator-hudi] jaimin-shah commented on a change in pull request #728: adding support for complex keys

jaimin-shah commented on a change in pull request #728: adding support for complex keys URL: https://github.com/apache/incubator-hudi/pull/728#discussion_r295708650 ## File path: hoodie-spark/src/main/java/com/uber/hoodie/ComplexKeyGenerator.java ## @@ -0,0 +1,81 @@ +/* +

[GitHub] [incubator-hudi] katiefrost95 commented on issue #143: Tracking ticket for folks to be added to slack group

katiefrost95 commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-503930424 Hey, can I be added to the slack channel? - katiesfros...@gmail.com, thanks

[GitHub] [incubator-hudi] RonBarabash commented on issue #750: Adding support for optional skipping single archiving failures

RonBarabash commented on issue #750: Adding support for optional skipping single archiving failures URL: https://github.com/apache/incubator-hudi/pull/750#issuecomment-503916979 Hey @vinothchandar, thanks for the response! This error occurred sporadically after a running a streaming