[incubator-hudi] branch asf-site updated: [MINOR] Add Hudi Online Meetup (#1441)

2020-03-24 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 691c522 [MINOR] Add Hudi Online

[GitHub] [incubator-hudi] yanghua merged pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
yanghua merged pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441#discussion_r397618497 ## File path: docs/_docs/1_4_powered_by.cn.md ## @@ -48,10 +48,20 @@

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441#discussion_r397607372 ## File path: docs/_docs/1_4_powered_by.cn.md ## @@ -48,10 +48,20 @@

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
codecov-io edited a comment on issue #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#issuecomment-602121990 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1436?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441#discussion_r397600716 ## File path: docs/_docs/1_4_powered_by.cn.md ## @@ -48,10 +48,20 @@

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
yanghua commented on a change in pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441#discussion_r397600382 ## File path: docs/_docs/1_4_powered_by.cn.md ## @@ -48,10 +48,20 @@

[GitHub] [incubator-hudi] leesf opened a new pull request #1441: [MINOR] Add Hudi Online Meetup

2020-03-24 Thread GitBox
leesf opened a new pull request #1441: [MINOR] Add Hudi Online Meetup URL: https://github.com/apache/incubator-hudi/pull/1441 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.*

[GitHub] [incubator-hudi] xushiyan commented on issue #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
xushiyan commented on issue #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#issuecomment-603626530 @leesf Please kindly review the last commit for the commented testcases. Note that I omitted the `assertFalse(dfs.exists(new

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397592137 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #227

2020-03-24 Thread Apache Jenkins Server
See Changes: -- [...truncated 2.32 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397592137 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
xushiyan commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397590026 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397585126 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ##

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397585042 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ##

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397584926 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ##

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397584926 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ##

[GitHub] [incubator-hudi] leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic

2020-03-24 Thread GitBox
leesf commented on a change in pull request #1436: [HUDI-711] Refactor exporter main logic URL: https://github.com/apache/incubator-hudi/pull/1436#discussion_r397584857 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieSnapshotExporter.java ##

[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

2020-03-24 Thread GitBox
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-603506326 yes I am also wondering if we can do something about incompatibility.. that a large issue.. we

[GitHub] [incubator-hudi] bvaradar commented on issue #1371: [SUPPORT] Upsert for S3 Hudi dataset with large partitions takes a lot of time in writing

2020-03-24 Thread GitBox
bvaradar commented on issue #1371: [SUPPORT] Upsert for S3 Hudi dataset with large partitions takes a lot of time in writing URL: https://github.com/apache/incubator-hudi/issues/1371#issuecomment-603492434 https://github.com/apache/incubator-hudi/pull/1394 merged. Resolving this ticket.

[GitHub] [incubator-hudi] bvaradar closed issue #1371: [SUPPORT] Upsert for S3 Hudi dataset with large partitions takes a lot of time in writing

2020-03-24 Thread GitBox
bvaradar closed issue #1371: [SUPPORT] Upsert for S3 Hudi dataset with large partitions takes a lot of time in writing URL: https://github.com/apache/incubator-hudi/issues/1371 This is an automated message from the Apache

[GitHub] [incubator-hudi] bvaradar commented on issue #1284: [SUPPORT]

2020-03-24 Thread GitBox
bvaradar commented on issue #1284: [SUPPORT] URL: https://github.com/apache/incubator-hudi/issues/1284#issuecomment-603491329 closing this ticket due to inactivity This is an automated message from the Apache Git Service. To

[GitHub] [incubator-hudi] bvaradar closed issue #1284: [SUPPORT]

2020-03-24 Thread GitBox
bvaradar closed issue #1284: [SUPPORT] URL: https://github.com/apache/incubator-hudi/issues/1284 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-hudi] nsivabalan commented on issue #1438: How to get the file name corresponding to HoodieKey through the GlobalBloomIndex

2020-03-24 Thread GitBox
nsivabalan commented on issue #1438: How to get the file name corresponding to HoodieKey through the GlobalBloomIndex URL: https://github.com/apache/incubator-hudi/issues/1438#issuecomment-603483852 Sure. will take a look.

[GitHub] [incubator-hudi] afilipchik commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

2020-03-24 Thread GitBox
afilipchik commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-603459354 is there a way to avoid incompatibility? We have couple of streams with complex schemas written with

[GitHub] [incubator-hudi] xushiyan commented on issue #1440: [HUDI-731] Add ChainedTransformer

2020-03-24 Thread GitBox
xushiyan commented on issue #1440: [HUDI-731] Add ChainedTransformer URL: https://github.com/apache/incubator-hudi/pull/1440#issuecomment-603417382 @vinothchandar @yanghua Sure thanks for the input. Since the parsing approach saves some efforts for users, I can make it that way. May I have

[GitHub] [incubator-hudi] zhedoubushishi commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

2020-03-24 Thread GitBox
zhedoubushishi commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-603400047 > > > So anyone who has written data using databricks-avro will face issues reading. > > By

[GitHub] [incubator-hudi] zhedoubushishi commented on a change in pull request #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

2020-03-24 Thread GitBox
zhedoubushishi commented on a change in pull request #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#discussion_r397341690 ## File path:

[GitHub] [incubator-hudi] bvaradar commented on issue #1439: Hudi class loading problem

2020-03-24 Thread GitBox
bvaradar commented on issue #1439: Hudi class loading problem URL: https://github.com/apache/incubator-hudi/issues/1439#issuecomment-603394312 @melkimohamed : From your description, hive.reloadable.aux.jars.path=/usr/hudi/hudi-hive-bundle-0.5.0-incubating.jar It looks

[GitHub] [incubator-hudi] bvaradar commented on issue #1420: Broken Maven dependencies.

2020-03-24 Thread GitBox
bvaradar commented on issue #1420: Broken Maven dependencies. URL: https://github.com/apache/incubator-hudi/issues/1420#issuecomment-603380871 @deabreu : Were you able to build locally ? This is an automated message from the

[GitHub] [incubator-hudi] wangxianghu commented on a change in pull request #1409: [HUDI-714]Add javadoc and comments to hudi write method link

2020-03-24 Thread GitBox
wangxianghu commented on a change in pull request #1409: [HUDI-714]Add javadoc and comments to hudi write method link URL: https://github.com/apache/incubator-hudi/pull/1409#discussion_r397124853 ## File path: hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java

[GitHub] [incubator-hudi] umehrot2 commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

2020-03-24 Thread GitBox
umehrot2 commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r397037244 ## File path:

[GitHub] [incubator-hudi] umehrot2 commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

2020-03-24 Thread GitBox
umehrot2 commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r397012179 ## File path:

[GitHub] [incubator-hudi] umehrot2 commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

2020-03-24 Thread GitBox
umehrot2 commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r397009308 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

2020-03-24 Thread GitBox
pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396992378 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

2020-03-24 Thread GitBox
pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396992378 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

2020-03-24 Thread GitBox
pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396992378 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

2020-03-24 Thread GitBox
pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396965661 ## File path:

[GitHub] [incubator-hudi] yanghua commented on issue #1440: [HUDI-731] Add ChainedTransformer

2020-03-24 Thread GitBox
yanghua commented on issue #1440: [HUDI-731] Add ChainedTransformer URL: https://github.com/apache/incubator-hudi/pull/1440#issuecomment-603045516 @xushiyan 's explanation is correct. It's just about trade-off. Both of them can achieve the same function, but the mechanism of collecting