[GitHub] [hudi] selvarajperiyasamy commented on issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
selvarajperiyasamy commented on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-635772265 Hi @lamber-ken , is this Spark option available in 0.5.0. I tired , it didn’t work . When I checked in Hudi code base , this string not found anywhere . Attached the image

[GitHub] [hudi] vinothchandar commented on pull request #1638: HUDI-515 Resolve API conflict for Hive 2 & Hive 3

2020-05-28 Thread GitBox
vinothchandar commented on pull request #1638: URL: https://github.com/apache/hudi/pull/1638#issuecomment-635771496 @n3nash were you referring to an umbrella Jira for hive 3 work? This pr itself seems simple enough to go first right

[GitHub] [hudi] vinothchandar commented on pull request #1100: [HUDI-289] Implement a test suite to support long running test for Hudi writing and querying end-end

2020-05-28 Thread GitBox
vinothchandar commented on pull request #1100: URL: https://github.com/apache/hudi/pull/1100#issuecomment-635770389 I will review this over the weekend and merge with some changes on top.. this will speed things up This is

[hudi] branch asf-site updated: Travis CI build asf-site

2020-05-28 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new bfba9f6 Travis CI build asf-site bfba9f6 is

[GitHub] [hudi] leesf merged pull request #1681: [MINOR] Updated prerequisites: jq, Maven, and Java 8

2020-05-28 Thread GitBox
leesf merged pull request #1681: URL: https://github.com/apache/hudi/pull/1681 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] branch asf-site updated: [MINOR] Updated prerequisites: jq, Maven, and Java 8 (#1681)

2020-05-28 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 8231457 [MINOR] Updated prerequisites: jq,

[GitHub] [hudi] leesf merged pull request #1682: [MINOR] Add Text in powered_by

2020-05-28 Thread GitBox
leesf merged pull request #1682: URL: https://github.com/apache/hudi/pull/1682 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] branch asf-site updated: [MINOR] Add Text in powered_by (#1682)

2020-05-28 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new ac868bb [MINOR] Add Text in powered_by

[GitHub] [hudi] leesf commented on pull request #1682: [MINOR] Add Text in powered_by

2020-05-28 Thread GitBox
leesf commented on pull request #1682: URL: https://github.com/apache/hudi/pull/1682#issuecomment-635755576 @lamber-ken Thanks for your review, merging. This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] nsivabalan opened a new pull request #1683: Updating release docs for release-0.5.3

2020-05-28 Thread GitBox
nsivabalan opened a new pull request #1683: URL: https://github.com/apache/hudi/pull/1683 This pull request adds release docs for release-0.5.3 - Updates releases.md with release info for release-0.5.3 This pull request is updating release docs. ## Committer checklist

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #292

2020-05-28 Thread Apache Jenkins Server
See Changes: -- [...truncated 2.42 KB...] settings.xml toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging: simplelogger.properties

[GitHub] [hudi] malanb5 closed pull request #1680: [MINOR] Updated Prerequisites: Java 8 and Maven to be more consistent with th…

2020-05-28 Thread GitBox
malanb5 closed pull request #1680: URL: https://github.com/apache/hudi/pull/1680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] lamber-ken commented on pull request #1681: [MINOR] Updated prerequisite : jq

2020-05-28 Thread GitBox
lamber-ken commented on pull request #1681: URL: https://github.com/apache/hudi/pull/1681#issuecomment-635728301 cc @leesf This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] lamber-ken commented on issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
lamber-ken commented on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-635721693 Hi @selvarajperiyasamy, I guess you used `HiveSyncTool` directly just now. For spark, use ``` option("hoodie.datasource.hive_sync.use_jdbc", "false") ```

[jira] [Assigned] (HUDI-593) Use scala_version placeholder to replace scala version hard code e.g. 2.11

2020-05-28 Thread hong dongdong (Jira)
[ https://issues.apache.org/jira/browse/HUDI-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hong dongdong reassigned HUDI-593: -- Assignee: hong dongdong > Use scala_version placeholder to replace scala version hard code e.g.

[hudi] branch release-0.5.3 updated: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries (#1674)

2020-05-28 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/release-0.5.3 by this push: new 7b1a049 [HUDI-973]

[jira] [Created] (HUDI-977) deal the test module hudi-integ-test

2020-05-28 Thread lichangfu (Jira)
lichangfu created HUDI-977: -- Summary: deal the test module hudi-integ-test Key: HUDI-977 URL: https://issues.apache.org/jira/browse/HUDI-977 Project: Apache Hudi Issue Type: Test

[GitHub] [hudi] leesf opened a new pull request #1682: [MINOR] Add Text in powered_by

2020-05-28 Thread GitBox
leesf opened a new pull request #1682: URL: https://github.com/apache/hudi/pull/1682 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull

[GitHub] [hudi] selvarajperiyasamy commented on issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
selvarajperiyasamy commented on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-635699038 @lamber-ken Do you mean something like below in data source writer ? option(“use-jdbc”,”false”) This is

[GitHub] [hudi] lamber-ken commented on pull request #1680: Updated Prerequisites: Java 8 and Maven to be more consistent with th…

2020-05-28 Thread GitBox
lamber-ken commented on pull request #1680: URL: https://github.com/apache/hudi/pull/1680#issuecomment-635679066 hi @malanb5, please update the commit message and the title of this pr, you can refer to https://github.com/apache/hudi/pull/1681, thanks very much. 

[GitHub] [hudi] lamber-ken commented on pull request #1681: updated prerequisite : jq

2020-05-28 Thread GitBox
lamber-ken commented on pull request #1681: URL: https://github.com/apache/hudi/pull/1681#issuecomment-635677836 Hi @malanb5, please read contribute guide, need `[MINOR]` prefix https://hudi.apache.org/contributing.html#contributing-code

[GitHub] [hudi] lamber-ken commented on pull request #1681: updated prerequisite : jq

2020-05-28 Thread GitBox
lamber-ken commented on pull request #1681: URL: https://github.com/apache/hudi/pull/1681#issuecomment-635677043 Thanks @malanb5, waiting for others to take a final pass. This is an automated message from the Apache

[GitHub] [hudi] abhibhat98 edited a comment on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
abhibhat98 edited a comment on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635675628 Thanks @vinothchandar for a detailed peek into the design. I did this ` spark.sql("select * from test_123 where _hoodie_record_key = 'L1'").show` However, I only

[GitHub] [hudi] abhibhat98 commented on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
abhibhat98 commented on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635675628 Thanks @vinothchandar for a detailed peek into the design. I did this ` spark.sql("select * from test_123 where _hoodie_record_key = 'L1'").show` However, I only got

[jira] [Updated] (HUDI-890) Prepare for 0.5.3 patch release

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-890: - Description: The following commits are included in this release. * #1372 HUDI-652

[jira] [Resolved] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-973. -- Resolution: Fixed > RemoteHoodieTableFileSystemView supports non-partitioned table

[jira] [Commented] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119163#comment-17119163 ] sivabalan narayanan commented on HUDI-973: -- Merged into master via

[jira] [Updated] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-973: - Fix Version/s: 0.5.3 > RemoteHoodieTableFileSystemView supports non-partitioned table

[jira] [Updated] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-973: - Status: In Progress (was: Open) > RemoteHoodieTableFileSystemView supports

[jira] [Assigned] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-973: Assignee: Balaji Varadarajan > RemoteHoodieTableFileSystemView supports

[GitHub] [hudi] lamber-ken edited a comment on issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
lamber-ken edited a comment on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-635670798 hello @selvarajperiyasamy, try option `--use-jdbc false` This is an automated message from the Apache Git

[GitHub] [hudi] lamber-ken commented on issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
lamber-ken commented on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-635670798 hello @selvarajperiyasamy, try option `--use-jdbc` This is an automated message from the Apache Git Service. To

[GitHub] [hudi] malanb5 opened a new pull request #1681: updated prerequisite : jq

2020-05-28 Thread GitBox
malanb5 opened a new pull request #1681: URL: https://github.com/apache/hudi/pull/1681 ## What is the purpose of the pull request updated prerequisite : jq ## Brief change log updated prerequisite : jq ## Verify this pull request Documentation change no tests.

[GitHub] [hudi] malanb5 opened a new pull request #1680: Updated Prerequisites: Java 8 and Maven to be more consistent with th…

2020-05-28 Thread GitBox
malanb5 opened a new pull request #1680: URL: https://github.com/apache/hudi/pull/1680 ## What is the purpose of the pull request Updated Prerequisites: Java 8 and Maven to be more consistent with the README, minor formatting. ## Brief change log Minor formatting added

[GitHub] [hudi] selvarajperiyasamy opened a new issue #1679: How to disable Hive JDBC and enable metastore

2020-05-28 Thread GitBox
selvarajperiyasamy opened a new issue #1679: URL: https://github.com/apache/hudi/issues/1679 Team, My spark version is 2.3.0 Scala version 2.11.8 Hive version 1.2.2 I see the below comment in Hudi code. How can I start using metastore client for hive registrations?

[GitHub] [hudi] vinothchandar commented on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
vinothchandar commented on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635634106 @abhibhat98 Thanks for the thought provoking questions.. table history, is something we already support via the CLI tool.. >> Hudi has the history of everything, it can

[GitHub] [hudi] vinothchandar edited a comment on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
vinothchandar edited a comment on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635634106 @abhibhat98 Thanks for the thought provoking questions.. table history, is something we already support via the CLI tool.. >> Hudi has the history of everything,

[GitHub] [hudi] abhibhat98 edited a comment on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
abhibhat98 edited a comment on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635613375 Understood! Thanks Bhavani! I'd love to take this up. I'll research and get back to you on how to proceed. Follow up question on above. How'd a consumer know the timings of

[GitHub] [hudi] abhibhat98 commented on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
abhibhat98 commented on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635613375 Understood! Thanks Bhavani! I'd love to take this up. I'll research and get back to you on how to proceed. Follow up question on above. How'd a consumer know the timings of the

[GitHub] [hudi] bhasudha commented on issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
bhasudha commented on issue #1675: URL: https://github.com/apache/hudi/issues/1675#issuecomment-635598789 @abhibhat98 Thanks for reaching out. In short there is no direct API to support that use case in Hudi currently. This use case usually fits a K-V storage system that can return

[jira] [Created] (HUDI-976) Support new utility tool to fetch all versions of records involved in an Incremental query

2020-05-28 Thread Bhavani Sudha (Jira)
Bhavani Sudha created HUDI-976: -- Summary: Support new utility tool to fetch all versions of records involved in an Incremental query Key: HUDI-976 URL: https://issues.apache.org/jira/browse/HUDI-976

[GitHub] [hudi] bhasudha commented on issue #998: Incremental view not implemented yet, for merge-on-read datasets

2020-05-28 Thread GitBox
bhasudha commented on issue #998: URL: https://github.com/apache/hudi/issues/998#issuecomment-635588609 > @bhasudha - all the examples on https://hudi.apache.org/docs/quick-start-guide.html#setup-spark-shell for PIT queries are for reading from disk. Does documentation exist for

[GitHub] [hudi] bvaradar merged pull request #1616: [HUDI-786] Fixing read beyond inline length in InlineFS

2020-05-28 Thread GitBox
bvaradar merged pull request #1616: URL: https://github.com/apache/hudi/pull/1616 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[hudi] branch master updated: [HUDI-786] Fixing read beyond inline length in InlineFS (#1616)

2020-05-28 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5a0d3f1 [HUDI-786] Fixing read beyond inline

[GitHub] [hudi] sungjuly commented on issue #661: Tracking ticket for reporting Hudi usages from the community

2020-05-28 Thread GitBox
sungjuly commented on issue #661: URL: https://github.com/apache/hudi/issues/661#issuecomment-635561633 @vinothchandar yes, please do, thank you! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1616: [HUDI-786] Fixing read beyond inline length in InlineFS

2020-05-28 Thread GitBox
codecov-commenter edited a comment on pull request #1616: URL: https://github.com/apache/hudi/pull/1616#issuecomment-633278263 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1616?src=pr=h1) Report > Merging [#1616](https://codecov.io/gh/apache/hudi/pull/1616?src=pr=desc) into

[GitHub] [hudi] bvaradar opened a new pull request #1678: [WIP] [HUDI-242] Metadata Bootstrap changes

2020-05-28 Thread GitBox
bvaradar opened a new pull request #1678: URL: https://github.com/apache/hudi/pull/1678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (HUDI-242) Support Efficient bootstrap of large parquet datasets to Hudi

2020-05-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-242: Labels: pull-request-available (was: ) > Support Efficient bootstrap of large parquet datasets to

[GitHub] [hudi] creactiviti edited a comment on issue #1670: Error opening Hive split: Unknown converted type TIMESTAMP_MICROS

2020-05-28 Thread GitBox
creactiviti edited a comment on issue #1670: URL: https://github.com/apache/hudi/issues/1670#issuecomment-635504343 And as far as querying the parquet table using Presto I first created the table in Hive like so: ``` create external table orders_parquet ( order_id

[GitHub] [hudi] creactiviti commented on issue #1670: Error opening Hive split: Unknown converted type TIMESTAMP_MICROS

2020-05-28 Thread GitBox
creactiviti commented on issue #1670: URL: https://github.com/apache/hudi/issues/1670#issuecomment-635504343 And as far as querying the parquet table using Presto I first created the table in Hive like so: ``` create external table orders_parquet ( order_id

[hudi] branch master updated: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries (#1674)

2020-05-28 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 3574a89 [HUDI-973]

[GitHub] [hudi] bvaradar merged pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread GitBox
bvaradar merged pull request #1674: URL: https://github.com/apache/hudi/pull/1674 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] bvaradar commented on pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread GitBox
bvaradar commented on pull request #1674: URL: https://github.com/apache/hudi/pull/1674#issuecomment-635500217 @cxzl25 : Have opened a jira https://jira.apache.org/jira/browse/HUDI-975 to add unit-test in general for non-partitioned table across all of file-system view implementations.

[jira] [Updated] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-973: Status: Open (was: New) > RemoteHoodieTableFileSystemView supports non-partitioned table

[jira] [Updated] (HUDI-975) Add unit tests in TestHoodieTableFileSystemView to test view for non-partitioned table

2020-05-28 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-975: Status: Open (was: New) > Add unit tests in TestHoodieTableFileSystemView to test view for

[jira] [Updated] (HUDI-934) Hive query does not work with realtime table which contain decimal type

2020-05-28 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-934: Status: Open (was: New) > Hive query does not work with realtime table which contain

[jira] [Created] (HUDI-975) Add unit tests in TestHoodieTableFileSystemView to test view for non-partitioned table

2020-05-28 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-975: --- Summary: Add unit tests in TestHoodieTableFileSystemView to test view for non-partitioned table Key: HUDI-975 URL: https://issues.apache.org/jira/browse/HUDI-975

[GitHub] [hudi] codecov-commenter commented on pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread GitBox
codecov-commenter commented on pull request #1674: URL: https://github.com/apache/hudi/pull/1674#issuecomment-635496074 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1674?src=pr=h1) Report > Merging [#1674](https://codecov.io/gh/apache/hudi/pull/1674?src=pr=desc) into

[GitHub] [hudi] bvaradar commented on pull request #1459: [HUDI-418] [HUDI-421] Bootstrap Index using HFile and File System View Changes with unit-test

2020-05-28 Thread GitBox
bvaradar commented on pull request #1459: URL: https://github.com/apache/hudi/pull/1459#issuecomment-635492831 @umehrot2 : Will answer to the review comments in this PR during the code-review process. This is an automated

[GitHub] [hudi] bvaradar closed pull request #1459: [HUDI-418] [HUDI-421] Bootstrap Index using HFile and File System View Changes with unit-test

2020-05-28 Thread GitBox
bvaradar closed pull request #1459: URL: https://github.com/apache/hudi/pull/1459 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] bvaradar commented on pull request #1459: [HUDI-418] [HUDI-421] Bootstrap Index using HFile and File System View Changes with unit-test

2020-05-28 Thread GitBox
bvaradar commented on pull request #1459: URL: https://github.com/apache/hudi/pull/1459#issuecomment-635492457 @umehrot2 @vinothchandar : Closing this PR as I will be opening the master PR shortly. This is an automated

[GitHub] [hudi] creactiviti commented on issue #1670: Error opening Hive split: Unknown converted type TIMESTAMP_MICROS

2020-05-28 Thread GitBox
creactiviti commented on issue #1670: URL: https://github.com/apache/hudi/issues/1670#issuecomment-635482937 Thanks @bvaradar! this is interesting. here's what I got: ``` $ java -jar ~/parquet/parquet-tools-1.8.2.jar schema /tmp/parq/out.parquet/ message spark_schema {

[GitHub] [hudi] bvaradar commented on pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-28 Thread GitBox
bvaradar commented on pull request #1674: URL: https://github.com/apache/hudi/pull/1674#issuecomment-635464681 @cxzl25 : This is good catch. Thanks for fixing it. We handle by falling back to local file-system view but this is inefficient. This shows a gap in testing non-partitioned table

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1665: [HUDI-910]Introduce HoodieWriteInput for hudi write client

2020-05-28 Thread GitBox
codecov-commenter edited a comment on pull request #1665: URL: https://github.com/apache/hudi/pull/1665#issuecomment-633592885 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1665?src=pr=h1) Report > Merging [#1665](https://codecov.io/gh/apache/hudi/pull/1665?src=pr=desc) into

[GitHub] [hudi] leesf commented on pull request #1665: [HUDI-910]Introduce HoodieWriteInput for hudi write client

2020-05-28 Thread GitBox
leesf commented on pull request #1665: URL: https://github.com/apache/hudi/pull/1665#issuecomment-635387795 @vinothchandar just a reminder on this PR. This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HUDI-974) Fields out of order in MOR mode when using Hive

2020-05-28 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-974: --- Description: When querying MOR hudi dataset via hive hive table: CREATE EXTERNAL TABLE `unknown_rt`(

[jira] [Created] (HUDI-974) Fields out of order in MOR mode when using Hive

2020-05-28 Thread leesf (Jira)
leesf created HUDI-974: -- Summary: Fields out of order in MOR mode when using Hive Key: HUDI-974 URL: https://issues.apache.org/jira/browse/HUDI-974 Project: Apache Hudi Issue Type: Bug

[hudi] branch asf-site updated: Travis CI build asf-site

2020-05-28 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 808c07f Travis CI build asf-site 808c07f is

[hudi] branch asf-site updated: [HUDI-836] Add configs for Datadog metrics reporter (#1603)

2020-05-28 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 6755231 [HUDI-836] Add configs for Datadog

[GitHub] [hudi] yanghua merged pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter

2020-05-28 Thread GitBox
yanghua merged pull request #1603: URL: https://github.com/apache/hudi/pull/1603 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] abhibhat98 opened a new issue #1676: [SUPPORT] Updating all the records per key rather than the one with higher timestamp

2020-05-28 Thread GitBox
abhibhat98 opened a new issue #1676: URL: https://github.com/apache/hudi/issues/1676 For multiple keys in the same update, only the one with higher PRECOMBINE_FIELD_OPT_KEY is considered. The default payload is OverwriteWithLatestAvroPayload which, in preCombine, picks the payload

[GitHub] [hudi] abhibhat98 opened a new issue #1675: [SUPPORT] Get all changed records from an incremental query rather than the latest one

2020-05-28 Thread GitBox
abhibhat98 opened a new issue #1675: URL: https://github.com/apache/hudi/issues/1675 **Describe the problem you faced** When I do an incremental query, I only get the latest event per key. I want to get all the events as a log. e,g at time T1, key value as K1-V1 at time T2,