[GitHub] [hudi] xushiyan commented on pull request #1818: [HUDI-996] Move TestHBaseIndex to functional test suite

xushiyan commented on pull request #1818: URL: https://github.com/apache/hudi/pull/1818#issuecomment-657181391 @vinothchandar this is ready for review. Thanks. This is an automated message from the Apache Git Service. To

[GitHub] [hudi] zherenyu831 commented on issue #1798: Question reading partition path with less level is more faster than what document mentioned

zherenyu831 commented on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657193426 @umehrot2 @vinothchandar Sorry for lately reply. Here is my snapshot of spark ui. First query I used, files processed by resolveRelation was 950 ```

[GitHub] [hudi] zherenyu831 edited a comment on issue #1798: Question reading partition path with less level is more faster than what document mentioned

zherenyu831 edited a comment on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657193426 @umehrot2 @vinothchandar Sorry for lately reply. Here is my snapshot of spark ui. First query I used, files processed by resolveRelation was 950 ```

[GitHub] [hudi] zherenyu831 edited a comment on issue #1798: Question reading partition path with less level is more faster than what document mentioned

zherenyu831 edited a comment on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657193426 @umehrot2 @vinothchandar Thank you guys. and sorry for lately reply. Here is my snapshot of spark ui. First query I used, files processed by resolveRelation

[GitHub] [hudi] zherenyu831 edited a comment on issue #1798: Question reading partition path with less level is more faster than what document mentioned

zherenyu831 edited a comment on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657193426 @umehrot2 @vinothchandar Thank you guys. and sorry for lately reply. Here is my snapshot of spark ui. First query I used, files processed by resolveRelation

[GitHub] [hudi] RajasekarSribalan commented on issue #1823: [SUPPORT] MOR trigger compaction from Hudi CLI

RajasekarSribalan commented on issue #1823: URL: https://github.com/apache/hudi/issues/1823#issuecomment-657201567 Another issue is, I am getting below error during inline compaction. Pls help. com.esotericsoftware.kryo.KryoException: Unable to find class:

[jira] [Resolved] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1004. - Fix Version/s: 0.6.0 Resolution: Fixed > Support update metrics in HoodieDeltaStreamer >

[jira] [Updated] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1004: Status: Open (was: New) > Support update metrics in HoodieDeltaStreamer >

[jira] [Closed] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1004. --- > Support update metrics in HoodieDeltaStreamer > - > > Key:

[jira] [Updated] (HUDI-1062) Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

[ https://issues.apache.org/jira/browse/HUDI-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1062: Status: Open (was: New) > Remove unnecessary maxEvent check and add some log in KafkaOffsetGen >

[jira] [Closed] (HUDI-1062) Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

[ https://issues.apache.org/jira/browse/HUDI-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1062. --- > Remove unnecessary maxEvent check and add some log in KafkaOffsetGen >

[jira] [Resolved] (HUDI-1062) Remove unnecessary maxEvent check and add some log in KafkaOffsetGen

[ https://issues.apache.org/jira/browse/HUDI-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1062. - Fix Version/s: 0.6.0 Resolution: Fixed > Remove unnecessary maxEvent check and add some log in

[GitHub] [hudi] shenh062326 commented on a change in pull request #1819: [HUDI-1058] Make delete marker configurable

shenh062326 commented on a change in pull request #1819: URL: https://github.com/apache/hudi/pull/1819#discussion_r453293178 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java ## @@ -67,7 +74,8 @@ public

[jira] [Updated] (HUDI-1064) hoodie table name value trim （ value not trim，the code is not robust）

[ https://issues.apache.org/jira/browse/HUDI-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1064: Status: Open (was: New) > hoodie table name value trim （ value not trim，the code is not robust） >

[jira] [Resolved] (HUDI-1064) hoodie table name value trim （ value not trim，the code is not robust）

[ https://issues.apache.org/jira/browse/HUDI-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1064. - Fix Version/s: 0.6.0 Resolution: Fixed > hoodie table name value trim （ value not trim，the code is

[jira] [Closed] (HUDI-1064) hoodie table name value trim （ value not trim，the code is not robust）

[ https://issues.apache.org/jira/browse/HUDI-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1064. --- > hoodie table name value trim （ value not trim，the code is not robust） >

[jira] [Closed] (HUDI-1080) Fix backward compatiblity for com.uber input formats

[ https://issues.apache.org/jira/browse/HUDI-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1080. --- > Fix backward compatiblity for com.uber input formats > > >

[jira] [Commented] (HUDI-1082) Bug in deciding the upsert/insert buckets

[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156259#comment-17156259 ] Hong Shen commented on HUDI-1082: - I can work on this if you have not start. > Bug in deciding the

[GitHub] [hudi] bhasudha commented on issue #1823: [SUPPORT] MOR trigger compaction from Hudi CLI

bhasudha commented on issue #1823: URL: https://github.com/apache/hudi/issues/1823#issuecomment-657297717 @RajasekarSribalan For your first question, unfortunately currently in Spark Streaming writes only support inline compaction is supported. So you have to enable that config. Good

[GitHub] [hudi] bhasudha commented on issue #1800: [SUPPORT] finalize errors "at org.apache.hudi.table.HoodieTable.cleanFailedWrites"

bhasudha commented on issue #1800: URL: https://github.com/apache/hudi/issues/1800#issuecomment-657299606 IIUC you are enabling the consistency check config and still seeing the above issue ? @bvaradar could you help ?

[GitHub] [hudi] bhasudha commented on issue #1787: Exception During Insert

bhasudha commented on issue #1787: URL: https://github.com/apache/hudi/issues/1787#issuecomment-657289513 @asheeshgarg which version of Presto are you using ? Also, for querying through Presto, you don't have to write anything. As long as the table is registered in Hive, you can simply

[GitHub] [hudi] bhasudha commented on issue #1813: ERROR HoodieDeltaStreamer: Got error running delta sync once.

bhasudha commented on issue #1813: URL: https://github.com/apache/hudi/issues/1813#issuecomment-657295902 @jcunhafonte Could you try using the DeltaStreamer in continuous mode rather than using the scheduled job ? I think what's happening is the schema provider is [initialized for the

[GitHub] [hudi] bhasudha commented on issue #1777: [SUPPORT] org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were

bhasudha commented on issue #1777: URL: https://github.com/apache/hudi/issues/1777#issuecomment-657285023 @WaterKnight1998 Were you able to resolve ? This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] bhasudha commented on issue #1790: [SUPPORT] Querying MoR tables with DecimalType columns via Spark SQL fails

bhasudha commented on issue #1790: URL: https://github.com/apache/hudi/issues/1790#issuecomment-657296109 @garyli1019 could you help with this question ? This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] vinothchandar commented on a change in pull request #1818: [HUDI-996] Move TestHBaseIndex to functional test suite

vinothchandar commented on a change in pull request #1818: URL: https://github.com/apache/hudi/pull/1818#discussion_r453391187 ## File path: hudi-client/src/test/java/org/apache/hudi/testutils/providers/DFSProvider.java ## @@ -31,4 +33,6 @@ Path dfsBasePath(); + Path

[GitHub] [hudi] codecov-commenter commented on pull request #1774: [HUDI-703]Add unit test for HoodieSyncCommand

codecov-commenter commented on pull request #1774: URL: https://github.com/apache/hudi/pull/1774#issuecomment-657314447 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1774?src=pr=h1) Report > Merging [#1774](https://codecov.io/gh/apache/hudi/pull/1774?src=pr=desc) into

[GitHub] [hudi] RajasekarSribalan commented on issue #1823: [SUPPORT] MOR trigger compaction from Hudi CLI

RajasekarSribalan commented on issue #1823: URL: https://github.com/apache/hudi/issues/1823#issuecomment-657337822 Thank you for your response Bhavani. 1.May I know the purpose of compaction schedule and compaction run command from Hudi CLI? 2. If inline compaction is only

[GitHub] [hudi] vinothchandar closed issue #1546: Issue - Table Read fails in Spark Submit , Where as succeeds in spark-shell

vinothchandar closed issue #1546: URL: https://github.com/apache/hudi/issues/1546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] vinothchandar commented on issue #1798: Question reading partition path with less level is more faster than what document mentioned

vinothchandar commented on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657313523 @zherenyu831 this seems like an issue with the contents of `.aux` as well listed additionally.. than anything to do with the actual reading of data.. cc @bvaradar to confirm if

[GitHub] [hudi] hddong commented on pull request #1774: [HUDI-703]Add unit test for HoodieSyncCommand

hddong commented on pull request #1774: URL: https://github.com/apache/hudi/pull/1774#issuecomment-657322002 @yanghua please have a review when free. This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] garyli1019 commented on issue #1790: [SUPPORT] Querying MoR tables with DecimalType columns via Spark SQL fails

garyli1019 commented on issue #1790: URL: https://github.com/apache/hudi/issues/1790#issuecomment-657332648 Hi @zuyanton , thanks for report this issue. https://github.com/apache/hudi/commit/37838cea6094ddc66191df42e8b2c84f132d1623#diff-68b6e6f1a2c961fea254a2fc3b93ac23R213 is a

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #337

See Changes: -- [...truncated 2.30 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[GitHub] [hudi] RajasekarSribalan edited a comment on issue #1823: [SUPPORT] MOR trigger compaction from Hudi CLI

RajasekarSribalan edited a comment on issue #1823: URL: https://github.com/apache/hudi/issues/1823#issuecomment-657337822 Thank you for your response Bhavani. @bhasudha 1.May I know the purpose of compaction schedule and compaction run command from Hudi CLI? 2. If inline

[GitHub] [hudi] xushiyan commented on pull request #1818: [HUDI-996] Move TestHBaseIndex to functional test suite

xushiyan commented on pull request #1818: URL: https://github.com/apache/hudi/pull/1818#issuecomment-657361824 @yanghua @vinothchandar Thanks for the review. I can split the changes into 2 PRs 1. (this one) Improve TestHBaseIndex (refactoring, move test cases to unit tests) 2.

[GitHub] [hudi] xushiyan commented on a change in pull request #1819: [HUDI-1058] Make delete marker configurable

xushiyan commented on a change in pull request #1819: URL: https://github.com/apache/hudi/pull/1819#discussion_r453449647 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithLatestAvroPayload.java ## @@ -67,7 +74,8 @@ public