[GitHub] [hudi] danny0405 commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
danny0405 commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620088913 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] yanghua commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
yanghua commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620084018 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] xglv1985 commented on issue #2812: [SUPPORT]Got a parquet related error when incremental querying MOR table, using Spark 2.4

2021-04-26 Thread GitBox
xglv1985 commented on issue #2812: URL: https://github.com/apache/hudi/issues/2812#issuecomment-826628363 > @xglv1985 Thanks, were you able to try a different spark version to see if it works for you ? Yes, I am trying Spark 3.0.0 to see if it work for me, and I will feedback once

[GitHub] [hudi] danny0405 commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
danny0405 commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620077197 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] yanghua commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
yanghua commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620060235 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[jira] [Updated] (HUDI-1848) Add support for HMS in Hive-sync-tool

2021-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1848: - Labels: pull-request-available (was: ) > Add support for HMS in Hive-sync-tool >

[GitHub] [hudi] jsbali opened a new pull request #2879: [HUDI-1848] Adding support for HMS for running DDL queries in hive-sy…

2021-04-26 Thread GitBox
jsbali opened a new pull request #2879: URL: https://github.com/apache/hudi/pull/2879 …nc-tool ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot edited a comment on pull request #2643: DO NOT MERGE (Azure CI) test branch ci

2021-04-26 Thread GitBox
hudi-bot edited a comment on pull request #2643: URL: https://github.com/apache/hudi/pull/2643#issuecomment-792368481 ## CI report: * 9831a6c50e9f49f8a71c02fc6ac50ae1446f7c1f UNKNOWN * a569dbe9409910fbb83b3764b300574c0e52612e Azure:

[jira] [Created] (HUDI-1848) Add support for HMS in Hive-sync-tool

2021-04-26 Thread Jagmeet Bali (Jira)
Jagmeet Bali created HUDI-1848: -- Summary: Add support for HMS in Hive-sync-tool Key: HUDI-1848 URL: https://issues.apache.org/jira/browse/HUDI-1848 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] n3nash commented on issue #2791: [SUPPORT]Failed to enable hoodie.metadata.enable

2021-04-26 Thread GitBox
n3nash commented on issue #2791: URL: https://github.com/apache/hudi/issues/2791#issuecomment-826577828 @njalan I think there is an issue enabling the metadata table for a non-partitioned table. @prashantwason Can you file a JIRA for this ? -- This is an automated message from the

[GitHub] [hudi] n3nash commented on issue #2802: Hive read issues when different partition have different schemas.

2021-04-26 Thread GitBox
n3nash commented on issue #2802: URL: https://github.com/apache/hudi/issues/2802#issuecomment-826577275 @aditiwari01 Any luck reproducing the issue and providing the stack trace ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] n3nash commented on issue #2806: Spark upsert Hudi performance degrades significantly

2021-04-26 Thread GitBox
n3nash commented on issue #2806: URL: https://github.com/apache/hudi/issues/2806#issuecomment-826576866 @tmac2100 Yes, that is expected. The more number of fileIds touched by your updates, the higher the runtime. Although, there could be more factors governing this. 1. Are you

[GitHub] [hudi] n3nash commented on issue #2813: [SUPPORT] HoodieRealtimeRecordReader can only work on RealtimeSplit and not with hdfs://111.parquet:0+4

2021-04-26 Thread GitBox
n3nash commented on issue #2813: URL: https://github.com/apache/hudi/issues/2813#issuecomment-826567870 @qianjiangbing Thanks for confirming. @MyLanPangzi I just noticed that you were using Hive version : 1.1 cdh 5.6.12. This is a very old version of Hive. The latest Hudi builds

[GitHub] [hudi] danny0405 commented on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
danny0405 commented on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826558257 > @danny0405 @yanghua Can one of you take a look at this ? > > @hushenmin It looks like you're running into a `NullPointerException` at `at

[GitHub] [hudi] n3nash commented on issue #2812: [SUPPORT]Got a parquet related error when incremental querying MOR table, using Spark 2.4

2021-04-26 Thread GitBox
n3nash commented on issue #2812: URL: https://github.com/apache/hudi/issues/2812#issuecomment-826568351 @xglv1985 Thanks, were you able to try a different spark version to see if it works for you ? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] n3nash commented on issue #2832: [SUPPORT] Hive on Spark dont work

2021-04-26 Thread GitBox
n3nash commented on issue #2832: URL: https://github.com/apache/hudi/issues/2832#issuecomment-826558842 @nevgin Yes, this is true. Spark 2.x still uses Hive 1.x (https://github.com/apache/spark/blob/v2.4.6/pom.xml#L129) whereas spark 3.x uses Hive 2.x

[GitHub] [hudi] n3nash commented on issue #2841: hoodie.cleaner.commits.retained max value?

2021-04-26 Thread GitBox
n3nash commented on issue #2841: URL: https://github.com/apache/hudi/issues/2841#issuecomment-826556501 @AkshayChan Please find my answers inline > 1. Is there a greatest value possible for the number of commits the hudi cleaner can retain? The num_commits is eventually

[GitHub] [hudi] n3nash commented on issue #2829: Getting an Exception Property hoodie.deltastreamer.schemaprovider.registry.baseUrl not found

2021-04-26 Thread GitBox
n3nash commented on issue #2829: URL: https://github.com/apache/hudi/issues/2829#issuecomment-826559752 @pratyakshsharma Can you take this up ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] n3nash edited a comment on issue #2841: hoodie.cleaner.commits.retained max value?

2021-04-26 Thread GitBox
n3nash edited a comment on issue #2841: URL: https://github.com/apache/hudi/issues/2841#issuecomment-826556501 @AkshayChan Please find my answers inline > 1. Is there a greatest value possible for the number of commits the hudi cleaner can retain? The num_commits is

[GitHub] [hudi] n3nash commented on issue #2830: [SUPPORT]same _hoodie_record_key has duplicates data

2021-04-26 Thread GitBox
n3nash commented on issue #2830: URL: https://github.com/apache/hudi/issues/2830#issuecomment-826559498 @wsxGit Can you help answer @nsivabalan question so we can help triage your issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] n3nash commented on issue #2810: [SUPPORT] How Hudi with Spark Streaming works?

2021-04-26 Thread GitBox
n3nash commented on issue #2810: URL: https://github.com/apache/hudi/issues/2810#issuecomment-826568664 @rubenssoto Does @pengzhiwei2018 recommendation solve your problem ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] hudi-bot edited a comment on pull request #2643: DO NOT MERGE (Azure CI) test branch ci

2021-04-26 Thread GitBox
hudi-bot edited a comment on pull request #2643: URL: https://github.com/apache/hudi/pull/2643#issuecomment-792368481 ## CI report: * 9831a6c50e9f49f8a71c02fc6ac50ae1446f7c1f UNKNOWN * a569dbe9409910fbb83b3764b300574c0e52612e Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2877: [HUDI-1844] Add option to flush when total buckets memory exceeds the…

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2877: URL: https://github.com/apache/hudi/pull/2877#issuecomment-826325691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] danny0405 commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
danny0405 commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620008316 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] danny0405 closed pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
danny0405 closed pull request #2870: URL: https://github.com/apache/hudi/pull/2870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2809: [HUDI-1789] Support reading older snapshots

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2809: URL: https://github.com/apache/hudi/pull/2809#issuecomment-826325732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] n3nash commented on issue #2848: [SUPPORT] maven | jdk.tools:jdk.tools:1.7

2021-04-26 Thread GitBox
n3nash commented on issue #2848: URL: https://github.com/apache/hudi/issues/2848#issuecomment-826542320 @raphaelauv I don't think there is any dependency of jdk.tools in Hudi repository. Looked through the POM as well as any transitive dependencies. hoodie % mvn dependency:tree >

[jira] [Updated] (HUDI-1847) Add ability to decouple configs for scheduling inline and running async

2021-04-26 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1847: -- Labels: sev:high (was: ) > Add ability to decouple configs for scheduling inline and running

[GitHub] [hudi] codecov-commenter commented on pull request #2875: [HUDI-1841] Tweak the min max commits to keep when setting up cleanin…

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2875: URL: https://github.com/apache/hudi/pull/2875#issuecomment-826288360 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2875?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] danny0405 commented on pull request #2719: [HUDI-1721] run_sync_tool support hive3

2021-04-26 Thread GitBox
danny0405 commented on pull request #2719: URL: https://github.com/apache/hudi/pull/2719#issuecomment-826205194 > @vinothchandar I test with hive2.3.8. it's not working. can we break down the one shell to hive2 or hive3 ? > > ls: cannot access

[GitHub] [hudi] hushenmin commented on issue #2648: [SUPPORT] a NPE error when reading MOR table in spark datasource

2021-04-26 Thread GitBox
hushenmin commented on issue #2648: URL: https://github.com/apache/hudi/issues/2648#issuecomment-826286695 I also meet the problem when i use spark/hive to read the mor rt table writed by flink.I think it's a critical bug. Hope you solve this problem first。 -- This is an automated

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2870: URL: https://github.com/apache/hudi/pull/2870#issuecomment-825644626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2864: [HUDI-1835] Fixing kafka native config param for auto offset reset

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2864: URL: https://github.com/apache/hudi/pull/2864#issuecomment-825222376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] yanghua commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
yanghua commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r619995900 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #2651: [HUDI-1591] [RFC-26] Implement Spark's FileIndex for Hudi to support queries via Hudi DataSource using non-globbed table pat

2021-04-26 Thread GitBox
pengzhiwei2018 commented on a change in pull request #2651: URL: https://github.com/apache/hudi/pull/2651#discussion_r619827800 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -258,4 +268,5 @@ public String

[GitHub] [hudi] RocMarshal commented on a change in pull request #2822: [Hotfix][hudi-sync] Refactor method up to parent-class

2021-04-26 Thread GitBox
RocMarshal commented on a change in pull request #2822: URL: https://github.com/apache/hudi/pull/2822#discussion_r619797349 ## File path: hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/AbstractSyncHoodieClient.java ## @@ -136,6 +141,42 @@ public

[GitHub] [hudi] hushenmin commented on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
hushenmin commented on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826547876 > @danny0405 @yanghua Can one of you take a look at this ? > > @hushenmin It looks like you're running into a `NullPointerException` at `at

[GitHub] [hudi] n3nash commented on issue #2869: [SUPPORT] java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable when I q

2021-04-26 Thread GitBox
n3nash commented on issue #2869: URL: https://github.com/apache/hudi/issues/2869#issuecomment-826537598 @putaozhi123 This is a known issue, a workaround has been mentioned here -> https://github.com/apache/hudi/issues/2544#issuecomment-815849870. Can you please try this and see if it

[jira] [Created] (HUDI-1847) Add ability to decouple configs for scheduling inline and running async

2021-04-26 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-1847: - Summary: Add ability to decouple configs for scheduling inline and running async Key: HUDI-1847 URL: https://issues.apache.org/jira/browse/HUDI-1847 Project:

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #2645: [HUDI-1659] Basic Implementation Of Spark Sql Support

2021-04-26 Thread GitBox
pengzhiwei2018 commented on a change in pull request #2645: URL: https://github.com/apache/hudi/pull/2645#discussion_r619739911 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/execution/HoodieLazyInsertIterable.java ## @@ -42,6 +43,8 @@ public

[GitHub] [hudi] leesf commented on a change in pull request #2875: [HUDI-1841] Tweak the min max commits to keep when setting up cleanin…

2021-04-26 Thread GitBox
leesf commented on a change in pull request #2875: URL: https://github.com/apache/hudi/pull/2875#discussion_r619807352 ## File path: hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java ## @@ -152,6 +154,22 @@ private static void

[GitHub] [hudi] n3nash closed issue #2855: [SUPPORT] hudi-utilities documentation

2021-04-26 Thread GitBox
n3nash closed issue #2855: URL: https://github.com/apache/hudi/issues/2855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2875: [HUDI-1841] Tweak the min max commits to keep when setting up cleanin…

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2875: URL: https://github.com/apache/hudi/pull/2875#issuecomment-826288360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] vinothchandar commented on a change in pull request #2645: [HUDI-1659] Basic Implementation Of Spark Sql Support

2021-04-26 Thread GitBox
vinothchandar commented on a change in pull request #2645: URL: https://github.com/apache/hudi/pull/2645#discussion_r619592740 ## File path: pom.xml ## @@ -112,6 +112,7 @@ 3.0.0 3 +hudi-spark2 Review comment: hudi.spark.module? is this change

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2876: [HUDI-1845] Exception Throws When Sync Non-Partitioned Table To Hive …

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2876: URL: https://github.com/apache/hudi/pull/2876#issuecomment-826325291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codecov-commenter commented on pull request #2877: [HUDI-1844] Add option to flush when total buckets memory exceeds the…

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2877: URL: https://github.com/apache/hudi/pull/2877#issuecomment-826325691 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2877?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] codecov-commenter commented on pull request #2809: [HUDI-1789] Support reading older snapshots

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2809: URL: https://github.com/apache/hudi/pull/2809#issuecomment-826325732 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2809?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] leesf commented on a change in pull request #2822: [Hotfix][hudi-sync] Refactor method up to parent-class

2021-04-26 Thread GitBox
leesf commented on a change in pull request #2822: URL: https://github.com/apache/hudi/pull/2822#discussion_r619804808 ## File path: hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java ## @@ -635,4 +635,20 @@ public void

[GitHub] [hudi] jsbali commented on a change in pull request #2809: [HUDI-1789] Support reading older snapshots

2021-04-26 Thread GitBox
jsbali commented on a change in pull request #2809: URL: https://github.com/apache/hudi/pull/2809#discussion_r619816328 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java ## @@ -438,11 +437,20 @@ public static

[GitHub] [hudi] nsivabalan merged pull request #2864: [HUDI-1835] Fixing kafka native config param for auto offset reset

2021-04-26 Thread GitBox
nsivabalan merged pull request #2864: URL: https://github.com/apache/hudi/pull/2864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-826048129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] yanghua merged pull request #2877: [HUDI-1844] Add option to flush when total buckets memory exceeds the…

2021-04-26 Thread GitBox
yanghua merged pull request #2877: URL: https://github.com/apache/hudi/pull/2877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] n3nash commented on issue #2855: [SUPPORT] hudi-utilities documentation

2021-04-26 Thread GitBox
n3nash commented on issue #2855: URL: https://github.com/apache/hudi/issues/2855#issuecomment-826538747 @raphaelauv I'm closing this ticket since you have found the documentation you are looking for. We will add more docs to read me under the filed JIRA. -- This is an automated message

[GitHub] [hudi] hudi-bot commented on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
hudi-bot commented on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826138628 ## CI report: * fb7b1ee06b79fea7215b5fa4bf8117e69ad989a1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] yanghua commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-04-26 Thread GitBox
yanghua commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-826203297 > @yanghua - I have added the unit tests, Can you please review and merge? thanks for addressing my concerns. @vinothchandar will take over this PR. -- This is an

[GitHub] [hudi] n3nash commented on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
n3nash commented on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826532502 @danny0405 @yanghua Can one of you take a look at this ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] danny0405 commented on a change in pull request #2875: [HUDI-1841] Tweak the min max commits to keep when setting up cleanin…

2021-04-26 Thread GitBox
danny0405 commented on a change in pull request #2875: URL: https://github.com/apache/hudi/pull/2875#discussion_r619816639 ## File path: hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java ## @@ -152,6 +154,22 @@ private static void

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2645: [HUDI-1659] Basic Implementation Of Spark Sql Support

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2645: URL: https://github.com/apache/hudi/pull/2645#issuecomment-822128091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] n3nash edited a comment on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
n3nash edited a comment on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826532502 @danny0405 @yanghua Can one of you take a look at this ? @hushenmin It looks like you're running into a `NullPointerException` at `at

[GitHub] [hudi] vingov commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-04-26 Thread GitBox
vingov commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-826157848 @yanghua - I have added the unit tests, Can you please review and merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] n3nash commented on issue #2878: [SUPPORT] Unable to read Hudi MOR data set in a test on 0.8

2021-04-26 Thread GitBox
n3nash commented on issue #2878: URL: https://github.com/apache/hudi/issues/2878#issuecomment-826531742 @nsivabalan Can you see if this is still an issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] jsbali commented on pull request #2809: [HUDI-1789] Support reading older snapshots

2021-04-26 Thread GitBox
jsbali commented on pull request #2809: URL: https://github.com/apache/hudi/pull/2809#issuecomment-826325078 Hey @satishkotha Thanks for the review. I have made the following changes 1. Got rid of the new config and reused the old consume.commit one. 2. Added javadocs and

[GitHub] [hudi] codecov-commenter commented on pull request #2876: [HUDI-1845] Exception Throws When Sync Non-Partitioned Table To Hive …

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2876: URL: https://github.com/apache/hudi/pull/2876#issuecomment-826325291 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2876?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
hudi-bot edited a comment on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826138628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2822: [Hotfix][hudi-sync] Refactor method up to parent-class

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2822: URL: https://github.com/apache/hudi/pull/2822#issuecomment-821787385 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] xushiyan commented on a change in pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
xushiyan commented on a change in pull request #2873: URL: https://github.com/apache/hudi/pull/2873#discussion_r619698467 ## File path: hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeRecordReader.java ## @@ -237,13 +231,13 @@ public void

[GitHub] [hudi] xushiyan removed a comment on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
xushiyan removed a comment on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826155635 @hudi-bot run travis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] codecov-commenter commented on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826139460 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2873?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #2643: DO NOT MERGE (Azure CI) test branch ci

2021-04-26 Thread GitBox
hudi-bot edited a comment on pull request #2643: URL: https://github.com/apache/hudi/pull/2643#issuecomment-792368481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826139460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codecov-commenter commented on pull request #2747: [HUDI-1743] Added support for SqlFileBasedTransformer

2021-04-26 Thread GitBox
codecov-commenter commented on pull request #2747: URL: https://github.com/apache/hudi/pull/2747#issuecomment-826048129 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2747?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] xushiyan commented on pull request #2873: [HUDI-1811] Fix TestHoodieRealtimeRecordReader (Azure CI)

2021-04-26 Thread GitBox
xushiyan commented on pull request #2873: URL: https://github.com/apache/hudi/pull/2873#issuecomment-826155635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [hudi] hushenmin commented on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
hushenmin commented on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826547876 > @danny0405 @yanghua Can one of you take a look at this ? > > @hushenmin It looks like you're running into a `NullPointerException` at `at

[GitHub] [hudi] danny0405 commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
danny0405 commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r620008316 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] n3nash commented on issue #2848: [SUPPORT] maven | jdk.tools:jdk.tools:1.7

2021-04-26 Thread GitBox
n3nash commented on issue #2848: URL: https://github.com/apache/hudi/issues/2848#issuecomment-826542320 @raphaelauv I don't think there is any dependency of jdk.tools in Hudi repository. Looked through the POM as well as any transitive dependencies. hoodie % mvn dependency:tree >

[GitHub] [hudi] n3nash closed issue #2855: [SUPPORT] hudi-utilities documentation

2021-04-26 Thread GitBox
n3nash closed issue #2855: URL: https://github.com/apache/hudi/issues/2855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] n3nash commented on issue #2855: [SUPPORT] hudi-utilities documentation

2021-04-26 Thread GitBox
n3nash commented on issue #2855: URL: https://github.com/apache/hudi/issues/2855#issuecomment-826538747 @raphaelauv I'm closing this ticket since you have found the documentation you are looking for. We will add more docs to read me under the filed JIRA. -- This is an automated message

[GitHub] [hudi] n3nash commented on issue #2869: [SUPPORT] java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable when I q

2021-04-26 Thread GitBox
n3nash commented on issue #2869: URL: https://github.com/apache/hudi/issues/2869#issuecomment-826537598 @putaozhi123 This is a known issue, a workaround has been mentioned here -> https://github.com/apache/hudi/issues/2544#issuecomment-815849870. Can you please try this and see if it

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2875: [HUDI-1841] Tweak the min max commits to keep when setting up cleanin…

2021-04-26 Thread GitBox
codecov-commenter edited a comment on pull request #2875: URL: https://github.com/apache/hudi/pull/2875#issuecomment-826288360 #

[GitHub] [hudi] n3nash edited a comment on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
n3nash edited a comment on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826532502 @danny0405 @yanghua Can one of you take a look at this ? @hushenmin It looks like you're running into a `NullPointerException` at `at

[GitHub] [hudi] n3nash commented on issue #2874: [SUPPORT] Can not query mor `_rt` table by FlinkSQL Writing .

2021-04-26 Thread GitBox
n3nash commented on issue #2874: URL: https://github.com/apache/hudi/issues/2874#issuecomment-826532502 @danny0405 @yanghua Can one of you take a look at this ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] yanghua commented on a change in pull request #2870: [HUDI-1837] Add optional instant range to log record scanner for log block filtering

2021-04-26 Thread GitBox
yanghua commented on a change in pull request #2870: URL: https://github.com/apache/hudi/pull/2870#discussion_r619995900 ## File path: hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java ## @@ -279,6 +279,12 @@ private FlinkOptions() {

[GitHub] [hudi] n3nash commented on issue #2878: [SUPPORT] Unable to read Hudi MOR data set in a test on 0.8

2021-04-26 Thread GitBox
n3nash commented on issue #2878: URL: https://github.com/apache/hudi/issues/2878#issuecomment-826531742 @nsivabalan Can you see if this is still an issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

<    1   2