[GitHub] [hudi] liujinhui1994 commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
liujinhui1994 commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710769505 ## File path: hudi-client/hudi-client-common/src/test/java/org/apache/hudi/metrics/TestHoodieGraphiteMetrics.java ## @@ -0,0 +1,60 @@ +/* + *

[GitHub] [hudi] yanghua commented on pull request #3671: [Hudi-2418] add HiveSchemaProvider

2021-09-16 Thread GitBox
yanghua commented on pull request #3671: URL: https://github.com/apache/hudi/pull/3671#issuecomment-921517144 @fengjian428 thanks for your contribution. Please check the CI failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] yanghua commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
yanghua commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710767279 ## File path: hudi-client/hudi-client-common/src/test/java/org/apache/hudi/metrics/TestHoodieGraphiteMetrics.java ## @@ -0,0 +1,60 @@ +/* + * Licensed to

[GitHub] [hudi] yanghua commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
yanghua commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710767279 ## File path: hudi-client/hudi-client-common/src/test/java/org/apache/hudi/metrics/TestHoodieGraphiteMetrics.java ## @@ -0,0 +1,60 @@ +/* + * Licensed to

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-16 Thread GitBox
xiarixiaoyao edited a comment on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-921467046 @codope thanks for your review. Let me answer some your questions first,pls forgive me for being busy today, I need to modify something for the pr of zorder

[GitHub] [hudi] xiarixiaoyao commented on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-16 Thread GitBox
xiarixiaoyao commented on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-921467046 @codope thanks for your review. Let me answer some your questions first, I’m a bit busy today, I need to modify something for the pr of zorder **Let's add docs for all

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-09-16 Thread GitBox
xiarixiaoyao commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r710667271 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/RDDSpatialCurveOptimizationSortPartitioner.java ## @@

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-09-16 Thread GitBox
xiarixiaoyao commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r710666486 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieOptimizeConfig.java ## @@ -0,0 +1,134 @@ +/* + * Licensed to

[hudi] branch master updated (57d5da6 -> 8a65217)

2021-09-16 Thread wangxianghu
This is an automated email from the ASF dual-hosted git repository. wangxianghu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 57d5da6 [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables (#3679) add 8a65217 [MINOR] Fix

[GitHub] [hudi] wangxianghu merged pull request #3675: [MINOR] Fix typo,'compatiblity' corrected to 'compatibility'

2021-09-16 Thread GitBox
wangxianghu merged pull request #3675: URL: https://github.com/apache/hudi/pull/3675 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HUDI-2402) Hive Sync supports Kerberos authentication

2021-09-16 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2402: Fix Version/s: (was: 0.9.0) 0.10.0 > Hive Sync supports Kerberos

[jira] [Resolved] (HUDI-2335) Implement insertions to logs for Kafka connect

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar resolved HUDI-2335. -- Fix Version/s: 0.10.0 Resolution: Fixed > Implement insertions to logs for Kafka connect

[jira] [Reopened] (HUDI-2330) Implement MOR support for Java Write Client: Insert only to logs

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reopened HUDI-2330: -- > Implement MOR support for Java Write Client: Insert only to logs >

[jira] [Resolved] (HUDI-2330) Implement MOR support for Java Write Client: Insert only to logs

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar resolved HUDI-2330. -- Fix Version/s: 0.10.0 Resolution: Fixed > Implement MOR support for Java Write Client:

[jira] [Updated] (HUDI-2330) Implement MOR support for Java Write Client: Insert only to logs

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2330: - Status: Closed (was: Patch Available) > Implement MOR support for Java Write Client: Insert only

[jira] [Updated] (HUDI-2330) Implement MOR support for Java Write Client: Insert only to logs

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2330: - Status: Patch Available (was: In Progress) > Implement MOR support for Java Write Client: Insert

[GitHub] [hudi] vinothchandar commented on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
vinothchandar commented on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921317594 CI is failing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[hudi] branch master updated: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables (#3679)

2021-09-16 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 57d5da6 [HUDI-2330][HUDI-2335] Adding support

[GitHub] [hudi] vinothchandar merged pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
vinothchandar merged pull request #3679: URL: https://github.com/apache/hudi/pull/3679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] rmahindra123 commented on pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
rmahindra123 commented on pull request #3679: URL: https://github.com/apache/hudi/pull/3679#issuecomment-921300090 > @rmahindra123 can you give me a quick review of this? lgtm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] hudi-bot edited a comment on pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3679: URL: https://github.com/apache/hudi/pull/3679#issuecomment-921265173 ## CI report: * 95f2f0d2ed51d04084be38c3b2c15785b76f503d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * 478a4a1774e5143608079bb60e1fdbca8c574aa9 Azure:

[jira] [Created] (HUDI-2446) Test edge cases around correctness and offset management and certify

2021-09-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-2446: Summary: Test edge cases around correctness and offset management and certify Key: HUDI-2446 URL: https://issues.apache.org/jira/browse/HUDI-2446 Project: Apache

[jira] [Resolved] (HUDI-2378) Add common and pre validate configs to website

2021-09-16 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra resolved HUDI-2378. --- Fix Version/s: 0.10.0 Resolution: Fixed > Add common and pre validate configs to

[jira] [Updated] (HUDI-2333) Implement schema provider and registry for Kafka connect

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2333: - Fix Version/s: 0.10.0 > Implement schema provider and registry for Kafka connect >

[jira] [Resolved] (HUDI-2333) Implement schema provider and registry for Kafka connect

2021-09-16 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra resolved HUDI-2333. --- Resolution: Fixed > Implement schema provider and registry for Kafka connect >

[jira] [Updated] (HUDI-2325) Implement and test Hive Sync support for Kafka Connect

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2325: - Status: Patch Available (was: In Progress) > Implement and test Hive Sync support for Kafka

[jira] [Updated] (HUDI-2325) Implement and test Hive Sync support for Kafka Connect

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2325: - Status: In Progress (was: Open) > Implement and test Hive Sync support for Kafka Connect >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3679: URL: https://github.com/apache/hudi/pull/3679#issuecomment-921265173 ## CI report: * 95f2f0d2ed51d04084be38c3b2c15785b76f503d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
hudi-bot commented on pull request #3679: URL: https://github.com/apache/hudi/pull/3679#issuecomment-921265173 ## CI report: * 95f2f0d2ed51d04084be38c3b2c15785b76f503d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] vinothchandar commented on pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
vinothchandar commented on pull request #3679: URL: https://github.com/apache/hudi/pull/3679#issuecomment-921259161 @rmahindra123 can you give me a quick review of this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Updated] (HUDI-1071) Upgrade dropwizard metrics to make use of SettableGauge

2021-09-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1071: - Labels: newbie (was: ) > Upgrade dropwizard metrics to make use of SettableGauge >

[jira] [Updated] (HUDI-2330) Implement MOR support for Java Write Client: Insert only to logs

2021-09-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2330: - Labels: pull-request-available (was: ) > Implement MOR support for Java Write Client: Insert

[GitHub] [hudi] vinothchandar opened a new pull request #3679: [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables in Kafka Connect Sink

2021-09-16 Thread GitBox
vinothchandar opened a new pull request #3679: URL: https://github.com/apache/hudi/pull/3679 - Inserts go into logs, hashed by Kafka and Hudi partitions - Fixed issues with the setupKafka script - Bumped up the default commit interval to 300 seconds - Minor renaming ##

[jira] [Created] (HUDI-2445) Implement time based bucketing of inserts

2021-09-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-2445: Summary: Implement time based bucketing of inserts Key: HUDI-2445 URL: https://issues.apache.org/jira/browse/HUDI-2445 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-904) Segregate metrics configs by reporter type

2021-09-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-904: Labels: newbie (was: ) > Segregate metrics configs by reporter type >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * b1eef40fd1ea04c059f75ce5f34b8f637ef73f5f Azure:

[GitHub] [hudi] xushiyan commented on issue #3581: [SUPPORT] Slow snapshot query performance

2021-09-16 Thread GitBox
xushiyan commented on issue #3581: URL: https://github.com/apache/hudi/issues/3581#issuecomment-921230495 @codejoyan for reading can you also try enabling the metadata config too? like `.option("hoodie.metadata.enable", "true")`. certainly we need to make the docs better.. -- This is

[GitHub] [hudi] hudi-bot edited a comment on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * b1eef40fd1ea04c059f75ce5f34b8f637ef73f5f Azure:

[jira] [Updated] (HUDI-2432) Fix restore by adding a requested instant and restore plan

2021-09-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2432: -- Description: Fix restore by adding a requested instant and restore plan   Trying to

[GitHub] [hudi] hudi-bot edited a comment on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * b1eef40fd1ea04c059f75ce5f34b8f637ef73f5f Azure:

[jira] [Updated] (HUDI-2432) Fix restore by adding a requested instant and restore plan

2021-09-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2432: -- Description: Fix restore by adding a requested instant and restore plan   Trying to

[GitHub] [hudi] xushiyan commented on pull request #3659: [MINOR] Use fixed JDK version for CI, with applicable OpenJDK distro

2021-09-16 Thread GitBox
xushiyan commented on pull request #3659: URL: https://github.com/apache/hudi/pull/3659#issuecomment-921206253 @geertjanw no worries. mind fixing it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] nsivabalan commented on a change in pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
nsivabalan commented on a change in pull request #3678: URL: https://github.com/apache/hudi/pull/3678#discussion_r710436094 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/BaseRollbackHelper.java ## @@ -126,6 +127,8 @@ public

[GitHub] [hudi] hudi-bot edited a comment on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * b1eef40fd1ea04c059f75ce5f34b8f637ef73f5f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
hudi-bot commented on pull request #3678: URL: https://github.com/apache/hudi/pull/3678#issuecomment-921203076 ## CI report: * b1eef40fd1ea04c059f75ce5f34b8f637ef73f5f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2444) Fix missing files as part of clean metadata or rollback metadata if retried after failed

2021-09-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2444: - Labels: pull-request-available (was: ) > Fix missing files as part of clean metadata or rollback

[GitHub] [hudi] nsivabalan opened a new pull request #3678: [HUDI-2444] Fixing cleaning and rollback if retried after failed attempt

2021-09-16 Thread GitBox
nsivabalan opened a new pull request #3678: URL: https://github.com/apache/hudi/pull/3678 ## What is the purpose of the pull request - If cleaning or rollback failed on first attempt and then we re-attempt, commit metadata could miss files that were deleted during 1st attempt. We

[jira] [Updated] (HUDI-2444) Fix missing files as part of clean metadata or rollback metadata if retried after failed

2021-09-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2444: -- Parent: HUDI-1292 Issue Type: Sub-task (was: Improvement) > Fix missing files

[jira] [Created] (HUDI-2444) Fix missing files as part of clean metadata or rollback metadata if retried after failed

2021-09-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2444: - Summary: Fix missing files as part of clean metadata or rollback metadata if retried after failed Key: HUDI-2444 URL: https://issues.apache.org/jira/browse/HUDI-2444

[jira] [Updated] (HUDI-2432) Fix restore by adding a requested instant and restore plan

2021-09-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2432: -- Description: Fix restore by adding a requested instant and restore plan   Trying to

[GitHub] [hudi] zhedoubushishi commented on issue #3095: [SUPPORT] Not able to submit the spark-submit code into AWS EMR step execution

2021-09-16 Thread GitBox
zhedoubushishi commented on issue #3095: URL: https://github.com/apache/hudi/issues/3095#issuecomment-921114745 Sorry for the late reply. I also think this is not a Hudi specific issue, it's more like an EMR implementation issue. You can just open a customer support ticket to EMR. --

[jira] [Updated] (HUDI-2443) KVComparator in HFile for metadata table is tied to HBase version and shading

2021-09-16 Thread Ryan Pifer (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Pifer updated HUDI-2443: - Summary: KVComparator in HFile for metadata table is tied to HBase version and shading (was:

[jira] [Created] (HUDI-2443) KVComparator in HFile for metadata table is tied to HBase version

2021-09-16 Thread Ryan Pifer (Jira)
Ryan Pifer created HUDI-2443: Summary: KVComparator in HFile for metadata table is tied to HBase version Key: HUDI-2443 URL: https://issues.apache.org/jira/browse/HUDI-2443 Project: Apache Hudi

[jira] [Created] (HUDI-2442) Simplify out-of-box clustering configs

2021-09-16 Thread Vinoth Chandar (Jira)
Vinoth Chandar created HUDI-2442: Summary: Simplify out-of-box clustering configs Key: HUDI-2442 URL: https://issues.apache.org/jira/browse/HUDI-2442 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-1042) [Umbrella] Support clustering on filegroups

2021-09-16 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1042: - Status: Open (was: New) > [Umbrella] Support clustering on filegroups >

[GitHub] [hudi] stym06 commented on issue #2680: [SUPPORT]Hive sync error by using run_sync_tool.sh

2021-09-16 Thread GitBox
stym06 commented on issue #2680: URL: https://github.com/apache/hudi/issues/2680#issuecomment-921095856 @moranyuwen I was facing this issue and it got fixed when I ran it after turning on the hiveserver2 in the pod. My hunch is that it needs the hive-site.xml to know that it has to use

[GitHub] [hudi] stym06 opened a new issue #3677: DeltaStreamer exits with error while writing to Azure Blob storage[SUPPORT]

2021-09-16 Thread GitBox
stym06 opened a new issue #3677: URL: https://github.com/apache/hudi/issues/3677 **Describe the problem you faced** I'm running the DeltaStreamer application in continuous mode writing data into Azure blob storage. It runs fine for a few days, but exits with the following

[GitHub] [hudi] FelixKJose opened a new issue #3676: MOR table rolls out new parquet files at 10MB for new inserts - even though max file size set as 128MB

2021-09-16 Thread GitBox
FelixKJose opened a new issue #3676: URL: https://github.com/apache/hudi/issues/3676 I have been trying to set up data in HUDI MOR table for my consumption load test, but come across an interesting behavior. I have configured the write operation as 'upsert', even though all my writes are

[hudi] branch master updated (4deaa30 -> b8dad62)

2021-09-16 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 4deaa30 [HUDI-2404] Add metrics-jmx to spark and flink bundles (#3632) add b8dad62 [HUDI-2422] Adding

[GitHub] [hudi] nsivabalan merged pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
nsivabalan merged pull request #3651: URL: https://github.com/apache/hudi/pull/3651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * de4e738ce07f76e2cd0a5f2adc73e7d9093aed57 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * 9985d101215e2d5bf539b2f3bcac1e9b13982f91 Azure:

[hudi] branch master updated (2791fb9 -> 4deaa30)

2021-09-16 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 2791fb9 [HUDI-2423] Separate some config logic from HoodieMetricsConfig into HoodieMetricsGraphiteConfig

[GitHub] [hudi] nsivabalan merged pull request #3632: [HUDI-2404] Add metrics-jmx to spark and flink bundles

2021-09-16 Thread GitBox
nsivabalan merged pull request #3632: URL: https://github.com/apache/hudi/pull/3632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * 9985d101215e2d5bf539b2f3bcac1e9b13982f91 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-919855741 ## CI report: * 1a0b7d831b32d1704c1837a2df77bfe4d36724cc Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * 7252e8168b38036ab012f08cdcfac89976bf219e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * 7252e8168b38036ab012f08cdcfac89976bf219e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3667: URL: https://github.com/apache/hudi/pull/3667#issuecomment-919813603 ## CI report: * 13611371aa425cf2acf4fee39dd78265d47d4703 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3651: [HUDI-2422] Adding rollback plan and rollback requested instant

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3651: URL: https://github.com/apache/hudi/pull/3651#issuecomment-917815065 ## CI report: * 7252e8168b38036ab012f08cdcfac89976bf219e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3675: [MINOR] Fix typo,'compatiblity' corrected to 'compatibility'

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3675: URL: https://github.com/apache/hudi/pull/3675#issuecomment-920845738 ## CI report: * f3c5e11727731de42256eeb54efd910d7be637eb Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-919855741 ## CI report: * ba407cc5af64d50eaa18f5df4adf1307853c264d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-919855741 ## CI report: * ba407cc5af64d50eaa18f5df4adf1307853c264d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3667: URL: https://github.com/apache/hudi/pull/3667#issuecomment-919813603 ## CI report: * 53fea88f4c72ba3ec3c855f14bf244dbcaf0afc6 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3667: URL: https://github.com/apache/hudi/pull/3667#issuecomment-919813603 ## CI report: * 53fea88f4c72ba3ec3c855f14bf244dbcaf0afc6 Azure:

[GitHub] [hudi] liujinhui1994 commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
liujinhui1994 commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710060764 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java ## @@ -1475,6 +1475,10 @@ public String

[GitHub] [hudi] liujinhui1994 commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
liujinhui1994 commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710060531 ## File path: hudi-client/hudi-client-common/src/test/java/org/apache/hudi/metrics/TestHoodieGraphiteMetrics.java ## @@ -0,0 +1,60 @@ +/* + *

[GitHub] [hudi] hudi-bot edited a comment on pull request #3675: [MINOR] Fix typo,'compatiblity' corrected to 'compatibility'

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3675: URL: https://github.com/apache/hudi/pull/3675#issuecomment-920845738 ## CI report: * f3c5e11727731de42256eeb54efd910d7be637eb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3675: [MINOR] Fix typo,'compatiblity' corrected to 'compatibility'

2021-09-16 Thread GitBox
hudi-bot commented on pull request #3675: URL: https://github.com/apache/hudi/pull/3675#issuecomment-920845738 ## CI report: * f3c5e11727731de42256eeb54efd910d7be637eb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] dongkelun opened a new pull request #3675: [MINOR] Fix typo,'compatiblity' corrected to 'compatibility'

2021-09-16 Thread GitBox
dongkelun opened a new pull request #3675: URL: https://github.com/apache/hudi/pull/3675 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] yanghua commented on a change in pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
yanghua commented on a change in pull request #3667: URL: https://github.com/apache/hudi/pull/3667#discussion_r710042087 ## File path: hudi-client/hudi-client-common/src/test/java/org/apache/hudi/metrics/TestHoodieGraphiteMetrics.java ## @@ -0,0 +1,60 @@ +/* + * Licensed to

[jira] [Updated] (HUDI-2441) To support partial update function which can move and update the data from the old partition to the new partition , when the data with same key change it's partition

2021-09-16 Thread David_Liang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David_Liang updated HUDI-2441: -- Description: to considerate such a scene, there 2 reocod  as follow in the source table ||post_id 

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * afb1012c9ddd8e01f2a41a22c692b8eb78fe7973 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-914114290 ## CI report: * 4fb2b5a2be46cdec8ebf0a186c7bc3a97e78f92f Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3667: URL: https://github.com/apache/hudi/pull/3667#issuecomment-919813603 ## CI report: * 53fea88f4c72ba3ec3c855f14bf244dbcaf0afc6 Azure:

[jira] [Issue Comment Deleted] (HUDI-2441) To support partial update function which can move and update the data from the old partition to the new partition , when the data with same key change it's p

2021-09-16 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang updated HUDI-2441: - Comment: was deleted (was: [~yanghua], I have interest in working for this issue. Could you

[jira] [Assigned] (HUDI-2441) To support partial update function which can move and update the data from the old partition to the new partition , when the data with same key change it's partition

2021-09-16 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang reassigned HUDI-2441: Assignee: Nicholas Jiang > To support partial update function which can move and update

[GitHub] [hudi] deep-teliacompany edited a comment on issue #3132: [SUPPORT]- HUDI table returns no records when queried using Hive with partition columns in where clause

2021-09-16 Thread GitBox
deep-teliacompany edited a comment on issue #3132: URL: https://github.com/apache/hudi/issues/3132#issuecomment-920748080 Hi, I am not using hive sync tool, will try and and will share the observations. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot edited a comment on pull request #3674: [WIP][HUDI-2440] Add dependency change diff script for dependency governace

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3674: URL: https://github.com/apache/hudi/pull/3674#issuecomment-920690239 ## CI report: * 61baa2fb144ba3521441b03c51b12da9ff9b11bf Azure:

[GitHub] [hudi] deep-teliacompany commented on issue #3132: [SUPPORT]- HUDI table returns no records when queried using Hive with partition columns in where clause

2021-09-16 Thread GitBox
deep-teliacompany commented on issue #3132: URL: https://github.com/apache/hudi/issues/3132#issuecomment-920748080 Testing this and will revert once have my findings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] hudi-bot edited a comment on pull request #3667: [HUDI-2434] Add GraphiteReporter reporter periodSeconds config

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3667: URL: https://github.com/apache/hudi/pull/3667#issuecomment-919813603 ## CI report: * 10f05d42a895e0c1083bb0ce3f7102f59416a012 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-914114290 ## CI report: * 669716935d27d9a7263b6e32882e246b36b100ca Azure:

[jira] [Commented] (HUDI-2441) To support partial update function which can move and update the data from the old partition to the new partition , when the data with same key change it's partition

2021-09-16 Thread Nicholas Jiang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416003#comment-17416003 ] Nicholas Jiang commented on HUDI-2441: -- [~yanghua], I have interest in working for this issue. Could

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * afb1012c9ddd8e01f2a41a22c692b8eb78fe7973 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3614: [HUDI-2370] Supports data encryption

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3614: URL: https://github.com/apache/hudi/pull/3614#issuecomment-914114290 ## CI report: * 669716935d27d9a7263b6e32882e246b36b100ca Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * 71b58bec7d73e445fbf1c4486aab7013226902ba Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * 71b58bec7d73e445fbf1c4486aab7013226902ba Azure:

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
zhangyue19921010 commented on a change in pull request #3623: URL: https://github.com/apache/hudi/pull/3623#discussion_r709894663 ## File path: packaging/hudi-presto-bundle/pom.xml ## @@ -158,11 +158,66 @@ org.apache.hudi hudi-common ${project.version} +

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-16 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * 71b58bec7d73e445fbf1c4486aab7013226902ba Azure:

  1   2   >