[jira] [Updated] (HUDI-5058) The primary key cannot be empty when Flink reads an error from the hudi table

2022-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5058: - Labels: pull-request-available (was: ) > The primary key cannot be empty when Flink reads an

[GitHub] [hudi] waywtdcc opened a new pull request, #7009: [HUDI-5058]Fix read spark table error : primary key col cat not nullable

2022-10-19 Thread GitBox
waywtdcc opened a new pull request, #7009: URL: https://github.com/apache/hudi/pull/7009 ### Change Logs Fix read spark table error : primary key col cat not nullable ### Impact ### Risk level (write none, low medium or high below) ### Documentation Update

[GitHub] [hudi] dongkelun commented on pull request #6448: [HUDI-4647] Change the default value of HIVE_SYNC_MODE in MergeInto to HMS

2022-10-19 Thread GitBox
dongkelun commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1284969757 > > > @dongkelun thanks for the patch. there are 2 reasons to why we have to close this for now: > > > > > > 1. we should keep default value consistent for different scenarios >

[GitHub] [hudi] yesemsanthoshkumar commented on a diff in pull request #6726: [HUDI-4630] Add transformer capability to individual feeds in MultiTableDeltaStreamer

2022-10-19 Thread GitBox
yesemsanthoshkumar commented on code in PR #6726: URL: https://github.com/apache/hudi/pull/6726#discussion_r1000163418 ## hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieMultiTableDeltaStreamer.java: ## @@ -245,10 +246,13 @@ public void

[GitHub] [hudi] xushiyan opened a new pull request, #7008: [MINOR] Update GitHub setting for branch protection

2022-10-19 Thread GitBox
xushiyan opened a new pull request, #7008: URL: https://github.com/apache/hudi/pull/7008 ### Change Logs - require branches to be up to date before merging - require at least 1 approving review ### Impact Stricter rules for landing PRs. ### Risk level (write

[GitHub] [hudi] vamshigv commented on pull request #6905: column name sanitization for row source

2022-10-19 Thread GitBox
vamshigv commented on PR #6905: URL: https://github.com/apache/hudi/pull/6905#issuecomment-1284964742 > JIRA is already created and I updated the description both in the PR and in JIRA (https://issues.apache.org/jira/browse/HUDI-5001). There are some finishing touches so marked the

[jira] [Updated] (HUDI-5001) Sanitize avro column names for RowSource

2022-10-19 Thread Vamshi Gudavarthi (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vamshi Gudavarthi updated HUDI-5001: Description: This issue is within the scope of row sources. The actual issue is that if the

[GitHub] [hudi] scxwhite closed pull request #5030: [HUDI-3617] MOR compact improve

2022-10-19 Thread GitBox
scxwhite closed pull request #5030: [HUDI-3617] MOR compact improve URL: https://github.com/apache/hudi/pull/5030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1284943541 ## CI report: * 7ba738aafa3f0df7307511963570400d39c10ceb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5903: [HUDI-4281] Using hudi to build a large number of tables in spark on hive causes OOM

2022-10-19 Thread GitBox
hudi-bot commented on PR #5903: URL: https://github.com/apache/hudi/pull/5903#issuecomment-1284941952 ## CI report: * 20a9d31f10a3a50434c73f2088659930a08ef1ea Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1284937074 ## CI report: * 7ba738aafa3f0df7307511963570400d39c10ceb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5903: [HUDI-4281] Using hudi to build a large number of tables in spark on hive causes OOM

2022-10-19 Thread GitBox
hudi-bot commented on PR #5903: URL: https://github.com/apache/hudi/pull/5903#issuecomment-1284935680 ## CI report: * 20a9d31f10a3a50434c73f2088659930a08ef1ea Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7005: Utilities and spark ci gh actions

2022-10-19 Thread GitBox
hudi-bot commented on PR #7005: URL: https://github.com/apache/hudi/pull/7005#issuecomment-1284931557 ## CI report: * 8d571a216ccbeb61d6365785b215291fc0d1d899 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1284931351 ## CI report: * 7ba738aafa3f0df7307511963570400d39c10ceb Azure:

[GitHub] [hudi] xushiyan commented on a diff in pull request #6284: [HUDI-4526] Improve spillableMapBasePath disk directory is full

2022-10-19 Thread GitBox
xushiyan commented on code in PR #6284: URL: https://github.com/apache/hudi/pull/6284#discussion_r1000140401 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieMemoryConfig.java: ## @@ -77,10 +78,20 @@ public class HoodieMemoryConfig extends

[GitHub] [hudi] nsivabalan commented on pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

2022-10-19 Thread GitBox
nsivabalan commented on PR #3035: URL: https://github.com/apache/hudi/pull/3035#issuecomment-1284922050 @fanaticjo : We landed a partial payload support via https://github.com/apache/hudi/pull/4676. Let us know if we can close this patch or if its possible to enhance the 4676 or if

[GitHub] [hudi] nsivabalan commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

2022-10-19 Thread GitBox
nsivabalan commented on PR #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1284921339 @stayrascal : We landed a partial payload support via https://github.com/apache/hudi/pull/4676. Let us know if we can close this patch or if its possible to enhance the 4676 or if

[GitHub] [hudi] nsivabalan commented on pull request #4838: [WIP] Adding support to casting array and struct cols to string in S3 incr source

2022-10-19 Thread GitBox
nsivabalan commented on PR #4838: URL: https://github.com/apache/hudi/pull/4838#issuecomment-1284919751 Closing this as we don't have much take aways. if there is an interest from the community, we can re-open. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] nsivabalan closed pull request #4838: [WIP] Adding support to casting array and struct cols to string in S3 incr source

2022-10-19 Thread GitBox
nsivabalan closed pull request #4838: [WIP] Adding support to casting array and struct cols to string in S3 incr source URL: https://github.com/apache/hudi/pull/4838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5030: [HUDI-3617] MOR compact improve

2022-10-19 Thread GitBox
nsivabalan commented on code in PR #5030: URL: https://github.com/apache/hudi/pull/5030#discussion_r1000139459 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java: ## @@ -123,25 +133,24 @@ public long getNumMergedRecordsInLog() {

[GitHub] [hudi] slfan1989 commented on a diff in pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
slfan1989 commented on code in PR #6946: URL: https://github.com/apache/hudi/pull/6946#discussion_r1000138749 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/hbase/SparkHoodieHBaseIndex.java: ## @@ -145,22 +154,22 @@ public HBaseIndexQPSResourceAllocator

[GitHub] [hudi] nsivabalan commented on pull request #6703: [HUDI-4866] Fixes the issue that the archive operation is invalid whe…

2022-10-19 Thread GitBox
nsivabalan commented on PR #6703: URL: https://github.com/apache/hudi/pull/6703#issuecomment-1284918114 @watermelon12138 : let me know if the patch still makes sense. I can take another look. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] nsivabalan commented on pull request #5319: [WIP] Adjusting `DeltaStreamer` shutdown sequence to avoid awaiting for 24h

2022-10-19 Thread GitBox
nsivabalan commented on PR #5319: URL: https://github.com/apache/hudi/pull/5319#issuecomment-1284916324 @alexeykudinkin : can we close this one if not required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] nsivabalan commented on pull request #5903: [HUDI-4281] Using hudi to build a large number of tables in spark on hive causes OOM

2022-10-19 Thread GitBox
nsivabalan commented on PR #5903: URL: https://github.com/apache/hudi/pull/5903#issuecomment-1284916055 have rebased w/ latest master. once CI is green, we can land. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] slfan1989 commented on a diff in pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
slfan1989 commented on code in PR #6946: URL: https://github.com/apache/hudi/pull/6946#discussion_r1000134826 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/hbase/SparkHoodieHBaseIndex.java: ## @@ -145,22 +154,22 @@ public HBaseIndexQPSResourceAllocator

[GitHub] [hudi] nsivabalan commented on a diff in pull request #6498: [HUDI-4878] Fix incremental cleaner use case

2022-10-19 Thread GitBox
nsivabalan commented on code in PR #6498: URL: https://github.com/apache/hudi/pull/6498#discussion_r980376754 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java: ## @@ -148,23 +148,52 @@ public List

[GitHub] [hudi] slfan1989 commented on a diff in pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
slfan1989 commented on code in PR #6946: URL: https://github.com/apache/hudi/pull/6946#discussion_r1000134826 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/hbase/SparkHoodieHBaseIndex.java: ## @@ -145,22 +154,22 @@ public HBaseIndexQPSResourceAllocator

[GitHub] [hudi] nsivabalan commented on a diff in pull request #6632: [HUDI-4753] more accurate record size estimation for log writing and spillable map

2022-10-19 Thread GitBox
nsivabalan commented on code in PR #6632: URL: https://github.com/apache/hudi/pull/6632#discussion_r1000134094 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -202,22 +199,19 @@ public R get(Object key) { @Override

[GitHub] [hudi] slfan1989 commented on pull request #6893: [HUDI-4997] use jackson-v2 replace jackson-v1 import.

2022-10-19 Thread GitBox
slfan1989 commented on PR #6893: URL: https://github.com/apache/hudi/pull/6893#issuecomment-1284908980 @xushiyan Thank you very much for helping to review the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on pull request #6856: [HUDI-4968] Update misleading read.streaming.skip_compaction config

2022-10-19 Thread GitBox
nsivabalan commented on PR #6856: URL: https://github.com/apache/hudi/pull/6856#issuecomment-1284903460 @danny0405 : can you follow up on the review when you get a chance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] nsivabalan commented on pull request #6930: [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr

2022-10-19 Thread GitBox
nsivabalan commented on PR #6930: URL: https://github.com/apache/hudi/pull/6930#issuecomment-1284901885 did we test both spark2 and spark3 for hive sync? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] dongkelun opened a new pull request, #6448: [HUDI-4647] Change the default value of HIVE_SYNC_MODE in MergeInto to HMS

2022-10-19 Thread GitBox
dongkelun opened a new pull request, #6448: URL: https://github.com/apache/hudi/pull/6448 By default, an error will be reported when synchronizing hive. When using SQL, it is troublesome to fill in the JDBC URL parameter every time ```java Caused by:

[GitHub] [hudi] xushiyan commented on pull request #6448: [HUDI-4647] Change the default value of HIVE_SYNC_MODE in MergeInto to HMS

2022-10-19 Thread GitBox
xushiyan commented on PR #6448: URL: https://github.com/apache/hudi/pull/6448#issuecomment-1284900637 > > @dongkelun thanks for the patch. there are 2 reasons to why we have to close this for now: > > > > 1. we should keep default value consistent for different scenarios > > 2. we

[hudi] branch release-feature-rfc46 updated: [MINOR] Make sure all `HoodieRecord`s are appropriately serializable by Kryo (#6977)

2022-10-19 Thread akudinkin
This is an automated email from the ASF dual-hosted git repository. akudinkin pushed a commit to branch release-feature-rfc46 in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/release-feature-rfc46 by this push: new 9a1aa0a683

[GitHub] [hudi] alexeykudinkin merged pull request #6977: [MINOR] Make sure all `HoodieRecord`s are appropriately serializable by Kryo

2022-10-19 Thread GitBox
alexeykudinkin merged PR #6977: URL: https://github.com/apache/hudi/pull/6977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] alexeykudinkin commented on pull request #6977: [MINOR] Make sure all `HoodieRecord`s are appropriately serializable by Kryo

2022-10-19 Thread GitBox
alexeykudinkin commented on PR #6977: URL: https://github.com/apache/hudi/pull/6977#issuecomment-1284899610 CI is green: https://user-images.githubusercontent.com/428277/196855042-1ffdaa0e-e345-4ad3-a4df-028203620f9f.png;>

[GitHub] [hudi] hudi-bot commented on pull request #7007: [HUDI-4809] Glue support drop partitions

2022-10-19 Thread GitBox
hudi-bot commented on PR #7007: URL: https://github.com/apache/hudi/pull/7007#issuecomment-1284887680 ## CI report: * a13855ef1e551ffe30aca9ce99a32243962bb8a2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6985: [HUDI-5046] Support all the hive sync options for flink sql

2022-10-19 Thread GitBox
hudi-bot commented on PR #6985: URL: https://github.com/apache/hudi/pull/6985#issuecomment-1284887542 ## CI report: * 831a255f14b45d2c1ad83e268924f9681d6ea90f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-19 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1284887497 ## CI report: * b7deb72b079e0638c8f033098f26827e6952e9ef Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6838: [MINOR] Update azure image and balance CI jobs

2022-10-19 Thread GitBox
hudi-bot commented on PR #6838: URL: https://github.com/apache/hudi/pull/6838#issuecomment-1284887255 ## CI report: * 3b01f5fd8a8be1d5b7dfca7adc882771f7fa787d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7007: [HUDI-4809] Glue support drop partitions

2022-10-19 Thread GitBox
hudi-bot commented on PR #7007: URL: https://github.com/apache/hudi/pull/7007#issuecomment-1284883895 ## CI report: * a13855ef1e551ffe30aca9ce99a32243962bb8a2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6985: [HUDI-5046] Support all the hive sync options for flink sql

2022-10-19 Thread GitBox
hudi-bot commented on PR #6985: URL: https://github.com/apache/hudi/pull/6985#issuecomment-1284883765 ## CI report: * 831a255f14b45d2c1ad83e268924f9681d6ea90f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-19 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1284883720 ## CI report: * b7deb72b079e0638c8f033098f26827e6952e9ef Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6838: [MINOR] Update azure image and balance CI jobs

2022-10-19 Thread GitBox
hudi-bot commented on PR #6838: URL: https://github.com/apache/hudi/pull/6838#issuecomment-1284883432 ## CI report: * 3b01f5fd8a8be1d5b7dfca7adc882771f7fa787d Azure:

[GitHub] [hudi] YannByron commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-19 Thread GitBox
YannByron commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1284882140 @Zouxxyy I see `scanPartitions` and `dropPartitions` are the major logic for `RepairTableCommand`. But both of them depends on the partition directories in FileSystem, that is not

[GitHub] [hudi] xushiyan commented on a diff in pull request #6824: [HUDI-4946] fix merge into with no preCombineField has dup row by onl…

2022-10-19 Thread GitBox
xushiyan commented on code in PR #6824: URL: https://github.com/apache/hudi/pull/6824#discussion_r1000114134 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala: ## @@ -160,7 +167,7 @@ case class

[GitHub] [hudi] hudi-bot commented on pull request #7003: [minor] add more test for rfc46

2022-10-19 Thread GitBox
hudi-bot commented on PR #7003: URL: https://github.com/apache/hudi/pull/7003#issuecomment-1284879445 ## CI report: * 8a5e66bc59593dfe25f0896dc8f40f6f4d4d5f30 Azure:

[GitHub] [hudi] nsivabalan closed pull request #6971: [HUDI-3900] Fixing configs for hdfs mini cluster

2022-10-19 Thread GitBox
nsivabalan closed pull request #6971: [HUDI-3900] Fixing configs for hdfs mini cluster URL: https://github.com/apache/hudi/pull/6971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[hudi] branch master updated (779a96506f -> 6acb775432)

2022-10-19 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 779a96506f [HUDI-4997] Use jackson-v2 import instead of jackson-v1 (#6893) add 6acb775432 [HUDI-3900] Fixing

[GitHub] [hudi] nsivabalan merged pull request #6981: [HUDI-3900] Fix tempDir usage in TestHoodieLogFormat

2022-10-19 Thread GitBox
nsivabalan merged PR #6981: URL: https://github.com/apache/hudi/pull/6981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (93c09f2226 -> 779a96506f)

2022-10-19 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 93c09f2226 [HUDI-4781] Allow omit metadata fields for hive sync (#6471) add 779a96506f [HUDI-4997] Use

[GitHub] [hudi] xushiyan merged pull request #6893: [HUDI-4997] use jackson-v2 replace jackson-v1 import.

2022-10-19 Thread GitBox
xushiyan merged PR #6893: URL: https://github.com/apache/hudi/pull/6893 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Closed] (HUDI-4781) Allow omit metadata fields for hive sync

2022-10-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4781. Resolution: Fixed > Allow omit metadata fields for hive sync > > >

[jira] [Updated] (HUDI-4781) Allow omit metadata fields for hive sync

2022-10-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4781: - Issue Type: New Feature (was: Improvement) > Allow omit metadata fields for hive sync >

[jira] [Updated] (HUDI-4781) Allow omit metadata fields for hive sync

2022-10-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4781: - Component/s: meta-sync > Allow omit metadata fields for hive sync >

[jira] [Updated] (HUDI-4781) Allow omit metadata fields for hive sync

2022-10-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4781: - Priority: Major (was: Minor) > Allow omit metadata fields for hive sync >

[hudi] branch master updated (c8c1ee473c -> 93c09f2226)

2022-10-19 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c8c1ee473c [HUDI-4998] Infer partition extractor class first from meta sync partition fields (#6899) add

[GitHub] [hudi] xushiyan merged pull request #6471: [HUDI-4781] Allow omit metadata fields for hive sync

2022-10-19 Thread GitBox
xushiyan merged PR #6471: URL: https://github.com/apache/hudi/pull/6471 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch asf-site updated: Updating 0.12.0 docs for known regression: (#6996)

2022-10-19 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 664294894c Updating 0.12.0 docs for

[GitHub] [hudi] bhasudha merged pull request #6996: [HUDI-5052][DOCS] Updating 0.12.0 docs for known regression

2022-10-19 Thread GitBox
bhasudha merged PR #6996: URL: https://github.com/apache/hudi/pull/6996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HUDI-4809) Hudi Support AWS Glue DropPartitions

2022-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4809: - Labels: pull-request-available (was: ) > Hudi Support AWS Glue DropPartitions >

[GitHub] [hudi] honeyaya opened a new pull request, #7007: HUDI-4809 glue support drop partitions

2022-10-19 Thread GitBox
honeyaya opened a new pull request, #7007: URL: https://github.com/apache/hudi/pull/7007 ### Change Logs Add Glue drop partitions (https://issues.apache.org/jira/browse/HUDI-4809) ### Impact little ### Risk level (write none, low medium or high below) low

[hudi] branch master updated (e6eb4e6f68 -> c8c1ee473c)

2022-10-19 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from e6eb4e6f68 [HUDI-5041] Fix lock metric register confict error (#6968) add c8c1ee473c [HUDI-4998] Infer partition

[GitHub] [hudi] xushiyan merged pull request #6899: [HUDI-4998] Inference of META_SYNC_PARTITION_EXTRACTOR_CLASS does not work with META_SYNC_PARTITION_FIELDS

2022-10-19 Thread GitBox
xushiyan merged PR #6899: URL: https://github.com/apache/hudi/pull/6899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] gubinjie closed issue #6914: [SUPPORT]Unable to merge duplicate data

2022-10-19 Thread GitBox
gubinjie closed issue #6914: [SUPPORT]Unable to merge duplicate data URL: https://github.com/apache/hudi/issues/6914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] danny0405 commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-19 Thread GitBox
danny0405 commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1284850120 Thanks for the contribution @hbgstc123 , you may need to rebase the latest master code and force push again. How to fetch the remote latest master: `git fetch upstream master`

[GitHub] [hudi] hudi-bot commented on pull request #6844: [HUDI-4960] Upgrade jetty version for timeline server

2022-10-19 Thread GitBox
hudi-bot commented on PR #6844: URL: https://github.com/apache/hudi/pull/6844#issuecomment-1284840291 ## CI report: * e6382ed4696ac9f205588e41a6d801e7fbed8480 UNKNOWN * ef02cf407f6e544ff2a629f6af0f5c63026362ce Azure:

[GitHub] [hudi] xushiyan commented on a diff in pull request #6971: [HUDI-3900] Fixing configs for hdfs mini cluster

2022-10-19 Thread GitBox
xushiyan commented on code in PR #6971: URL: https://github.com/apache/hudi/pull/6971#discussion_r193190 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/minicluster/HdfsTestService.java: ## @@ -85,7 +90,7 @@ public MiniDFSCluster start(boolean format) throws

[GitHub] [hudi] eshu commented on issue #6692: [SUPPORT] ClassCastException after migration to Hudi 0.12.0

2022-10-19 Thread GitBox
eshu commented on issue #6692: URL: https://github.com/apache/hudi/issues/6692#issuecomment-1284836860 @xushiyan Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer and Spark streaking sink

2022-10-19 Thread GitBox
hudi-bot commented on PR #5269: URL: https://github.com/apache/hudi/pull/5269#issuecomment-1284831101 ## CI report: * 5c5dde77caa0524e38ba5def698b490c0a35a6c4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer and Spark streaking sink

2022-10-19 Thread GitBox
hudi-bot commented on PR #5269: URL: https://github.com/apache/hudi/pull/5269#issuecomment-1284826838 ## CI report: * 5c5dde77caa0524e38ba5def698b490c0a35a6c4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2022-10-19 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284822757 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure:

[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2022-10-19 Thread GitBox
zhangyue19921010 commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284787088 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-19 Thread GitBox
hudi-bot commented on PR #6986: URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284772334 ## CI report: * 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-19 Thread GitBox
hudi-bot commented on PR #6986: URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284765831 ## CI report: * 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6999: [HUDI-5057] Fix msck repair hudi table

2022-10-19 Thread GitBox
hudi-bot commented on PR #6999: URL: https://github.com/apache/hudi/pull/6999#issuecomment-1284761222 ## CI report: * eb58602dcc2bde6ecbc79a74a0ee1ea5a8320f49 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2022-10-19 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284760321 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure:

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer and Spark streaking sink

2022-10-19 Thread GitBox
alexeykudinkin commented on code in PR #5269: URL: https://github.com/apache/hudi/pull/5269#discussion_r140906 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/async/HoodieAsyncTableService.java: ## @@ -20,24 +20,37 @@ package org.apache.hudi.async;

[GitHub] [hudi] hudi-bot commented on pull request #5416: [HUDI-3963] Use Lock-Free Message Queue Disruptor Improving Hoodie Writing Efficiency

2022-10-19 Thread GitBox
hudi-bot commented on PR #5416: URL: https://github.com/apache/hudi/pull/5416#issuecomment-1284718195 ## CI report: * b838e1f406902c9bdfb5e84d53ef5a5effd0765b UNKNOWN * 6114ee2aa59f087e5ef0b1b53979eec143b33f5e UNKNOWN * 92760dbf5a047fe1f9941fa4b36c944eb3bec5c7 UNKNOWN *

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer and Spark streaking sink

2022-10-19 Thread GitBox
nsivabalan commented on code in PR #5269: URL: https://github.com/apache/hudi/pull/5269#discussion_r106472 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/async/HoodieAsyncTableService.java: ## @@ -47,4 +60,11 @@ public void start(Function

[GitHub] [hudi] hudi-bot commented on pull request #7006: [HUDI-5063] Enabling run time stats to be serialized with commit metadata

2022-10-19 Thread GitBox
hudi-bot commented on PR #7006: URL: https://github.com/apache/hudi/pull/7006#issuecomment-1284652738 ## CI report: * 41131dab45357ef46c2f3a274da88feff085d97c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7006: [HUDI-5063] Enabling run time stats to be serialized with commit metadata

2022-10-19 Thread GitBox
hudi-bot commented on PR #7006: URL: https://github.com/apache/hudi/pull/7006#issuecomment-1284649059 ## CI report: * 41131dab45357ef46c2f3a274da88feff085d97c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7000: [HUDI-5060] Make all clean policies support incremental mode to find partition paths

2022-10-19 Thread GitBox
hudi-bot commented on PR #7000: URL: https://github.com/apache/hudi/pull/7000#issuecomment-1284645328 ## CI report: * 2f0d3f94991326b1f9885bf844f0b2e3fc6dda3a Azure:

[GitHub] [hudi] rahil-c commented on issue #6747: [SUPPORT] AWSDmsAvroPayload not found querying _rt table MoR

2022-10-19 Thread GitBox
rahil-c commented on issue #6747: URL: https://github.com/apache/hudi/issues/6747#issuecomment-1284641124 The `/usr/lib/hudi/hudi-utilities-bundle.jar` is needed, And the `spark-avro` jar is required if your running any hudi release under 0.11.0 (if you are on 0.11.0 or above you can omit

[jira] [Updated] (HUDI-5063) totalScantime and other run time stats missing from commit metadata

2022-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5063: - Labels: pull-request-available (was: ) > totalScantime and other run time stats missing from

[GitHub] [hudi] nsivabalan opened a new pull request, #7006: [HUDI-5063] Enabling run time stats to be serialized with commit metadata

2022-10-19 Thread GitBox
nsivabalan opened a new pull request, #7006: URL: https://github.com/apache/hudi/pull/7006 ### Change Logs Recently we landed a patch where we reduced duplicate stats from commit metadata (https://github.com/apache/hudi/pull/6646/) But looks like runtime stats were not

[GitHub] [hudi] hudi-bot commented on pull request #7005: Utilities and spark ci gh actions

2022-10-19 Thread GitBox
hudi-bot commented on PR #7005: URL: https://github.com/apache/hudi/pull/7005#issuecomment-1284595934 ## CI report: * 8d571a216ccbeb61d6365785b215291fc0d1d899 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true

2022-10-19 Thread GitBox
hudi-bot commented on PR #6986: URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284595815 ## CI report: * 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1284595628 ## CI report: * b9d722eeb13c5d97e76fa65547c27a5965379618 Azure:

[jira] [Updated] (HUDI-4201) Add tooling to delete empty non-completed instants from timeline

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4201: -- Sprint: 2022/09/05, 2022/09/19, 2022/10/04 (was: 2022/09/05, 2022/09/19, 2022/10/04,

[GitHub] [hudi] hudi-bot commented on pull request #7005: Utilities and spark ci gh actions

2022-10-19 Thread GitBox
hudi-bot commented on PR #7005: URL: https://github.com/apache/hudi/pull/7005#issuecomment-1284591411 ## CI report: * 8d571a216ccbeb61d6365785b215291fc0d1d899 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6946: [HUDI-5027] Improve getHBaseConnection Use Constants Replace HardCode.

2022-10-19 Thread GitBox
hudi-bot commented on PR #6946: URL: https://github.com/apache/hudi/pull/6946#issuecomment-1284590858 ## CI report: * b9d722eeb13c5d97e76fa65547c27a5965379618 Azure:

[jira] [Commented] (HUDI-1570) Add Avg record size in commit metadata

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620607#comment-17620607 ] sivabalan narayanan commented on HUDI-1570: --- Add a FAQ on how to fetch the record size for a

[jira] [Commented] (HUDI-4201) Add tooling to delete empty non-completed instants from timeline

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620608#comment-17620608 ] sivabalan narayanan commented on HUDI-4201: --- There isn't much we could do here. So, lets punt on

[jira] [Closed] (HUDI-4983) Integrate hudi-examples to be tested with azure CI test run

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4983. - Resolution: Fixed we have the bundle testing initiatives to cover this. already we have

[jira] [Updated] (HUDI-5063) totalScantime and other run time stats missing from commit metadata

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5063: -- Sprint: 2022/10/18 > totalScantime and other run time stats missing from commit

[jira] [Created] (HUDI-5063) totalScantime and other run time stats missing from commit metadata

2022-10-19 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5063: - Summary: totalScantime and other run time stats missing from commit metadata Key: HUDI-5063 URL: https://issues.apache.org/jira/browse/HUDI-5063 Project:

[jira] [Assigned] (HUDI-5063) totalScantime and other run time stats missing from commit metadata

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5063: - Assignee: sivabalan narayanan > totalScantime and other run time stats missing

[jira] [Updated] (HUDI-5063) totalScantime and other run time stats missing from commit metadata

2022-10-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5063: -- Fix Version/s: 0.12.2 > totalScantime and other run time stats missing from commit

[GitHub] [hudi] hudi-bot commented on pull request #6976: [HUDI-5042]fix clustering schedule problem in flink

2022-10-19 Thread GitBox
hudi-bot commented on PR #6976: URL: https://github.com/apache/hudi/pull/6976#issuecomment-1284585858 ## CI report: * b7deb72b079e0638c8f033098f26827e6952e9ef Azure:

  1   2   3   4   >