Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-21 Thread via GitHub
nsivabalan commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1822259003 testHoodieAsyncClusteringJobWithScheduleAndExecute{String, HoodieRecordType}[1] is known to be flaky. -- This is an automated message from the Apache Git Service. To

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
majian1998 closed pull request #10120: [HUDI-7110] Add call procedure for show column stats information URL: https://github.com/apache/hudi/pull/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [HUDI-7133] Improve dbt example for better guidance [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10155: URL: https://github.com/apache/hudi/pull/10155#issuecomment-1822243889 ## CI report: * af912a3fed7270708fad935b7df55fb508cd5536 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1822243792 ## CI report: * 7a678c8f26e7b94fca3812d29e9ddca59b083127 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7123] Improve CI scripts [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10136: URL: https://github.com/apache/hudi/pull/10136#issuecomment-1822243691 ## CI report: * dd3d933d329208fcdf9c00ed2dcb12a7e22cce26 Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1822243625 ## CI report: * bbf765005b4e1e92730e0dff736bcb561d928b7b Azure:

Re: [PR] [HUDI-7095] Making perf enhancements to JSON serde [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10097: URL: https://github.com/apache/hudi/pull/10097#issuecomment-1822243447 ## CI report: * 43400ce2317882c76a68eb3a855c9dd814c92234 Azure:

Re: [PR] [HUDI-7133] Improve dbt example for better guidance [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10155: URL: https://github.com/apache/hudi/pull/10155#issuecomment-1822235911 ## CI report: * af912a3fed7270708fad935b7df55fb508cd5536 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7123] Improve CI scripts [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10136: URL: https://github.com/apache/hudi/pull/10136#issuecomment-1822235518 ## CI report: * dd3d933d329208fcdf9c00ed2dcb12a7e22cce26 Azure:

Re: [I] [SUPPORT] hudi-examples-dbt not running with spark thrift server [hudi]

2023-11-21 Thread via GitHub
xushiyan commented on issue #6125: URL: https://github.com/apache/hudi/issues/6125#issuecomment-181787 @sambhav13 I'm updating the instructions in the dbt example (using spark 3.2 and hudi 0.14.0). Please check this out and let us know if it helps.

[jira] [Updated] (HUDI-7133) Improve dbt example for better guidance

2023-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7133: - Labels: pull-request-available (was: ) > Improve dbt example for better guidance >

[PR] [HUDI-7133] Improve dbt example for better guidance [hudi]

2023-11-21 Thread via GitHub
xushiyan opened a new pull request, #10155: URL: https://github.com/apache/hudi/pull/10155 ### Change Logs Update dbt example with more detailed instructions. ### Impact Improve dbt example for learning. ### Risk level None. ### Documentation Update

[jira] [Created] (HUDI-7133) Improve dbt example for better guidance

2023-11-21 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-7133: Summary: Improve dbt example for better guidance Key: HUDI-7133 URL: https://issues.apache.org/jira/browse/HUDI-7133 Project: Apache Hudi Issue Type: Improvement

[jira] [Closed] (HUDI-7096) Improve Incr Query for partitions touched based on start and end

2023-11-21 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7096. - Fix Version/s: 0.14.1 Resolution: Fixed > Improve Incr Query for partitions touched based on start

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1822191008 ## CI report: * c359d7d70cb6e34ce4d62b2f71f39b91b49ea334 Azure:

(hudi) branch master updated (2522f6de6f1 -> c5af85dfd91)

2023-11-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 2522f6de6f1 [HUDI-7128] DeleteMarkerProcedures support delete in batch mode (#10148) add c5af85dfd91

Re: [PR] [HUDI-7096] Improving incremental query to fetch partitions based on commit metadata [hudi]

2023-11-21 Thread via GitHub
nsivabalan merged PR #10098: URL: https://github.com/apache/hudi/pull/10098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10152: URL: https://github.com/apache/hudi/pull/10152#issuecomment-1822183860 ## CI report: * 9764bc6527e5e3e83ed08263484beb45c1796d47 UNKNOWN * b1748e270c379c479bb3286e635482d204b853c5 Azure:

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1822183724 ## CI report: * 03451f7cd016ee9fb078f4d78f3b771e8719c233 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1822183538 ## CI report: * c359d7d70cb6e34ce4d62b2f71f39b91b49ea334 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1822176875 ## CI report: * c586dc82aa4c791cabd8f3172ee1f982f71433da Azure:

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1822176736 ## CI report: * 03451f7cd016ee9fb078f4d78f3b771e8719c233 Azure:

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10152: URL: https://github.com/apache/hudi/pull/10152#issuecomment-1822169857 ## CI report: * 9764bc6527e5e3e83ed08263484beb45c1796d47 UNKNOWN * b1748e270c379c479bb3286e635482d204b853c5 Azure:

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1822169590 ## CI report: * 48df6bbec2473dbbbedb1b723896acb17056e80f Azure:

Re: [I] Fail to add default partition [hudi]

2023-11-21 Thread via GitHub
ad1happy2go commented on issue #10154: URL: https://github.com/apache/hudi/issues/10154#issuecomment-1822169099 @njalan Do your partition column in data contains NULLS? When are you facing this error? Looks like you are trying to add the null partition. It may not be hudi related but more

Re: [I] [SUPPORT] Clean action failure triggers an exception while trying to check whether metadata is a table [hudi]

2023-11-21 Thread via GitHub
ad1happy2go commented on issue #10127: URL: https://github.com/apache/hudi/issues/10127#issuecomment-1822166971 @shubhamn21 Were you able to resolve this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] [SUPPORT] HUDI MOR table type compaction failed post adding new field in the schema [hudi]

2023-11-21 Thread via GitHub
ad1happy2go commented on issue #10138: URL: https://github.com/apache/hudi/issues/10138#issuecomment-1822164822 @abhisheksahani91 There looks like related to this which is yet to be fixed. https://github.com/apache/hudi/pull/5269 To unblock you can disable the timeline server for

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
majian1998 commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1822158300 Improvement: Using FileSystemView to obtain the latest file slices, only displaying valid and up-to-date file information when showing column stats information. At the same time,

[jira] [Updated] (HUDI-7132) Data may be lost in flink#chk

2023-11-21 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7132: - Description:

[jira] [Updated] (HUDI-7132) Data may be lost in flink#chk

2023-11-21 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7132: - Description:

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
danny0405 commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1822104388 You got some test failures: https://github.com/apache/hudi/actions/runs/6940997119/job/18914907624?pr=10120, you can rebase with the latest master to fix the Azure tests. -- This is

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1822079113 ## CI report: * 34ffc8261d951bde500df7688800b2ed6afb4fa6 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1822078574 ## CI report: * c359d7d70cb6e34ce4d62b2f71f39b91b49ea334 Azure:

Re: [I] [SUPPORT] Invalid number of file groups for partition:column_stats [hudi]

2023-11-21 Thread via GitHub
ocean-zhc commented on issue #7657: URL: https://github.com/apache/hudi/issues/7657#issuecomment-1822048995 > I have came across the same problem using 0.12.0 version. I have set > > hoodie.metadata.index.bloom.filter.enable=false hoodie.metadata.index.column.stats.enable=false >

(hudi) branch master updated (a1afcdd989c -> 2522f6de6f1)

2023-11-21 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from a1afcdd989c [HUDI-7115] Add in new options for the bigquery sync (#10125) add 2522f6de6f1 [HUDI-7128]

[jira] [Closed] (HUDI-7128) DeleteProcedures support delete in batch mode

2023-11-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7128. Resolution: Fixed Fixed via master branch: 2522f6de6f13f44bac89c81bb753c58a52cc780c > DeleteProcedures

Re: [PR] [HUDI-7128] DeleteProcedures support batch mode [hudi]

2023-11-21 Thread via GitHub
danny0405 merged PR #10148: URL: https://github.com/apache/hudi/pull/10148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (HUDI-7059) Read record positions with filter pushdown using Spark parquet reader

2023-11-21 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-7059: --- Assignee: Lin Liu > Read record positions with filter pushdown using Spark parquet reader >

Re: [PR] [HUDI-7128] DeleteProcedures support batch mode [hudi]

2023-11-21 Thread via GitHub
xuzifu666 commented on PR #10148: URL: https://github.com/apache/hudi/pull/10148#issuecomment-1822035400 @danny0405 @yihua PTAL CI error seems not related to the changed code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1822035335 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10095: URL: https://github.com/apache/hudi/pull/10095#issuecomment-1822035213 ## CI report: * b77d51aac5e370b00bab3acfccd471cf03a1c718 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1822030407 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10095: URL: https://github.com/apache/hudi/pull/10095#issuecomment-1822030206 ## CI report: * b77d51aac5e370b00bab3acfccd471cf03a1c718 Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1822025391 ## CI report: * 697114b6ec4f578123363a89a6846e352bc3a53e Azure:

Re: [PR] [HUDI-7097] Fixing instantiation of Hms Uri with HiveSync tool [hudi]

2023-11-21 Thread via GitHub
nsivabalan commented on code in PR #10099: URL: https://github.com/apache/hudi/pull/10099#discussion_r1401453914 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -103,15 +103,29 @@ public class HiveSyncTool extends HoodieSyncTool

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1822019706 ## CI report: * 697114b6ec4f578123363a89a6846e352bc3a53e Azure:

[jira] [Commented] (HUDI-7131) The requested schema is not compatible with the file schema

2023-11-21 Thread loukey_j (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788607#comment-17788607 ] loukey_j commented on HUDI-7131: The schema of the table has not changed, only the partition value of the

[jira] [Commented] (HUDI-7131) The requested schema is not compatible with the file schema

2023-11-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788605#comment-17788605 ] Danny Chen commented on HUDI-7131: -- It looks like an known issue, we do not support schema evolution on

[jira] [Updated] (HUDI-7132) Data may be lost in flink#chk

2023-11-21 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7132: - Description:

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-21 Thread via GitHub
danny0405 commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1821998306 @VitoMakarevich Nice catch, can you fix the compile error: error

[jira] [Updated] (HUDI-7132) Data may be lost in flink#chk

2023-11-21 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7132: - Description:

[jira] [Created] (HUDI-7132) Data may be lost in flink#chk

2023-11-21 Thread Bo Cui (Jira)
Bo Cui created HUDI-7132: Summary: Data may be lost in flink#chk Key: HUDI-7132 URL: https://issues.apache.org/jira/browse/HUDI-7132 Project: Apache Hudi Issue Type: Bug Components: flink

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821990146 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-5968] Fix global index duplicate and handle custom payload when update partition [hudi]

2023-11-21 Thread via GitHub
loukey-lj commented on PR #8490: URL: https://github.com/apache/hudi/pull/8490#issuecomment-1821987556 @xushiyan @nsivabalan https://issues.apache.org/jira/browse/HUDI-7131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Updated] (HUDI-7131) The requested schema is not compatible with the file schema

2023-11-21 Thread loukey_j (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] loukey_j updated HUDI-7131: --- Affects Version/s: 0.14.0 > The requested schema is not compatible with the file schema >

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821984783 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

[jira] [Created] (HUDI-7131) The requested schema is not compatible with the file schema

2023-11-21 Thread loukey_j (Jira)
loukey_j created HUDI-7131: -- Summary: The requested schema is not compatible with the file schema Key: HUDI-7131 URL: https://issues.apache.org/jira/browse/HUDI-7131 Project: Apache Hudi Issue

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10152: URL: https://github.com/apache/hudi/pull/10152#issuecomment-1821984821 ## CI report: * 9764bc6527e5e3e83ed08263484beb45c1796d47 UNKNOWN * d0fe92994777e2067d654e2585c75c91616f8598 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
nsivabalan commented on code in PR #10150: URL: https://github.com/apache/hudi/pull/10150#discussion_r1401416490 ## hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java: ## @@ -89,6 +90,18 @@ public Option getInsertValue(Schema schema,

Re: [PR] [HUDI-7097] Fixing instantiation of Hms Uri with HiveSync tool [hudi]

2023-11-21 Thread via GitHub
xushiyan commented on code in PR #10099: URL: https://github.com/apache/hudi/pull/10099#discussion_r1401411887 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -103,15 +103,29 @@ public class HiveSyncTool extends HoodieSyncTool implements

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
danny0405 commented on code in PR #10150: URL: https://github.com/apache/hudi/pull/10150#discussion_r1401397731 ## hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java: ## @@ -89,6 +90,18 @@ public Option getInsertValue(Schema schema,

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
danny0405 commented on code in PR #10150: URL: https://github.com/apache/hudi/pull/10150#discussion_r1401397731 ## hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java: ## @@ -89,6 +90,18 @@ public Option getInsertValue(Schema schema,

Re: [PR] [HUDI-7096] Improving incremental query to fetch partitions based on commit metadata [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10098: URL: https://github.com/apache/hudi/pull/10098#issuecomment-1821947415 ## CI report: * abd651f8dcbde53717e473efe1c15d4bd486b0eb Azure:

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1821947343 ## CI report: * fff2ac40c67fdcd15fdf4b65890e00d63aa60a0a Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821947562 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-7095] Making perf enhancements to JSON serde [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10097: URL: https://github.com/apache/hudi/pull/10097#issuecomment-1821947383 ## CI report: * aaf5a310c5ac999c81498308fdc11d6d5171463d Azure:

Re: [PR] [HUDI-7023] Support querying without syncing partition metadata to catalog [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10153: URL: https://github.com/apache/hudi/pull/10153#issuecomment-1821942182 ## CI report: * 46a4c3344c79fd9a61db78620e8c40e7d98bcd36 Azure:

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1821942128 ## CI report: * 3d1e603aea3cc23614de38d511b5d4ddeac92f5d Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821942086 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10102: URL: https://github.com/apache/hudi/pull/10102#issuecomment-1821941946 ## CI report: * a7f01f6ad7008830e6e2993b0ba5c986ca493093 Azure:

Re: [PR] [HUDI-7096] Improving incremental query to fetch partitions based on commit metadata [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10098: URL: https://github.com/apache/hudi/pull/10098#issuecomment-1821941910 ## CI report: * abd651f8dcbde53717e473efe1c15d4bd486b0eb Azure:

Re: [PR] [HUDI-7095] Making perf enhancements to JSON serde [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10097: URL: https://github.com/apache/hudi/pull/10097#issuecomment-1821941849 ## CI report: * aaf5a310c5ac999c81498308fdc11d6d5171463d Azure:

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1821941789 ## CI report: * fff2ac40c67fdcd15fdf4b65890e00d63aa60a0a Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-21 Thread via GitHub
nsivabalan commented on code in PR #10122: URL: https://github.com/apache/hudi/pull/10122#discussion_r1401369779 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnReadWithCompact.java: ## @@ -159,6 +159,8 @@ public void

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1821901635 ## CI report: * b124e2a54cd9b3fec6d19c7c131b93234cd8c68c Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821901599 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10102: URL: https://github.com/apache/hudi/pull/10102#issuecomment-1821901457 ## CI report: * c3ff2511a30564e5a5ff0cb407326ff6ef0584e3 Azure:

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1821895329 ## CI report: * b124e2a54cd9b3fec6d19c7c131b93234cd8c68c Azure:

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10102: URL: https://github.com/apache/hudi/pull/10102#issuecomment-1821895011 ## CI report: * c3ff2511a30564e5a5ff0cb407326ff6ef0584e3 Azure:

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
linliu-code commented on code in PR #10102: URL: https://github.com/apache/hudi/pull/10102#discussion_r1401341463 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java: ## @@ -260,7 +260,7 @@ private void scanInternalV1(Option keySpecOpt)

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
linliu-code commented on code in PR #10102: URL: https://github.com/apache/hudi/pull/10102#discussion_r1401341463 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java: ## @@ -260,7 +260,7 @@ private void scanInternalV1(Option keySpecOpt)

Re: [PR] [HUDI-7102] Fix a bug for time travel queries on MOR tables [hudi]

2023-11-21 Thread via GitHub
linliu-code commented on code in PR #10102: URL: https://github.com/apache/hudi/pull/10102#discussion_r1401341463 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java: ## @@ -260,7 +260,7 @@ private void scanInternalV1(Option keySpecOpt)

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-21 Thread via GitHub
VitoMakarevich commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1821859085 Added a test, at least `fileIndex`-related tests pass(I ran only them), also can verify that executing `.refresh` on existing Index does not refresh a list of files as reported,

Re: [PR] [HUDI-7129] Fix bug when upgrade from table version three using UpgradeOrDowngradeProcedure [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10147: URL: https://github.com/apache/hudi/pull/10147#issuecomment-1821842256 ## CI report: * 994d062df78afd5062dec418cddff167daff42d8 Azure:

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1821786342 ## CI report: * 03451f7cd016ee9fb078f4d78f3b771e8719c233 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821715566 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10150: URL: https://github.com/apache/hudi/pull/10150#issuecomment-1821703634 ## CI report: * ea3efa0db6b2a2e88508641d6ffb7eec9c33bf00 Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821703416 ## CI report: * 3d48bfc5c41a59a1114eb73a5ef9a7b7fda5eccf Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1821683706 ## CI report: * 697114b6ec4f578123363a89a6846e352bc3a53e Azure:

(hudi) branch master updated: [HUDI-7115] Add in new options for the bigquery sync (#10125)

2023-11-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new a1afcdd989c [HUDI-7115] Add in new options for

Re: [PR] [HUDI-7115] Add in new options for the bigquery sync [hudi]

2023-11-21 Thread via GitHub
nsivabalan merged PR #10125: URL: https://github.com/apache/hudi/pull/10125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7129] Fix bug when upgrade from table version three using UpgradeOrDowngradeProcedure [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10147: URL: https://github.com/apache/hudi/pull/10147#issuecomment-1821620165 ## CI report: * 1dee5fb303eff272371c638d07d80806676fd5aa Azure:

Re: [PR] [HUDI-7130] Adding support for configuring value serializer with JsonKakfaSource [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10149: URL: https://github.com/apache/hudi/pull/10149#issuecomment-1821620236 ## CI report: * e809a39b71dcfa3ddcfc6348b6740391b2a08dbd Azure:

(hudi) branch branch-0.x created (now 0908f648152)

2023-11-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch branch-0.x in repository https://gitbox.apache.org/repos/asf/hudi.git at 0908f648152 [HUDI-6999] Adding row writer support to HoodieStreamer (#9913) This branch includes the following

Re: [I] [SUPPORT] HUDI MOR table type compaction failed post adding new field in the schema [hudi]

2023-11-21 Thread via GitHub
abhisheksahani91 commented on issue #10138: URL: https://github.com/apache/hudi/issues/10138#issuecomment-1821565443 @ad1happy2go I also want to add the point the connection refused error is observed when I am generating the high load on hudi ingestion

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10152: URL: https://github.com/apache/hudi/pull/10152#issuecomment-1821555262 ## CI report: * 9764bc6527e5e3e83ed08263484beb45c1796d47 UNKNOWN * d0fe92994777e2067d654e2585c75c91616f8598 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1821554730 ## CI report: * 898d03c01442d0b4ac84056f25ff49f1f9aba0c0 Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821544132 ## CI report: * 4913158456e1dfaa1366ba7bd5029578f3bf4cef Azure:

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10152: URL: https://github.com/apache/hudi/pull/10152#issuecomment-1821544519 ## CI report: * 9764bc6527e5e3e83ed08263484beb45c1796d47 UNKNOWN * d0fe92994777e2067d654e2585c75c91616f8598 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1821543331 ## CI report: * 898d03c01442d0b4ac84056f25ff49f1f9aba0c0 Azure:

  1   2   >