Re: [PR] [HUDI-7552] Remove the suffix for MDT table service instants [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10945: URL: https://github.com/apache/hudi/pull/10945#issuecomment-2033604303 ## CI report: * 6c3830bb4de1887f41aebc139b3fc837e446ead5 Azure:

(hudi) branch master updated: [HUDI-7569] [RLI] Fix wrong result generated by query (#10955)

2024-04-02 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 59e32b7e686 [HUDI-7569] [RLI] Fix wrong result

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bvaradar merged PR #10955: URL: https://github.com/apache/hudi/pull/10955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7552] Remove the suffix for MDT table service instants [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on code in PR #10945: URL: https://github.com/apache/hudi/pull/10945#discussion_r1548924402 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1613,21 +1595,6 @@ private static int

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bhat-vinay commented on code in PR #10955: URL: https://github.com/apache/hudi/pull/10955#discussion_r1548899945 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestRecordLevelIndexWithSQL.scala: ## @@ -155,4 +156,36 @@ class

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bhat-vinay commented on code in PR #10955: URL: https://github.com/apache/hudi/pull/10955#discussion_r1548899945 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestRecordLevelIndexWithSQL.scala: ## @@ -155,4 +156,36 @@ class

Re: [PR] [HUDI-7235] Fix checkpoint bug for S3/GCS Incremental Source [hudi]

2024-04-02 Thread via GitHub
bvaradar commented on code in PR #10336: URL: https://github.com/apache/hudi/pull/10336#discussion_r1548893991 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/IncrSourceHelper.java: ## @@ -183,7 +183,8 @@ public static Pair>> filterAndGen

Re: [PR] [HUDI-7486] Classify schema exceptions when converting from avro to spark row representation [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10778: URL: https://github.com/apache/hudi/pull/10778#issuecomment-2033485364 ## CI report: * 1770608bb63ae127c56ee13ecbab225a685d49e2 Azure:

[jira] [Updated] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat updated HUDI-7569: -- Sprint: Sprint 2024-03-25 > Fix wrong result while using RLI for pruning files >

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bvaradar commented on code in PR #10955: URL: https://github.com/apache/hudi/pull/10955#discussion_r1548866685 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestRecordLevelIndexWithSQL.scala: ## @@ -155,4 +156,36 @@ class

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
jonvex commented on code in PR #10954: URL: https://github.com/apache/hudi/pull/10954#discussion_r1548833622 ## hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark24HoodieParquetReader.scala: ## @@ -0,0 +1,222 @@ +/* + *

Re: [PR] [HUDI-7486] Classify schema exceptions when converting from avro to spark row representation [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10778: URL: https://github.com/apache/hudi/pull/10778#issuecomment-2033411270 ## CI report: * 51380200fafd1b3917658c549ab3caa3e5a408f5 Azure:

Re: [PR] [HUDI-7564] Fix HiveSyncConfig inconsistency [hudi]

2024-04-02 Thread via GitHub
voonhous commented on PR #10951: URL: https://github.com/apache/hudi/pull/10951#issuecomment-2033409586 > The timestamp(3) sync from 0.14.x has been changed to be as `timestamp` type instead of `long` for the metastore sync, guess that is the direct cause of the issue, would merge it first

Re: [PR] [HUDI-7486] Classify schema exceptions when converting from avro to spark row representation [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10778: URL: https://github.com/apache/hudi/pull/10778#issuecomment-2033406371 ## CI report: * 51380200fafd1b3917658c549ab3caa3e5a408f5 Azure:

[jira] [Updated] (HUDI-7559) Fix functional index (on column stats): Handle NPE in filterQueriesWithRecordKey(...)

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7559: Status: Patch Available (was: In Progress) > Fix functional index (on column stats): Handle NPE in >

Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]

2024-04-02 Thread via GitHub
wombatu-kun commented on code in PR #10942: URL: https://github.com/apache/hudi/pull/10942#discussion_r1548807847 ## hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java: ## @@ -31,12 +32,21 @@ * * @param

[jira] [Updated] (HUDI-6497) Replace FileSystem, Path, and FileStatus usage in hudi-common

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6497: Sprint: Sprint 2024-03-25 > Replace FileSystem, Path, and FileStatus usage in hudi-common >

[jira] [Updated] (HUDI-6497) Replace FileSystem, Path, and FileStatus usage in hudi-common

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6497: Status: Patch Available (was: In Progress) > Replace FileSystem, Path, and FileStatus usage in hudi-common

[jira] [Updated] (HUDI-6497) Replace FileSystem, Path, and FileStatus usage in hudi-common

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6497: Status: In Progress (was: Open) > Replace FileSystem, Path, and FileStatus usage in hudi-common >

Re: [PR] [HUDI-7552] Remove the suffix for MDT table service instants [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on code in PR #10945: URL: https://github.com/apache/hudi/pull/10945#discussion_r1548805924 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1375,33 +1375,15 @@ public static Set

Re: [PR] Bump org.apache.commons:commons-configuration2 from 2.8.0 to 2.10.1 in /packaging/hudi-cli-bundle [hudi]

2024-04-02 Thread via GitHub
yihua closed pull request #10920: Bump org.apache.commons:commons-configuration2 from 2.8.0 to 2.10.1 in /packaging/hudi-cli-bundle URL: https://github.com/apache/hudi/pull/10920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Bump org.apache.commons:commons-configuration2 from 2.8.0 to 2.10.1 in /packaging/hudi-cli-bundle [hudi]

2024-04-02 Thread via GitHub
dependabot[bot] commented on PR #10920: URL: https://github.com/apache/hudi/pull/10920#issuecomment-2033378172 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] [HUDI-7552] Remove the suffix for MDT table service instants [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on code in PR #10945: URL: https://github.com/apache/hudi/pull/10945#discussion_r1548805276 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1613,21 +1595,6 @@ private static int

(hudi) branch dependabot/maven/packaging/hudi-cli-bundle/org.apache.commons-commons-configuration2-2.10.1 deleted (was 882fb5ceed4)

2024-04-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/packaging/hudi-cli-bundle/org.apache.commons-commons-configuration2-2.10.1 in repository https://gitbox.apache.org/repos/asf/hudi.git was 882fb5ceed4 Bump

Re: [PR] Bump org.apache.commons:commons-configuration2 from 2.8.0 to 2.10.1 in /packaging/hudi-cli-bundle [hudi]

2024-04-02 Thread via GitHub
yihua commented on PR #10920: URL: https://github.com/apache/hudi/pull/10920#issuecomment-2033378151 This may break Hudi CLI functionality. Closing it now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (HUDI-6497) Replace FileSystem, Path, and FileStatus usage in hudi-common

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6497: Fix Version/s: 0.15.0 1.0.0 > Replace FileSystem, Path, and FileStatus usage in

[jira] [Updated] (HUDI-6497) Replace FileSystem, Path, and FileStatus usage in hudi-common

2024-04-02 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6497: Fix Version/s: (was: 1.1.0) > Replace FileSystem, Path, and FileStatus usage in hudi-common >

Re: [PR] [HUDI-7552] Remove the suffix for MDT table service instants [hudi]

2024-04-02 Thread via GitHub
yihua commented on code in PR #10945: URL: https://github.com/apache/hudi/pull/10945#discussion_r1548795405 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -1367,10 +1362,8 @@ protected void

Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on code in PR #10942: URL: https://github.com/apache/hudi/pull/10942#discussion_r1548797462 ## hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java: ## @@ -31,12 +32,21 @@ * * @param

[jira] [Updated] (HUDI-7552) Remove the suffix for MDT table service instants

2024-04-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7552: - Description: We wanna remove the very specific design for MDT so that it's behavior is in sync with the

[jira] [Updated] (HUDI-7564) Fix HiveSync configuration inconsistencies

2024-04-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7564: - Fix Version/s: 0.15.0 > Fix HiveSync configuration inconsistencies >

[jira] [Closed] (HUDI-7564) Fix HiveSync configuration inconsistencies

2024-04-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7564. Resolution: Fixed Fixed via master branch: 8bb6bee6234ce3fd56462877ce28d70cd62a8e18 > Fix HiveSync

(hudi) branch master updated: [HUDI-7564] Fix HiveSyncConfig inconsistency (#10951)

2024-04-02 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8bb6bee6234 [HUDI-7564] Fix HiveSyncConfig

Re: [PR] [HUDI-7564] Fix HiveSyncConfig inconsistency [hudi]

2024-04-02 Thread via GitHub
danny0405 merged PR #10951: URL: https://github.com/apache/hudi/pull/10951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7564] Fix HiveSyncConfig inconsistency [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on PR #10951: URL: https://github.com/apache/hudi/pull/10951#issuecomment-2033344958 The timestamp(3) sync from 0.14.x has been changed to be as `timestamp` type instead of `long` for the metastore sync, guess that is the direct cause of the issue, would merge it first

Re: [I] [Discuss] Do we need to promote the bucket number as table config instead of a write config [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on issue #8459: URL: https://github.com/apache/hudi/issues/8459#issuecomment-2033342512 cc @yihua , I think this request is still valuable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] [SUPPORT] Metadata table not cleaned / compacted, log files growing rapidly [hudi]

2024-04-02 Thread via GitHub
danny0405 commented on issue #8567: URL: https://github.com/apache/hudi/issues/8567#issuecomment-2033341535 There might be no good solutions for 0..x release, recently we have merged a PR that address this issue on master branch: https://github.com/apache/hudi/pull/10874, it would be

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
yihua commented on code in PR #10954: URL: https://github.com/apache/hudi/pull/10954#discussion_r1548749928 ## hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark24HoodieParquetReader.scala: ## @@ -0,0 +1,222 @@ +/* + *

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
yihua commented on code in PR #10954: URL: https://github.com/apache/hudi/pull/10954#discussion_r1548747722 ## hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark24HoodieParquetReader.scala: ## @@ -0,0 +1,222 @@ +/* + *

[jira] [Updated] (HUDI-7503) concurrent executions of table service plan should not corrupt dataset

2024-04-02 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7503: --- Summary: concurrent executions of table service plan should not corrupt dataset (was: concurrent

Re: [PR] [HUDI-7486] Classify schema exceptions when converting from avro to spark row representation [hudi]

2024-04-02 Thread via GitHub
yihua commented on code in PR #10778: URL: https://github.com/apache/hudi/pull/10778#discussion_r1548735728 ## hudi-common/src/main/java/org/apache/hudi/exception/HoodieRecordCreationException.java: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [DO NOT MERGE][HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2033250518 ## CI report: * 66f7add237e807bc7ad7a870ee39f3c60762b728 Azure:

Re: [PR] [DO NOT MERGE][HUDI-7566] Add schema evolution to spark file readers [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10956: URL: https://github.com/apache/hudi/pull/10956#issuecomment-2033191436 ## CI report: * 2f069604e8c23796aedd9eda8458a95a583db6d1 Azure:

Re: [PR] [DO NOT MERGE][HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2033112418 ## CI report: * 66f7add237e807bc7ad7a870ee39f3c60762b728 Azure:

Re: [PR] [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark [hudi]

2024-04-02 Thread via GitHub
jonvex closed pull request #10278: [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark URL: https://github.com/apache/hudi/pull/10278 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [DO NOT MERGE][HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2033100902 ## CI report: * 66f7add237e807bc7ad7a870ee39f3c60762b728 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10278: URL: https://github.com/apache/hudi/pull/10278#issuecomment-2033098416 ## CI report: * 5d82fe635b49c83e18621b8286ba6682786b5a9a Azure:

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2033089327 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d Azure:

Re: [PR] [MINOR] use Temurin jdk [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10948: URL: https://github.com/apache/hudi/pull/10948#issuecomment-2033089269 ## CI report: * cf26cf98330470d271a3529d0bd36060fea601dd Azure:

[jira] [Updated] (HUDI-7567) Break-up schema evolution: Integrate the readers into the new fg reader

2024-04-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7567: - Labels: pull-request-available (was: ) > Break-up schema evolution: Integrate the readers into

[PR] [HUDI-7567] Add schema evolution to fg reader [hudi]

2024-04-02 Thread via GitHub
jonvex opened a new pull request, #10957: URL: https://github.com/apache/hudi/pull/10957 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

Re: [PR] [DO NOT MERGE][HUDI-7566] Add schema evolution to spark file readers [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10956: URL: https://github.com/apache/hudi/pull/10956#issuecomment-2033015465 ## CI report: * 2f069604e8c23796aedd9eda8458a95a583db6d1 Azure:

Re: [PR] [HUDI-7498] Fix schema for HoodieTimestampAwareParquetInputFormat [hudi]

2024-04-02 Thread via GitHub
yihua commented on PR #10846: URL: https://github.com/apache/hudi/pull/10846#issuecomment-2033016182 @codope Have you made more progress on this fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10278: URL: https://github.com/apache/hudi/pull/10278#issuecomment-2033013929 ## CI report: * bf216d4e4bd9a24d35cf45690dd46dd7bea5910b Azure:

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10955: URL: https://github.com/apache/hudi/pull/10955#issuecomment-2033006127 ## CI report: * 0587be7239c626084fc8657ae55f3818519c568b Azure:

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2033006083 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d Azure:

Re: [PR] [DO NOT MERGE][HUDI-7566] Add schema evolution to spark file readers [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10956: URL: https://github.com/apache/hudi/pull/10956#issuecomment-2033006213 ## CI report: * 2f069604e8c23796aedd9eda8458a95a583db6d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [MINOR] use Temurin jdk [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10948: URL: https://github.com/apache/hudi/pull/10948#issuecomment-2033006013 ## CI report: * 3109fe81b4d356316fb2b2837270c226a36ccf50 Azure:

Re: [PR] [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10278: URL: https://github.com/apache/hudi/pull/10278#issuecomment-2033004783 ## CI report: * bf216d4e4bd9a24d35cf45690dd46dd7bea5910b Azure:

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2032995749 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [MINOR] use Temurin jdk [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10948: URL: https://github.com/apache/hudi/pull/10948#issuecomment-2032995684 ## CI report: * 3109fe81b4d356316fb2b2837270c226a36ccf50 Azure:

Re: [PR] [DO NOT MERGE][HUDI-7566] Add schema evolution to spark file readers [hudi]

2024-04-02 Thread via GitHub
jonvex closed pull request #10956: [DO NOT MERGE][HUDI-7566] Add schema evolution to spark file readers URL: https://github.com/apache/hudi/pull/10956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Updated] (HUDI-7566) Break-up schema evolution: add schema evolution changes to ported spark

2024-04-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7566: - Labels: pull-request-available (was: ) > Break-up schema evolution: add schema evolution changes

[PR] [HUDI-7566] Add schema evolution to spark file readers [hudi]

2024-04-02 Thread via GitHub
jonvex opened a new pull request, #10956: URL: https://github.com/apache/hudi/pull/10956 ### Change Logs Subtask of https://issues.apache.org/jira/browse/HUDI-7045 Extracts from https://github.com/apache/hudi/pull/10278 This pr adds in schema evolution to the file readers.

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2032852190 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d Azure:

Re: [PR] [MINOR] Upgrade mockito to 3.12.4 [hudi]

2024-04-02 Thread via GitHub
yihua merged PR #10953: URL: https://github.com/apache/hudi/pull/10953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch master updated: [MINOR] Upgrade mockito to 3.12.4 (#10953)

2024-04-02 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new bb51aca75d0 [MINOR] Upgrade mockito to 3.12.4

[jira] [Commented] (HUDI-7559) Fix functional index (on column stats): Handle NPE in filterQueriesWithRecordKey(...)

2024-04-02 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833284#comment-17833284 ] Sagar Sumit commented on HUDI-7559: --- beta1 testing was based on range query instead of equality check.

Re: [I] [SUPPORT] Requesting Support for insert_overwrite in Delta Streamer [hudi]

2024-04-02 Thread via GitHub
soumilshah1995 commented on issue #10896: URL: https://github.com/apache/hudi/issues/10896#issuecomment-2032790976 ill send email [d...@hudi.apache.org](mailto:d...@hudi.apache.org) ill close this thread -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] [SUPPORT] Requesting Support for insert_overwrite in Delta Streamer [hudi]

2024-04-02 Thread via GitHub
soumilshah1995 closed issue #10896: [SUPPORT] Requesting Support for insert_overwrite in Delta Streamer URL: https://github.com/apache/hudi/issues/10896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [SUPPORT] Requesting Support for insert_overwrite in Delta Streamer [hudi]

2024-04-02 Thread via GitHub
soumilshah1995 commented on issue #10896: URL: https://github.com/apache/hudi/issues/10896#issuecomment-2032780049 you want me to close this ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [SUPPORT] Requesting Support for insert_overwrite in Delta Streamer [hudi]

2024-04-02 Thread via GitHub
soumilshah1995 commented on issue #10896: URL: https://github.com/apache/hudi/issues/10896#issuecomment-2032779435 Roger that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10955: URL: https://github.com/apache/hudi/pull/10955#issuecomment-2032724837 ## CI report: * 0587be7239c626084fc8657ae55f3818519c568b Azure:

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2032724723 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d Azure:

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10955: URL: https://github.com/apache/hudi/pull/10955#issuecomment-2032709320 ## CI report: * 0587be7239c626084fc8657ae55f3818519c568b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10954: URL: https://github.com/apache/hudi/pull/10954#issuecomment-2032709221 ## CI report: * a20e9d4c236a04becc36724f22972c8eb925c15d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

[jira] [Updated] (HUDI-7559) Fix functional index (on column stats): Handle NPE in filterQueriesWithRecordKey(...)

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat updated HUDI-7559: -- Story Points: 4 > Fix functional index (on column stats): Handle NPE in >

[jira] [Assigned] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat reassigned HUDI-7569: - Story Points: 4 Assignee: Vinaykumar Bhat > Fix wrong result while using RLI for

Re: [PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bhat-vinay commented on PR #10955: URL: https://github.com/apache/hudi/pull/10955#issuecomment-2032614653 @codope tagging you for a review. Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7569: - Labels: hudi-1.0.0-beta2 pull-request-available (was: hudi-1.0.0-beta2) > Fix wrong result while

[PR] [HUDI-7569] [RLI] Fix wrong result generated by query [hudi]

2024-04-02 Thread via GitHub
bhat-vinay opened a new pull request, #10955: URL: https://github.com/apache/hudi/pull/10955 Record level index speeds up queries (when appropriate config properties are enabled) by pruning files based on metadata's RLI partition entries. The current implementation can prune files only

[jira] [Updated] (HUDI-7565) Break-up schema evolution: port spark code to file readers

2024-04-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7565: - Labels: pull-request-available (was: ) > Break-up schema evolution: port spark code to file

[PR] [HUDI-7565] Create spark file readers to read a single file instead of an entire partition [hudi]

2024-04-02 Thread via GitHub
jonvex opened a new pull request, #10954: URL: https://github.com/apache/hudi/pull/10954 ### Change Logs Subtask of https://issues.apache.org/jira/browse/HUDI-7045 Extracts from https://github.com/apache/hudi/pull/10278 Spark parquet readers are created per partition. We want to

[jira] [Updated] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat updated HUDI-7569: -- Epic Link: HUDI-512 > Fix wrong result while using RLI for pruning files >

[jira] [Updated] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat updated HUDI-7569: -- Fix Version/s: 1.0.0 > Fix wrong result while using RLI for pruning files >

[jira] [Updated] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinaykumar Bhat updated HUDI-7569: -- Labels: hudi-1.0.0-beta2 (was: ) > Fix wrong result while using RLI for pruning files >

Re: [PR] [MINOR] upgrade mockito to 3.12.4 [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10953: URL: https://github.com/apache/hudi/pull/10953#issuecomment-2032494713 ## CI report: * a77326b33b311666c4bf8da1233040e37ef19746 Azure:

[jira] [Updated] (HUDI-7570) Update RFC with details on API changes

2024-04-02 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7570: -- Sprint: Sprint 2024-03-25 > Update RFC with details on API changes >

[jira] [Assigned] (HUDI-7570) Update RFC with details on API changes

2024-04-02 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7570: - Assignee: Sagar Sumit > Update RFC with details on API changes >

[jira] [Created] (HUDI-7570) Update RFC with details on API changes

2024-04-02 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7570: - Summary: Update RFC with details on API changes Key: HUDI-7570 URL: https://issues.apache.org/jira/browse/HUDI-7570 Project: Apache Hudi Issue Type: Task

[jira] [Created] (HUDI-7569) Fix wrong result while using RLI for pruning files

2024-04-02 Thread Vinaykumar Bhat (Jira)
Vinaykumar Bhat created HUDI-7569: - Summary: Fix wrong result while using RLI for pruning files Key: HUDI-7569 URL: https://issues.apache.org/jira/browse/HUDI-7569 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-6331) Update user doc of partial update for MERGE INTO

2024-04-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-6331: - Fix Version/s: 0.15.0 > Update user doc of partial update for MERGE INTO >

[jira] [Closed] (HUDI-6331) Update user doc of partial update for MERGE INTO

2024-04-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-6331. Resolution: Fixed > Update user doc of partial update for MERGE INTO >

Re: [PR] [HUDI-6331][DOCS] Update user doc of partial insert in MERGE INTO command [hudi]

2024-04-02 Thread via GitHub
xushiyan merged PR #10952: URL: https://github.com/apache/hudi/pull/10952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch asf-site updated: [HUDI-6331][DOCS] Update user doc of partial insert in MERGE INTO command (#10952)

2024-04-02 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new baeecb45db9 [HUDI-6331][DOCS] Update user

Re: [PR] [MINOR] upgrade mockito to 3.12.4 [hudi]

2024-04-02 Thread via GitHub
hudi-bot commented on PR #10953: URL: https://github.com/apache/hudi/pull/10953#issuecomment-2032323727 ## CI report: * a77326b33b311666c4bf8da1233040e37ef19746 Azure:

[jira] [Updated] (HUDI-7565) Break-up schema evolution: port spark code to file readers

2024-04-02 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-7565: -- Due Date: 3/Apr/24 > Break-up schema evolution: port spark code to file readers >

[jira] [Updated] (HUDI-7566) Break-up schema evolution: add schema evolution changes to ported spark

2024-04-02 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-7566: -- Due Date: 4/Apr/24 > Break-up schema evolution: add schema evolution changes to ported spark >

[jira] [Updated] (HUDI-7567) Break-up schema evolution: Integrate the readers into the new fg reader

2024-04-02 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-7567: -- Due Date: 5/Apr/24 > Break-up schema evolution: Integrate the readers into the new fg reader >

[jira] [Created] (HUDI-7568) Performance test fg reader

2024-04-02 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-7568: - Summary: Performance test fg reader Key: HUDI-7568 URL: https://issues.apache.org/jira/browse/HUDI-7568 Project: Apache Hudi Issue Type: Sub-task

  1   2   >