[jira] [Closed] (HUDI-7811) Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

2024-05-31 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7811. - Resolution: Fixed Fixed in the original PR itself -

[jira] [Assigned] (HUDI-7811) Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

2024-05-31 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7811: - Assignee: Sagar Sumit > Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition

[jira] [Created] (HUDI-7820) For bloom index reader path, prune based on min/max if colstats is enabled

2024-05-31 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7820: - Summary: For bloom index reader path, prune based on min/max if colstats is enabled Key: HUDI-7820 URL: https://issues.apache.org/jira/browse/HUDI-7820 Project: Apache

[jira] [Closed] (HUDI-7407) Add optional clean support to standalone compaction and clustering jobs

2024-05-30 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7407. - Resolution: Fixed > Add optional clean support to standalone compaction and clustering jobs >

[jira] [Created] (HUDI-7811) Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

2024-05-29 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7811: - Summary: Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path Key: HUDI-7811 URL: https://issues.apache.org/jira/browse/HUDI-7811 Project: Apache Hudi

[jira] [Commented] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-21 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848149#comment-17848149 ] Sagar Sumit commented on HUDI-7779: --- {quote}Corner case to consider. Lets add onto above scenario, where

[jira] [Closed] (HUDI-7775) Remove unused APIs in HoodieStorage

2024-05-19 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7775. - Resolution: Fixed > Remove unused APIs in HoodieStorage > --- > >

[jira] [Closed] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-05-17 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7652. - Resolution: Done > Add new MergeKey API to support simple and composite keys >

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-05-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Reviewers: Ethan Guo, Jonathan Vexler (was: Jonathan Vexler) > Integrate functional index using bloom

[jira] [Updated] (HUDI-7661) Create index readme to show how a new index implementation can be added

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7661: -- Story Points: 1 (was: 0.5) > Create index readme to show how a new index implementation can be added >

[jira] [Closed] (HUDI-7705) Column name is wrong when generating partition stats index key

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7705. - Resolution: Fixed > Column name is wrong when generating partition stats index key >

[jira] [Closed] (HUDI-7589) Add API to create HoodieStorage in HoodieIOFactory

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7589. - Resolution: Fixed > Add API to create HoodieStorage in HoodieIOFactory >

[jira] [Updated] (HUDI-7528) Fix RowCustomColumnsSortPartitioner to use repartition instead of coalesce

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7528: -- Status: Open (was: Patch Available) > Fix RowCustomColumnsSortPartitioner to use repartition instead

[jira] [Closed] (HUDI-7712) Account for file slices instead of just base files while initializing RLI for MOR table

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7712. - Resolution: Fixed > Account for file slices instead of just base files while initializing RLI for > MOR

[jira] [Updated] (HUDI-7689) Allow users to leverage HoodieTable and Engine Context in Compaction Strategy

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7689: -- Status: Open (was: Patch Available) > Allow users to leverage HoodieTable and Engine Context in

[jira] [Closed] (HUDI-7673) Enhance RLI validation w/ MDT validator for false positives

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7673. - Resolution: Fixed > Enhance RLI validation w/ MDT validator for false positives >

[jira] [Closed] (HUDI-7617) Fix issues for bulk insert user defined partitioner in StreamSync

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7617. - Resolution: Fixed > Fix issues for bulk insert user defined partitioner in StreamSync >

[jira] [Closed] (HUDI-7532) Fix schedule compact to only consider DCs after last compaction commit

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7532. - Resolution: Fixed > Fix schedule compact to only consider DCs after last compaction commit >

[jira] [Closed] (HUDI-7535) Add metrics for source parallelism for Kafka and S3/GCS sources

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7535. - Resolution: Fixed > Add metrics for source parallelism for Kafka and S3/GCS sources >

[jira] [Closed] (HUDI-7523) Add HOODIE_SPARK_DATASOURCE_OPTIONS to be used in HoodieIncrSource

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7523. - Resolution: Fixed > Add HOODIE_SPARK_DATASOURCE_OPTIONS to be used in HoodieIncrSource >

[jira] [Closed] (HUDI-7508) Avoid converting iterator to list HoodieStreamerUtils.createHoodieRecords

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7508. - Resolution: Fixed > Avoid converting iterator to list HoodieStreamerUtils.createHoodieRecords >

[jira] [Closed] (HUDI-7501) Use source profile for S3 and GCS sources

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7501. - Resolution: Fixed > Use source profile for S3 and GCS sources > -

[jira] [Closed] (HUDI-7460) Fix compaction schedule with pending delta commits

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7460. - Resolution: Fixed > Fix compaction schedule with pending delta commits >

[jira] [Closed] (HUDI-7429) Fix avg record size estimation for delta commits and replace commits

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7429. - Resolution: Fixed > Fix avg record size estimation for delta commits and replace commits >

[jira] [Closed] (HUDI-7624) Fix index lookup duration to track tag location duration

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7624. - Resolution: Fixed > Fix index lookup duration to track tag location duration >

[jira] [Closed] (HUDI-7639) Refactor HoodieFileIndex so that different indexes can be used via optimizer rules

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7639. - Resolution: Fixed > Refactor HoodieFileIndex so that different indexes can be used via optimizer > rules

[jira] [Updated] (HUDI-7639) Refactor HoodieFileIndex so that different indexes can be used via optimizer rules

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7639: -- Status: Patch Available (was: In Progress) > Refactor HoodieFileIndex so that different indexes can be

[jira] [Closed] (HUDI-7654) Implement the pre-CBO rules

2024-05-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7654. - Resolution: Fixed > Implement the pre-CBO rules > --- > > Key:

[jira] [Assigned] (HUDI-7749) Upgrade Spark patch version to include a fix related to data correctness

2024-05-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7749: - Assignee: Sagar Sumit > Upgrade Spark patch version to include a fix related to data correctness

[jira] [Closed] (HUDI-7749) Upgrade Spark patch version to include a fix related to data correctness

2024-05-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7749. - Resolution: Done > Upgrade Spark patch version to include a fix related to data correctness >

[jira] [Updated] (HUDI-7749) Upgrade Spark patch version to include a fix related to data correctness

2024-05-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7749: -- Fix Version/s: 1.0.0 > Upgrade Spark patch version to include a fix related to data correctness >

[jira] [Commented] (HUDI-7475) Disable ITs in hudi-aws module

2024-05-14 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846233#comment-17846233 ] Sagar Sumit commented on HUDI-7475: --- Is there a way to fix the failing test? How was it working before?

[jira] [Updated] (HUDI-7749) Upgrade Spark patch version to include a fix related to data correctness

2024-05-13 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7749: -- Description: https://issues.apache.org/jira/browse/SPARK-44805 shows data correctness issue with Spark

[jira] [Created] (HUDI-7749) Upgrade Spark patch version to include a fix related to data correctness

2024-05-13 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7749: - Summary: Upgrade Spark patch version to include a fix related to data correctness Key: HUDI-7749 URL: https://issues.apache.org/jira/browse/HUDI-7749 Project: Apache Hudi

[jira] [Created] (HUDI-7748) Add logs and drop _hoodie_is_deleted in Transformer

2024-05-12 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7748: - Summary: Add logs and drop _hoodie_is_deleted in Transformer Key: HUDI-7748 URL: https://issues.apache.org/jira/browse/HUDI-7748 Project: Apache Hudi Issue Type:

[jira] [Closed] (HUDI-7721) Fix broken build on master

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7721. - Fix Version/s: 1.0.0 Resolution: Fixed > Fix broken build on master > -- >

[jira] [Updated] (HUDI-7146) Implement secondary index

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7146: -- Reviewers: Danny Chen > Implement secondary index > - > > Key:

[jira] [Updated] (HUDI-7639) Refactor HoodieFileIndex so that different indexes can be used via optimizer rules

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7639: -- Story Points: 5 > Refactor HoodieFileIndex so that different indexes can be used via optimizer > rules

[jira] [Commented] (HUDI-7701) Metadata table initailization with pending instants

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843681#comment-17843681 ] Sagar Sumit commented on HUDI-7701: --- Fixed in

[jira] [Updated] (HUDI-7701) Metadata table initailization with pending instants

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7701: -- Status: Patch Available (was: In Progress) > Metadata table initailization with pending instants >

[jira] [Closed] (HUDI-7701) Metadata table initailization with pending instants

2024-05-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7701. - Resolution: Fixed > Metadata table initailization with pending instants >

[jira] [Updated] (HUDI-7146) Implement secondary index

2024-05-03 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7146: -- Status: Patch Available (was: In Progress) > Implement secondary index > - > >

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-05-02 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Reviewers: Jonathan Vexler (was: Ethan Guo, Jonathan Vexler) > Integrate functional index using bloom

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-05-02 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Reviewers: Ethan Guo, Jonathan Vexler (was: Ethan Guo) > Integrate functional index using bloom filter

[jira] [Updated] (HUDI-7700) Support query hint to inject indexes in query plans

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7700: -- Sprint: Sprint 2023-04-26 > Support query hint to inject indexes in query plans >

[jira] [Updated] (HUDI-7700) Support query hint to inject indexes in query plans

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7700: -- Story Points: 6 > Support query hint to inject indexes in query plans >

[jira] [Created] (HUDI-7700) Support query hint to inject indexes in query plans

2024-05-01 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7700: - Summary: Support query hint to inject indexes in query plans Key: HUDI-7700 URL: https://issues.apache.org/jira/browse/HUDI-7700 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-7661) Create index readme to show how a new index implementation can be added

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7661: -- Summary: Create index readme to show how a new index implementation can be added (was: Update docs to

[jira] [Updated] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7661: -- Story Points: 0.5 (was: 1) > Update docs to show how a new index implementation can be added >

[jira] [Updated] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7661: -- Sprint: Sprint 2023-04-26 > Update docs to show how a new index implementation can be added >

[jira] [Assigned] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7661: - Assignee: Sagar Sumit > Update docs to show how a new index implementation can be added >

[jira] [Assigned] (HUDI-7696) Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7696: - Assignee: Sagar Sumit > Consolidate convertFilesToPartitionStatsRecords and >

[jira] [Updated] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7661: -- Story Points: 1 > Update docs to show how a new index implementation can be added >

[jira] [Assigned] (HUDI-7691) Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7691: - Assignee: Sagar Sumit > Move MDT partition type related logic in HoodieBackedTableMetadataWriter

[jira] [Updated] (HUDI-7692) Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7692: -- Sprint: Sprint 2023-04-26 > Move MDT partiiton type code in HoodieMetadataPaylaod to

[jira] [Updated] (HUDI-7696) Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7696: -- Sprint: Sprint 2023-04-26 > Consolidate convertFilesToPartitionStatsRecords and >

[jira] [Updated] (HUDI-7691) Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7691: -- Sprint: Sprint 2023-04-26 > Move MDT partition type related logic in HoodieBackedTableMetadataWriter to

[jira] [Assigned] (HUDI-7692) Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7692: - Assignee: Sagar Sumit > Move MDT partiiton type code in HoodieMetadataPaylaod to

[jira] [Updated] (HUDI-7690) Initialize all indexes in parallel instead of computing type by type.

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7690: -- Story Points: 2 > Initialize all indexes in parallel instead of computing type by type. >

[jira] [Updated] (HUDI-7662) Expose a config to enable disable functional index

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7662: -- Story Points: 1 > Expose a config to enable disable functional index >

[jira] [Updated] (HUDI-7696) Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7696: -- Story Points: 1 > Consolidate convertFilesToPartitionStatsRecords and >

[jira] [Updated] (HUDI-7691) Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7691: -- Story Points: 2 > Move MDT partition type related logic in HoodieBackedTableMetadataWriter to >

[jira] [Updated] (HUDI-7692) Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7692: -- Story Points: 1 > Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType >

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Reviewers: Ethan Guo > Integrate functional index using bloom filter on reader side >

[jira] [Closed] (HUDI-7144) Support query for tables written as partitionBy but synced as non-partitioned

2024-05-01 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7144. - Resolution: Done > Support query for tables written as partitionBy but synced as non-partitioned >

[jira] [Created] (HUDI-7696) Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords

2024-04-30 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7696: - Summary: Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords Key: HUDI-7696 URL: https://issues.apache.org/jira/browse/HUDI-7696

[jira] [Created] (HUDI-7692) Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType

2024-04-30 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7692: - Summary: Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType Key: HUDI-7692 URL: https://issues.apache.org/jira/browse/HUDI-7692 Project: Apache

[jira] [Created] (HUDI-7691) Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType

2024-04-30 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7691: - Summary: Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType Key: HUDI-7691 URL: https://issues.apache.org/jira/browse/HUDI-7691

[jira] [Created] (HUDI-7690) Initialize all indexes in parallel instead of computing type by type.

2024-04-30 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7690: - Summary: Initialize all indexes in parallel instead of computing type by type. Key: HUDI-7690 URL: https://issues.apache.org/jira/browse/HUDI-7690 Project: Apache Hudi

[jira] [Updated] (HUDI-7026) Create functional indexes on multiple columns with a single create index command

2024-04-29 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7026: -- Priority: Minor (was: Major) > Create functional indexes on multiple columns with a single create

[jira] [Closed] (HUDI-7579) Functional index (on col stats) creation fails to process all files/partitions

2024-04-29 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7579. - Resolution: Fixed > Functional index (on col stats) creation fails to process all files/partitions >

[jira] [Closed] (HUDI-7667) Create util method to get offset range for fetching new data in KafkaSource

2024-04-29 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7667. - Fix Version/s: 1.0.0 Resolution: Fixed > Create util method to get offset range for fetching new

[jira] [Created] (HUDI-7683) Fix noisy info log from HoodieMetadataMetrics

2024-04-28 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7683: - Summary: Fix noisy info log from HoodieMetadataMetrics Key: HUDI-7683 URL: https://issues.apache.org/jira/browse/HUDI-7683 Project: Apache Hudi Issue Type: Task

[jira] [Closed] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7648. - Resolution: Fixed > Refactor MetadataPartitionType so as to enahance reuse across metadata writer >

[jira] [Updated] (HUDI-7144) Support query for tables written as partitionBy but synced as non-partitioned

2024-04-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7144: -- Story Points: 8 (was: 2) > Support query for tables written as partitionBy but synced as

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-04-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Status: Patch Available (was: In Progress) > Integrate functional index using bloom filter on reader

[jira] [Updated] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-04-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7007: -- Story Points: 6 (was: 4) > Integrate functional index using bloom filter on reader side >

[jira] [Created] (HUDI-7665) Upgrade Table Version

2024-04-25 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7665: - Summary: Upgrade Table Version Key: HUDI-7665 URL: https://issues.apache.org/jira/browse/HUDI-7665 Project: Apache Hudi Issue Type: Task Reporter:

[jira] [Created] (HUDI-7662) Expose a config to enable disable functional index

2024-04-24 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7662: - Summary: Expose a config to enable disable functional index Key: HUDI-7662 URL: https://issues.apache.org/jira/browse/HUDI-7662 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-04-24 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7661: - Summary: Update docs to show how a new index implementation can be added Key: HUDI-7661 URL: https://issues.apache.org/jira/browse/HUDI-7661 Project: Apache Hudi

[jira] [Updated] (HUDI-7651) Add util methods for creating meta client

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7651: -- Reviewers: Sagar Sumit > Add util methods for creating meta client >

[jira] [Updated] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7652: -- Reviewers: Danny Chen, Ethan Guo > Add new MergeKey API to support simple and composite keys >

[jira] [Updated] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7652: -- Status: Patch Available (was: In Progress) > Add new MergeKey API to support simple and composite keys

[jira] [Updated] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7652: -- Status: In Progress (was: Open) > Add new MergeKey API to support simple and composite keys >

[jira] [Closed] (HUDI-7653) Refactor HoodieFileIndex for more flexibility

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7653. - Resolution: Done > Refactor HoodieFileIndex for more flexibility >

[jira] [Updated] (HUDI-7653) Refactor HoodieFileIndex for more flexibility

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7653: -- Status: Patch Available (was: In Progress) > Refactor HoodieFileIndex for more flexibility >

[jira] [Updated] (HUDI-7653) Refactor HoodieFileIndex for more flexibility

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7653: -- Fix Version/s: 1.0.0 > Refactor HoodieFileIndex for more flexibility >

[jira] [Updated] (HUDI-7653) Refactor HoodieFileIndex for more flexibility

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7653: -- Labels: hudi-1.0.0-beta2 pull-request-available (was: pull-request-available) > Refactor

[jira] [Updated] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7652: -- Sprint: Sprint 2024-03-25 > Add new MergeKey API to support simple and composite keys >

[jira] [Created] (HUDI-7652) Add new MergeKey API to support simple and composite keys

2024-04-23 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7652: - Summary: Add new MergeKey API to support simple and composite keys Key: HUDI-7652 URL: https://issues.apache.org/jira/browse/HUDI-7652 Project: Apache Hudi Issue

[jira] [Assigned] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7648: - Assignee: Sagar Sumit > Refactor MetadataPartitionType so as to enahance reuse across metadata

[jira] [Updated] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7648: -- Reviewers: Ethan Guo > Refactor MetadataPartitionType so as to enahance reuse across metadata writer >

[jira] [Updated] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7648: -- Status: Patch Available (was: In Progress) > Refactor MetadataPartitionType so as to enahance reuse

[jira] [Updated] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7648: -- Sprint: Sprint 2024-03-25 > Refactor MetadataPartitionType so as to enahance reuse across metadata

[jira] [Created] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7648: - Summary: Refactor MetadataPartitionType so as to enahance reuse across metadata writer Key: HUDI-7648 URL: https://issues.apache.org/jira/browse/HUDI-7648 Project: Apache

[jira] [Updated] (HUDI-7648) Refactor MetadataPartitionType so as to enahance reuse across metadata writer

2024-04-22 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7648: -- Status: In Progress (was: Open) > Refactor MetadataPartitionType so as to enahance reuse across

[jira] [Created] (HUDI-7643) Fix TestStreamSyncUnitTests

2024-04-19 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7643: - Summary: Fix TestStreamSyncUnitTests Key: HUDI-7643 URL: https://issues.apache.org/jira/browse/HUDI-7643 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-7498) Fix schema for HoodieTimestampAwareParquetInputFormat

2024-04-18 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7498: -- Fix Version/s: 0.15.0 > Fix schema for HoodieTimestampAwareParquetInputFormat >

[jira] [Created] (HUDI-7639) Refactor HoodieFileIndex so that different indexes can be used via optimizer rules

2024-04-18 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7639: - Summary: Refactor HoodieFileIndex so that different indexes can be used via optimizer rules Key: HUDI-7639 URL: https://issues.apache.org/jira/browse/HUDI-7639 Project:

[jira] [Assigned] (HUDI-7007) Integrate functional index using bloom filter on reader side

2024-04-17 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7007: - Assignee: Sagar Sumit (was: Vinaykumar Bhat) > Integrate functional index using bloom filter on

  1   2   3   4   5   6   7   8   9   10   >