yihua commented on code in PR #9142: URL: https://github.com/apache/hudi/pull/9142#discussion_r1255981238
########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | +| Supervised table service planning and execution | 1.0.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | +| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | +| Secondary index to improve query performance | 1.0.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | +| Index Function for Optimizing Query Performance | 1.0.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | | Lock-Free message queue to improve writing efficiency | 0.13.0 | [RFC-53](https://github.com/apache/hudi/blob/master/rfc/rfc-53/rfc-53.md), [HUDI-3963](https://issues.apache.org/jira/browse/HUDI-3963) | Review Comment: This is done in 0.13.0. ########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | +| Supervised table service planning and execution | 1.0.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | +| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | +| Secondary index to improve query performance | 1.0.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | +| Index Function for Optimizing Query Performance | 1.0.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | | Lock-Free message queue to improve writing efficiency | 0.13.0 | [RFC-53](https://github.com/apache/hudi/blob/master/rfc/rfc-53/rfc-53.md), [HUDI-3963](https://issues.apache.org/jira/browse/HUDI-3963) | -| Efficient bootstrap and migration of existing non-Hudi dataset | 1.0.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | +| Efficient bootstrap and migration of existing non-Hudi dataset | 0.14.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | | Lock free concurrency control | 1.0.0 onward | [HUDI-3187](https://issues.apache.org/jira/browse/HUDI-3187) | | Non-blocking/Lock-free updates during clustering | 1.0.0 onward | [HUDI-1042](https://issues.apache.org/jira/browse/HUDI-1042) | | Time Travel updates, deletes | 1.0.0 onward || | General purpose support for multi-table transactions | 1.0.0 onward || +| Lockless multi-writer support | 1.0.0 | [RFC-66](https://github.com/apache/hudi/pull/7907) | +| A more effective HoodieMergeHandler for COW table with parque | 1.0.0 | [RFC-68](https://github.com/apache/hudi/blob/f1afb1bf04abdc94a26d61dc302f36ec2bbeb15b/rfc/rfc-68/rfc-68.md) | + ## Programming APIs |Feature| Target Release |Tracking| |------------|----------------|-----------| -| Redesign and optimization of record payload abstraction | 0.13.0 | [RFC-46](https://github.com/apache/hudi/blob/master/rfc/rfc-46/rfc-46.md), [HUDI-3217](https://issues.apache.org/jira/browse/HUDI-3217) | -| Optimized storage layout for cloud object stores | 0.13.0 | [RFC-60](https://github.com/apache/hudi/pull/5113), [HUDI-3625](https://issues.apache.org/jira/browse/HUDI-3625) | -| Support of verification with multiple event_time fields | 1.0.0 onward | [RFC-59](https://github.com/apache/hudi/pull/6382), [HUDI-4569](https://issues.apache.org/jira/browse/HUDI-4569) | +| New Hudi Table Spec API for Query Integrations | 1.0.0 | [RFC-64](https://github.com/apache/hudi/pull/7080) | +| Snapshot view management | 1.0.0 | [RFC-61](https://github.com/apache/hudi/pull/6576), [HUDI-4678](https://issues.apache.org/jira/browse/HUDI-4678) | +| Engine specific record representation for better performance | 1.0.0 | [RFC-46](https://github.com/apache/hudi/blob/master/rfc/rfc-46/rfc-46.md), [HUDI-3217](https://issues.apache.org/jira/browse/HUDI-3217) | +| Optimized storage layout for cloud object stores | 1.0.0 | [RFC-60](https://github.com/apache/hudi/pull/5113), [HUDI-3625](https://issues.apache.org/jira/browse/HUDI-3625) | +| Support of verification with multiple event_time fields | 1.0.0 onward | [RFC-59](https://github.com/apache/hudi/pull/6382), [HUDI-4569](https://issues.apache.org/jira/browse/HUDI-4569) | ## Execution Engine Integration | Feature | Target Release | Tracking | |------------------------------------------------------------------------------------------------|--------------|-----------------------------------------------------------------------------------------------------------------| -| Integrate column stats index with all query engines | 0.13.0 | [RFC-58](https://github.com/apache/hudi/pull/6345), [HUDI-4552](https://issues.apache.org/jira/browse/HUDI-4552) | -| Upgrade to Spark 3 as the default profile | 0.13.0 | [HUDI-3431](https://issues.apache.org/jira/browse/HUDI-3431) | +| Support for Spark 3.4.0 | 0.14.0 | [HUDI-6198](https://issues.apache.org/jira/browse/HUDI-6198) | +| Integrate column stats index with all query engines | 0.14.0 | [RFC-58](https://github.com/apache/hudi/pull/6345), [HUDI-4552](https://issues.apache.org/jira/browse/HUDI-4552) | +| Upgrade to Spark 3 as the default profile | 0.14.0 | [HUDI-3431](https://issues.apache.org/jira/browse/HUDI-3431) | +| Performance for Spark-SQL writes | 0.14.0 | [HUDI-6315](https://issues.apache.org/jira/browse/HUDI-6315), [HUDI-6376](https://issues.apache.org/jira/browse/HUDI-6376) | | Spark datasource V2 read | 1.0.0 onward | [HUDI-4449](https://issues.apache.org/jira/browse/HUDI-4449) | | Materialized Views with incremental updates using Flink | 1.0.0 onward || | SQL DML support for Presto/Trino connectors (could be accelerated based on community feedback) | 1.0.0 onward || Review Comment: We don't plan to do this in near future. We should remove this entry. ########## website/src/pages/roadmap.md: ########## @@ -61,8 +64,10 @@ Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versi | Feature | Target Release | Tracking | |-----------------------------------------------------------------------------------------------------|--------------|-----------------------------------------------------------------------------------------------------------------------------------------| -| Protobuf source for DeltaStreamer | 0.13.0 | [RFC-57](https://github.com/apache/hudi/blob/master/rfc/rfc-57/rfc-57.md), [HUDI-4399](https://issues.apache.org/jira/browse/HUDI-4399) | -| Improving Hudi CLI features and usability | 0.13.0 | [HUDI-1388](https://issues.apache.org/jira/browse/HUDI-1388) | +| Improving Hudi CLI features and usability | 0.14.0 | [HUDI-1388](https://issues.apache.org/jira/browse/HUDI-1388) | Review Comment: This is done in 0.13.0. ########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | +| Supervised table service planning and execution | 1.0.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | +| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | +| Secondary index to improve query performance | 1.0.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | +| Index Function for Optimizing Query Performance | 1.0.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | | Lock-Free message queue to improve writing efficiency | 0.13.0 | [RFC-53](https://github.com/apache/hudi/blob/master/rfc/rfc-53/rfc-53.md), [HUDI-3963](https://issues.apache.org/jira/browse/HUDI-3963) | -| Efficient bootstrap and migration of existing non-Hudi dataset | 1.0.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | +| Efficient bootstrap and migration of existing non-Hudi dataset | 0.14.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | | Lock free concurrency control | 1.0.0 onward | [HUDI-3187](https://issues.apache.org/jira/browse/HUDI-3187) | | Non-blocking/Lock-free updates during clustering | 1.0.0 onward | [HUDI-1042](https://issues.apache.org/jira/browse/HUDI-1042) | | Time Travel updates, deletes | 1.0.0 onward || | General purpose support for multi-table transactions | 1.0.0 onward || +| Lockless multi-writer support | 1.0.0 | [RFC-66](https://github.com/apache/hudi/pull/7907) | +| A more effective HoodieMergeHandler for COW table with parque | 1.0.0 | [RFC-68](https://github.com/apache/hudi/blob/f1afb1bf04abdc94a26d61dc302f36ec2bbeb15b/rfc/rfc-68/rfc-68.md) | + ## Programming APIs |Feature| Target Release |Tracking| |------------|----------------|-----------| -| Redesign and optimization of record payload abstraction | 0.13.0 | [RFC-46](https://github.com/apache/hudi/blob/master/rfc/rfc-46/rfc-46.md), [HUDI-3217](https://issues.apache.org/jira/browse/HUDI-3217) | -| Optimized storage layout for cloud object stores | 0.13.0 | [RFC-60](https://github.com/apache/hudi/pull/5113), [HUDI-3625](https://issues.apache.org/jira/browse/HUDI-3625) | -| Support of verification with multiple event_time fields | 1.0.0 onward | [RFC-59](https://github.com/apache/hudi/pull/6382), [HUDI-4569](https://issues.apache.org/jira/browse/HUDI-4569) | +| New Hudi Table Spec API for Query Integrations | 1.0.0 | [RFC-64](https://github.com/apache/hudi/pull/7080) | Review Comment: We landed this in 0.14.0: MOR snapshot reader to integrate with query engines without using Hadoop APIs https://github.com/apache/hudi/pull/9066 . Is it worth mentioning? ########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | +| Supervised table service planning and execution | 1.0.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | +| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | +| Secondary index to improve query performance | 1.0.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | +| Index Function for Optimizing Query Performance | 1.0.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | | Lock-Free message queue to improve writing efficiency | 0.13.0 | [RFC-53](https://github.com/apache/hudi/blob/master/rfc/rfc-53/rfc-53.md), [HUDI-3963](https://issues.apache.org/jira/browse/HUDI-3963) | -| Efficient bootstrap and migration of existing non-Hudi dataset | 1.0.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | +| Efficient bootstrap and migration of existing non-Hudi dataset | 0.14.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | | Lock free concurrency control | 1.0.0 onward | [HUDI-3187](https://issues.apache.org/jira/browse/HUDI-3187) | | Non-blocking/Lock-free updates during clustering | 1.0.0 onward | [HUDI-1042](https://issues.apache.org/jira/browse/HUDI-1042) | | Time Travel updates, deletes | 1.0.0 onward || | General purpose support for multi-table transactions | 1.0.0 onward || +| Lockless multi-writer support | 1.0.0 | [RFC-66](https://github.com/apache/hudi/pull/7907) | +| A more effective HoodieMergeHandler for COW table with parque | 1.0.0 | [RFC-68](https://github.com/apache/hudi/blob/f1afb1bf04abdc94a26d61dc302f36ec2bbeb15b/rfc/rfc-68/rfc-68.md) | Review Comment: ```suggestion | A more effective HoodieMergeHandler for COW table with parquet | 1.0.0 | [RFC-68](https://github.com/apache/hudi/blob/f1afb1bf04abdc94a26d61dc302f36ec2bbeb15b/rfc/rfc-68/rfc-68.md) | ``` ########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | Review Comment: Major part of metaserver is already implemented and released in 0.13.0. We should remove it. ########## website/src/pages/roadmap.md: ########## @@ -10,48 +10,51 @@ down by areas on our [stack](blog/2021/07/21/streaming-data-lake-platform/#hudi- ## Future Releases -Next major release : [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) (Jan 2023) +Next major release : [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) (July 2023) | Release | Timeline | |--------------------------------------------------------------------------|----------| -| [0.12.2](https://issues.apache.org/jira/projects/HUDI/versions/12352249) | Dec 2022 | -| [0.13.0](https://issues.apache.org/jira/projects/HUDI/versions/12352101) | Jan 2023 | +| [0.14.0](https://issues.apache.org/jira/projects/HUDI/versions/12352700) | July 2023 | | [1.0.0](https://issues.apache.org/jira/projects/HUDI/versions/12351210) | TBD | ## Transactional Database Layer | Feature | Target Release | Tracking | |----------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Metaserver for all metadata | 0.13.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | -| Supervised table service planning and execution | 0.13.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | -| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | [RFC-51](https://github.com/apache/hudi/blob/master/rfc/rfc-51/rfc-51.md), [HUDI-3478](https://issues.apache.org/jira/browse/HUDI-3478) | -| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | -| Consistent hashing index for dynamic buckets | 0.13.0 | [RFC-42](https://github.com/apache/hudi/blob/master/rfc/rfc-42/rfc-42.md), [HUDI-3000](https://issues.apache.org/jira/browse/HUDI-3000) | -| Secondary index to improve query performance | 0.13.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | -| Reducing write amplification with Log Compaction in MOR | 0.13.0 | [RFC-48](https://github.com/apache/hudi/pull/5041), [HUDI-3580](https://issues.apache.org/jira/browse/HUDI-3580) | -| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | [RFC-56](https://github.com/apache/hudi/pull/6003), [HUDI-1575](https://issues.apache.org/jira/browse/HUDI-1575) | -| Index Function for Optimizing Query Performance | 0.13.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | +| Support for primary key-less table | 0.14.0 | [HUDI-4699](https://issues.apache.org/jira/browse/HUDI-4699) | +| Metaserver for all metadata | 1.0.0 | [Strawman design](https://docs.google.com/presentation/d/1QBgLw11TM2Qf1KUESofGrQDb63EuggNCpPaxc82Kldo/edit#slide=id.ged084e5bf6_0_278), [RFC-36](https://github.com/apache/hudi/pull/4718), [HUDI-3345](https://issues.apache.org/jira/browse/HUDI-3345) | +| Supervised table service planning and execution | 1.0.0 | [RFC-43](https://github.com/apache/hudi/pull/4309), [HUDI-4147](https://issues.apache.org/jira/browse/HUDI-4147) | +| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | [RFC-08](https://cwiki.apache.org/confluence/display/HUDI/RFC-08++Record+level+indexing+mechanisms+for+Hudi+datasets), [HUDI-53](https://issues.apache.org/jira/browse/HUDI-53) | +| Secondary index to improve query performance | 1.0.0 | [RFC-52](https://github.com/apache/hudi/pull/5370), [HUDI-3907](https://issues.apache.org/jira/browse/HUDI-3907) | +| Index Function for Optimizing Query Performance | 1.0.0 | [RFC-63](https://github.com/apache/hudi/pull/7235), [HUDI-512](https://issues.apache.org/jira/browse/HUDI-512) | | Lock-Free message queue to improve writing efficiency | 0.13.0 | [RFC-53](https://github.com/apache/hudi/blob/master/rfc/rfc-53/rfc-53.md), [HUDI-3963](https://issues.apache.org/jira/browse/HUDI-3963) | -| Efficient bootstrap and migration of existing non-Hudi dataset | 1.0.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | +| Efficient bootstrap and migration of existing non-Hudi dataset | 0.14.0 onward | [HUDI-1265](https://issues.apache.org/jira/browse/HUDI-1265) | | Lock free concurrency control | 1.0.0 onward | [HUDI-3187](https://issues.apache.org/jira/browse/HUDI-3187) | | Non-blocking/Lock-free updates during clustering | 1.0.0 onward | [HUDI-1042](https://issues.apache.org/jira/browse/HUDI-1042) | | Time Travel updates, deletes | 1.0.0 onward || | General purpose support for multi-table transactions | 1.0.0 onward || +| Lockless multi-writer support | 1.0.0 | [RFC-66](https://github.com/apache/hudi/pull/7907) | +| A more effective HoodieMergeHandler for COW table with parque | 1.0.0 | [RFC-68](https://github.com/apache/hudi/blob/f1afb1bf04abdc94a26d61dc302f36ec2bbeb15b/rfc/rfc-68/rfc-68.md) | + ## Programming APIs |Feature| Target Release |Tracking| |------------|----------------|-----------| -| Redesign and optimization of record payload abstraction | 0.13.0 | [RFC-46](https://github.com/apache/hudi/blob/master/rfc/rfc-46/rfc-46.md), [HUDI-3217](https://issues.apache.org/jira/browse/HUDI-3217) | -| Optimized storage layout for cloud object stores | 0.13.0 | [RFC-60](https://github.com/apache/hudi/pull/5113), [HUDI-3625](https://issues.apache.org/jira/browse/HUDI-3625) | -| Support of verification with multiple event_time fields | 1.0.0 onward | [RFC-59](https://github.com/apache/hudi/pull/6382), [HUDI-4569](https://issues.apache.org/jira/browse/HUDI-4569) | +| New Hudi Table Spec API for Query Integrations | 1.0.0 | [RFC-64](https://github.com/apache/hudi/pull/7080) | +| Snapshot view management | 1.0.0 | [RFC-61](https://github.com/apache/hudi/pull/6576), [HUDI-4678](https://issues.apache.org/jira/browse/HUDI-4678) | +| Engine specific record representation for better performance | 1.0.0 | [RFC-46](https://github.com/apache/hudi/blob/master/rfc/rfc-46/rfc-46.md), [HUDI-3217](https://issues.apache.org/jira/browse/HUDI-3217) | Review Comment: This is released in 0.13.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
