This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new b075ac471e Minor: consolidate doc roadmap pages (#11426)
b075ac471e is described below
commit b075ac471e6d27dfe40b6586a72070a9ec4751a9
Author: Andrew Lamb <[email protected]>
AuthorDate: Fri Jul 12 15:27:16 2024 -0400
Minor: consolidate doc roadmap pages (#11426)
---
docs/source/contributor-guide/quarterly_roadmap.md | 96 ----------------------
docs/source/contributor-guide/roadmap.md | 81 ++++++++++++++++++
docs/source/index.rst | 1 -
3 files changed, 81 insertions(+), 97 deletions(-)
diff --git a/docs/source/contributor-guide/quarterly_roadmap.md
b/docs/source/contributor-guide/quarterly_roadmap.md
deleted file mode 100644
index ee82617225..0000000000
--- a/docs/source/contributor-guide/quarterly_roadmap.md
+++ /dev/null
@@ -1,96 +0,0 @@
-<!---
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-
-# Quarterly Roadmap
-
-A quarterly roadmap will be published to give the DataFusion community
visibility into the priorities of the projects contributors. This roadmap is
not binding.
-
-## 2023 Q4
-
-- Improve data output (`COPY`, `INSERT` and DataFrame) output capability
[#6569](https://github.com/apache/datafusion/issues/6569)
-- Implementation of `ARRAY` types and related functions
[#6980](https://github.com/apache/datafusion/issues/6980)
-- Write an industrial paper about DataFusion for SIGMOD
[#6782](https://github.com/apache/datafusion/issues/6782)
-
-## 2022 Q2
-
-### DataFusion Core
-
-- IO Improvements
- - Reading, registering, and writing more file formats from both DataFrame
API and SQL
- - Additional options for IO including partitioning and metadata support
-- Work Scheduling
- - Improve predictability, observability and performance of IO and CPU-bound
work
- - Develop a more explicit story for managing parallelism during plan
execution
-- Memory Management
- - Add more operators for memory limited execution
-- Performance
- - Incorporate row-format into operators such as aggregate
- - Add row-format benchmarks
- - Explore JIT-compiling complex expressions
- - Explore LLVM for JIT, with inline Rust functions as the primary goal
- - Improve performance of Sort and Merge using Row Format / JIT expressions
-- Documentation
- - General improvements to DataFusion website
- - Publish design documents
-- Streaming
- - Create `StreamProvider` trait
-
-### Ballista
-
-- Make production ready
- - Shuffle file cleanup
- - Fill functional gaps between DataFusion and Ballista
- - Improve task scheduling and data exchange efficiency
- - Better error handling
- - Task failure
- - Executor lost
- - Schedule restart
- - Improve monitoring and logging
- - Auto scaling support
-- Support for multi-scheduler deployments. Initially for resiliency and fault
tolerance but ultimately to support sharding for scalability and more efficient
caching.
-- Executor deployment grouping based on resource allocation
-
-### Extensions ([datafusion-contrib](https://github.com/datafusion-contrib))
-
-####
[DataFusion-Python](https://github.com/datafusion-contrib/datafusion-python)
-
-- Add missing functionality to DataFrame and SessionContext
-- Improve documentation
-
-####
[DataFusion-S3](https://github.com/datafusion-contrib/datafusion-objectstore-s3)
-
-- Create Python bindings to use with datafusion-python
-
-#### [DataFusion-Tui](https://github.com/datafusion-contrib/datafusion-tui)
-
-- Create multiple SQL editors
-- Expose more Context and query metadata
-- Support new data sources
- - BigTable, HDFS, HTTP APIs
-
-####
[DataFusion-BigTable](https://github.com/datafusion-contrib/datafusion-bigtable)
-
-- Python binding to use with datafusion-python
-- Timestamp range predicate pushdown
-- Multi-threaded partition aware execution
-- Production ready Rust SDK
-
-####
[DataFusion-Streams](https://github.com/datafusion-contrib/datafusion-streams)
-
-- Create experimental implementation of `StreamProvider` trait
diff --git a/docs/source/contributor-guide/roadmap.md
b/docs/source/contributor-guide/roadmap.md
index a6d78d9311..3d9c1ee371 100644
--- a/docs/source/contributor-guide/roadmap.md
+++ b/docs/source/contributor-guide/roadmap.md
@@ -43,3 +43,84 @@ start a conversation using a github issue or the
make review efficient and avoid surprises.
[The current list of `EPIC`s can be found
here](https://github.com/apache/datafusion/issues?q=is%3Aissue+is%3Aopen+epic).
+
+# Quarterly Roadmap
+
+A quarterly roadmap will be published to give the DataFusion community
+visibility into the priorities of the projects contributors. This roadmap is
not
+binding and we would welcome any/all contributions to help keep this list up to
+date.
+
+## 2023 Q4
+
+- Improve data output (`COPY`, `INSERT` and DataFrame) output capability
[#6569](https://github.com/apache/datafusion/issues/6569)
+- Implementation of `ARRAY` types and related functions
[#6980](https://github.com/apache/datafusion/issues/6980)
+- Write an industrial paper about DataFusion for SIGMOD
[#6782](https://github.com/apache/datafusion/issues/6782)
+
+## 2022 Q2
+
+### DataFusion Core
+
+- IO Improvements
+ - Reading, registering, and writing more file formats from both DataFrame
API and SQL
+ - Additional options for IO including partitioning and metadata support
+- Work Scheduling
+ - Improve predictability, observability and performance of IO and CPU-bound
work
+ - Develop a more explicit story for managing parallelism during plan
execution
+- Memory Management
+ - Add more operators for memory limited execution
+- Performance
+ - Incorporate row-format into operators such as aggregate
+ - Add row-format benchmarks
+ - Explore JIT-compiling complex expressions
+ - Explore LLVM for JIT, with inline Rust functions as the primary goal
+ - Improve performance of Sort and Merge using Row Format / JIT expressions
+- Documentation
+ - General improvements to DataFusion website
+ - Publish design documents
+- Streaming
+ - Create `StreamProvider` trait
+
+### Ballista
+
+- Make production ready
+ - Shuffle file cleanup
+ - Fill functional gaps between DataFusion and Ballista
+ - Improve task scheduling and data exchange efficiency
+ - Better error handling
+ - Task failure
+ - Executor lost
+ - Schedule restart
+ - Improve monitoring and logging
+ - Auto scaling support
+- Support for multi-scheduler deployments. Initially for resiliency and fault
tolerance but ultimately to support sharding for scalability and more efficient
caching.
+- Executor deployment grouping based on resource allocation
+
+### Extensions ([datafusion-contrib](https://github.com/datafusion-contrib))
+
+###
[DataFusion-Python](https://github.com/datafusion-contrib/datafusion-python)
+
+- Add missing functionality to DataFrame and SessionContext
+- Improve documentation
+
+###
[DataFusion-S3](https://github.com/datafusion-contrib/datafusion-objectstore-s3)
+
+- Create Python bindings to use with datafusion-python
+
+### [DataFusion-Tui](https://github.com/datafusion-contrib/datafusion-tui)
+
+- Create multiple SQL editors
+- Expose more Context and query metadata
+- Support new data sources
+ - BigTable, HDFS, HTTP APIs
+
+###
[DataFusion-BigTable](https://github.com/datafusion-contrib/datafusion-bigtable)
+
+- Python binding to use with datafusion-python
+- Timestamp range predicate pushdown
+- Multi-threaded partition aware execution
+- Production ready Rust SDK
+
+###
[DataFusion-Streams](https://github.com/datafusion-contrib/datafusion-streams)
+
+- Create experimental implementation of `StreamProvider` trait
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 8fbff208f5..ca6905c434 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -121,7 +121,6 @@ To get started, see
contributor-guide/testing
contributor-guide/howtos
contributor-guide/roadmap
- contributor-guide/quarterly_roadmap
contributor-guide/governance
contributor-guide/inviting
contributor-guide/specification/index
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
