This is an automated email from the ASF dual-hosted git repository.
xudong963 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new accd2255f0 Update Roadmap documentation (#16399)
accd2255f0 is described below
commit accd2255f05acc91827016ffdd96b66e774ed2dc
Author: Andrew Lamb <and...@nerdnetworks.org>
AuthorDate: Wed Jun 18 08:40:45 2025 -0400
Update Roadmap documentation (#16399)
* Update Roadmap documentation
* Incorporate feedback
---
benchmarks/README.md | 1 -
docs/source/contributor-guide/roadmap.md | 84 +++-----------------------------
2 files changed, 8 insertions(+), 77 deletions(-)
diff --git a/benchmarks/README.md b/benchmarks/README.md
index 01da886ffb..d0f413b2e9 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -518,7 +518,6 @@ In addition, topk_tpch is available from the bench.sh
script:
./bench.sh run topk_tpch
```
-
## IMDB
Run Join Order Benchmark (JOB) on IMDB dataset.
diff --git a/docs/source/contributor-guide/roadmap.md
b/docs/source/contributor-guide/roadmap.md
index 3d9c1ee371..79add1b86f 100644
--- a/docs/source/contributor-guide/roadmap.md
+++ b/docs/source/contributor-guide/roadmap.md
@@ -46,81 +46,13 @@ make review efficient and avoid surprises.
# Quarterly Roadmap
-A quarterly roadmap will be published to give the DataFusion community
-visibility into the priorities of the projects contributors. This roadmap is
not
-binding and we would welcome any/all contributions to help keep this list up to
-date.
+The DataFusion roadmap is driven by the priorities of contributors rather than
+any single organization or coordinating committee. We typically discuss our
+roadmap using GitHub issues, approximately quarterly, and invite you to join
the
+discussion.
-## 2023 Q4
+For more information:
-- Improve data output (`COPY`, `INSERT` and DataFrame) output capability
[#6569](https://github.com/apache/datafusion/issues/6569)
-- Implementation of `ARRAY` types and related functions
[#6980](https://github.com/apache/datafusion/issues/6980)
-- Write an industrial paper about DataFusion for SIGMOD
[#6782](https://github.com/apache/datafusion/issues/6782)
-
-## 2022 Q2
-
-### DataFusion Core
-
-- IO Improvements
- - Reading, registering, and writing more file formats from both DataFrame
API and SQL
- - Additional options for IO including partitioning and metadata support
-- Work Scheduling
- - Improve predictability, observability and performance of IO and CPU-bound
work
- - Develop a more explicit story for managing parallelism during plan
execution
-- Memory Management
- - Add more operators for memory limited execution
-- Performance
- - Incorporate row-format into operators such as aggregate
- - Add row-format benchmarks
- - Explore JIT-compiling complex expressions
- - Explore LLVM for JIT, with inline Rust functions as the primary goal
- - Improve performance of Sort and Merge using Row Format / JIT expressions
-- Documentation
- - General improvements to DataFusion website
- - Publish design documents
-- Streaming
- - Create `StreamProvider` trait
-
-### Ballista
-
-- Make production ready
- - Shuffle file cleanup
- - Fill functional gaps between DataFusion and Ballista
- - Improve task scheduling and data exchange efficiency
- - Better error handling
- - Task failure
- - Executor lost
- - Schedule restart
- - Improve monitoring and logging
- - Auto scaling support
-- Support for multi-scheduler deployments. Initially for resiliency and fault
tolerance but ultimately to support sharding for scalability and more efficient
caching.
-- Executor deployment grouping based on resource allocation
-
-### Extensions ([datafusion-contrib](https://github.com/datafusion-contrib))
-
-###
[DataFusion-Python](https://github.com/datafusion-contrib/datafusion-python)
-
-- Add missing functionality to DataFrame and SessionContext
-- Improve documentation
-
-###
[DataFusion-S3](https://github.com/datafusion-contrib/datafusion-objectstore-s3)
-
-- Create Python bindings to use with datafusion-python
-
-### [DataFusion-Tui](https://github.com/datafusion-contrib/datafusion-tui)
-
-- Create multiple SQL editors
-- Expose more Context and query metadata
-- Support new data sources
- - BigTable, HDFS, HTTP APIs
-
-###
[DataFusion-BigTable](https://github.com/datafusion-contrib/datafusion-bigtable)
-
-- Python binding to use with datafusion-python
-- Timestamp range predicate pushdown
-- Multi-threaded partition aware execution
-- Production ready Rust SDK
-
-###
[DataFusion-Streams](https://github.com/datafusion-contrib/datafusion-streams)
-
-- Create experimental implementation of `StreamProvider` trait
+1. [Search for issues labeled
`roadmap`](https://github.com/apache/datafusion/issues?q=is%3Aissue%20%20%20roadmap)
+2. [DataFusion Road Map: Q3-Q4
2025](https://github.com/apache/datafusion/issues/15878)
+3. [2024 Q4 / 2025 Q1
Roadmap](https://github.com/apache/datafusion/issues/13274)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@datafusion.apache.org
For additional commands, e-mail: commits-h...@datafusion.apache.org
