kosiew commented on code in PR #26926: URL: https://github.com/apache/datafusion/pull/26926#discussion_r4450449878
########## docs/source/user-guide/introduction.md: ########## @@ -91,98 +91,83 @@ parallelized streaming execution plans, file format support, etc. ## Known Users -Here are some active projects using DataFusion: +Projects with a public GitHub repository must meet **all** of these criteria to +appear in the active list: + +- At least one commit on the default branch in the last 6 months (calendar months). +- At least 100 stars. +- At least 5 contributors, as shown in the repository's GitHub sidebar. - <!-- "Active" means github repositories that had at least one commit in the last 6 months --> +Check these criteria when adding or updating an entry. Projects without a public +repository must instead link to public evidence of DataFusion use, such as a blog +post or documentation page. These criteria apply to Known Users, not the +Integrations and Extensions lists below. + +Here are some active projects using DataFusion: - [Arroyo](https://github.com/ArroyoSystems/arroyo) Distributed stream processing engine in Rust - [ArkFlow](https://github.com/arkflow-rs/arkflow) High-performance Rust stream processing engine -- [Auron] The Auron accelerator for big data engine (e.g., Spark, Flink) leverages native vectorized execution to accelerate query processing -- [Ballista] Distributed SQL Query Engine -- [CnosDB] Open Source Distributed Time Series Database +- [Auron](https://github.com/apache/auron) The Auron accelerator for big data engine (e.g., Spark, Flink) leverages native vectorized execution to accelerate query processing +- [Ballista](https://github.com/apache/datafusion-ballista) Distributed SQL Query Engine +- [CnosDB](https://github.com/cnosdb/cnosdb) Open Source Distributed Time Series Database - [Comet](https://github.com/apache/datafusion-comet) Apache Spark native query execution plugin -- [Cube Store] Cube’s universal semantic layer platform is the next evolution of OLAP technology for AI, BI, spreadsheets, and embedded analytics +- [Cube Store](https://github.com/cube-js/cube.js/tree/master/rust) Cube’s universal semantic layer platform is the next evolution of OLAP technology for AI, BI, spreadsheets, and embedded analytics - [datafusion-dft](https://github.com/datafusion-contrib/datafusion-dft) Batteries included CLI, TUI, and server implementations for DataFusion. -- [datapress](https://docs.datap-rs.org) An opinionated small and fast data server on parquet and delta tables. - [dbt Fusion engine](https://github.com/dbt-labs/dbt-fusion) The dbt Fusion engine, written in Rust, designed for speed and correctness with a native SQL understanding across DWH SQL dialects. -- [delta-rs] Native Rust implementation of Delta Lake -- [EDB Postgres Lakehouse] built with [Seafowl] +- [delta-rs](https://github.com/delta-io/delta-rs) Native Rust implementation of Delta Lake - [Feldera](https://github.com/feldera/feldera) Fast query engine for incremental computation -- [Funnel](https://funnel.io/) Data Platform powering Marketing Intelligence applications. -- [GlareDB](https://github.com/GlareDB/glaredb) Fast SQL database for querying and analyzing distributed data. -- [GreptimeDB] Open Source & Cloud Native Distributed Time Series Database -- [hiop](https://hiop.io) Serverless Data Logistic Platform -- [HoraeDB] Distributed Time-Series Database -- [Hotdata](https://www.hotdata.dev) On-demand databases for AI agents with a unified query engine for vector, OLAP, and full-text search. +- [Funnel](https://funnel.io/) Data Platform powering Marketing Intelligence applications. ([DataFusion use](https://funnel.io/devblog/the-data-that-wont-sit-still-engineering-at-the-edge-of-marketing-data)) +- [GreptimeDB](https://github.com/GreptimeTeam/greptimedb) Open Source & Cloud Native Distributed Time Series Database +- [hiop](https://hiop.io) Serverless Data Logistic Platform ([DataFusion use](https://github.com/apache/datafusion/pull/18114)) +- [Hotdata](https://www.hotdata.dev) On-demand databases for AI agents with a unified query engine for vector, OLAP, and full-text search. ([DataFusion use](https://www.hotdata.dev/blog/bringing-ducklake-to-apache-datafusion)) - [Iceberg-rust](https://github.com/apache/iceberg-rust) Rust implementation of Apache Iceberg -- [IceGate](https://icegate.tech) Observability data lake engine for metrics, traces, logs, and events, built on Apache Iceberg with OpenTelemetry ingestion -- [InfluxDB] Time Series Database -- [Kamu] Planet-scale streaming data pipeline +- [InfluxDB](https://github.com/influxdata/influxdb) Time Series Database +- [Kamu](https://github.com/kamu-data/kamu-cli) Planet-scale streaming data pipeline - [Kubeflow Trainer](https://github.com/kubeflow/trainer) Kubernetes-native project designed for scalable LLMs fine-tuning and distributed AI model training. - [LakeSoul](https://github.com/lakesoul-io/LakeSoul) Open source LakeHouse framework with native IO in Rust. -- [Lance](https://github.com/lancedb/lance) Modern columnar data format for ML -- [OpenObserve] Distributed cloud native observability platform +- [Lance](https://github.com/lance-format/lance) Modern columnar data format for ML +- [OpenObserve](https://github.com/openobserve/openobserve) Distributed cloud native observability platform - [ParadeDB](https://github.com/paradedb/paradedb) PostgreSQL for Search & Analytics -- [Parseable] Open columnar observability data lake platform for logs, metrics, and traces. -- [Massive.com](https://massive.com/) Stock Market API -- [Mycorr](https://mycorr.app) Shared, versioned tables with units, links and SQL for humans and agents -- [qv] Quickly view your data +- [Parseable](https://github.com/parseablehq/parseable) Open columnar observability data lake platform for logs, metrics, and traces. +- [Massive.com](https://github.com/massive-com) Stock Market API +- [Mycorr](https://mycorr.app) Shared, versioned tables with units, links and SQL for humans and agents ([DataFusion use](https://docs.mycorr.app/developers/mcp-server.html#querying-with-sql)) +- [qv](https://github.com/timvw/qv) Quickly view your data - [R2 Query Engine](https://blog.cloudflare.com/r2-sql-deep-dive/) Cloudflare's distributed engine for querying data in Iceberg Catalogs - [rerun.io](https://rerun.io/) Visualize and query robotics logs and transform them into training data. -- [Restate](https://github.com/restatedev) Easily build resilient applications using distributed durable async/await -- [ROAPI] Create full-fledged APIs for slowly moving datasets without writing a single line of code +- [Restate](https://github.com/restatedev/restate) Easily build resilient applications using distributed durable async/await - [Sail](https://github.com/lakehq/sail) Unifying stream, batch and AI workloads with Apache Spark compatibility - [SedonaDB](https://github.com/apache/sedona-db) A single-node analytical database engine with geospatial as a first-class citizen Review Comment: updated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
