villebro commented on code in PR #2834: URL: https://github.com/apache/datafusion-ballista/pull/2834#discussion_r4527108386
########## README.md: ########## @@ -143,6 +144,23 @@ Ballista supports two execution modes, selected with `ballista.execution.mode`. Pipelined execution (experimental) runs a job the way [Trino](https://trino.io) does: every task is scheduled at once and stages stream record batches to each other over Arrow Flight, with no intermediate data written to disk, so results reach the client sooner, at the cost of a failed task failing the whole job. It suits short, latency-sensitive queries on clusters with room for the whole job. See the [Pipelined Execution Guide](docs/source/user-guide/pipelined-execution.md) for details. +## One Engine for ETL and Interactive Queries + +A lakehouse aims to merge the data warehouse and the ETL platform, but it is often still served by two engines: Spark +for batch ETL and Trino for interactive analytics. Ballista's two execution modes let one engine serve both, from the Review Comment: While Spark is arguably the de facto leader in batch ETL, and Trino is certainly well established in enterprise settings, the interactive analytics engine landscape is arguably more fragmented (ClickHouse, StarRocks, Apache Doris/Pinot/Druid etc. come to mind). Should we consider making this section more agnostic to specific reference engines, and instead describe the workload categories themselves? This would also future-proof the documentation as the landscape evolves and new contenders emerge in either space. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
