VividByteWorker opened a new issue, #57921: URL: https://github.com/apache/doris/issues/57921
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Currently, the execution engine of Doris adopts batch computation. It can handle near-real-time online analytical scenarios well, but consumes relatively much resources when computing large datasets. If it can support incremental computation, it will achieve more efficient computing performance in some scenarios and cope with more real-time scenarios such as live streaming, logistics, and e-commerce. Based on this, I recommend introducing Table Stream and Dynamic Table to support incremental data computation and make Doris to be a streaming warehouse. <img width="2030" height="480" alt="Image" src="https://github.com/user-attachments/assets/c82d2384-d2f4-4668-adb1-a8207b8919f9"; /> ### Use case The benefits are as follows: 1. Obtain changes based on table streams to realize CDC (Change Data Capture) acquisition <img width="1558" height="444" alt="Image" src="https://github.com/user-attachments/assets/5fbb60c7-bbcd-4fee-aac5-dc0a00ca935e"; /> 2. Support online dual-stream join to achieve more efficient multi-stream column concatenation capability. 3. Realize online stream-based incremental aggregation computing. 4. Asynchronous Materialized Views with Incremental Computation: Implement materialized views that are refreshed incrementally, thereby accelerating near-real-time online serving queries. ### Related issues 1. to add table stream to aquire CDC 2. to add dynamic table 3. to add incremental computation runtime 4. to add optimizer rules for incremental computation ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
