shahar1 commented on PR #56531: URL: https://github.com/apache/airflow/pull/56531#issuecomment-3389827700
> Sorry, I am yet to watch the presentation or have access to the slides. Any initial benchmark numbers in comparison to the python dag-processor in terms of memory, speed and cpu? Hey, I just gave the talk yesterday so ot might take a while the video shows up. tl;dr Within the dag processor, I plan to implement only the processor manager in Rust - the serialization process will remain for now in Python*. The implementation in its current state is very initial, so I haven't implemented yet a proper benchmarking. From my current experiments it should be about 100+ times faster than the current Python processor. Hopefully I could get it materialized in the upcoming months as an experimental feature - feel free to get in touch on Slack if you want to get involved :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
