Sabbir02 opened a new issue, #45866: URL: https://github.com/apache/airflow/issues/45866
### Description The current implementation of the Papermill operator lacks the capability to connect with externally managed kernels, limiting its use to local kernel execution. This enhancement enables the operator to support running notebooks with externally managed kernels like Spark or Scala, improving flexibility and integration with Kubernetes for a smoother experience in Airflow. ### Use case/motivation This update allows Apache Airflow's Papermill operator to run Jupyter notebooks with various kernels, such as Spark or Scala, in addition to Python. This flexibility is crucial for teams working with diverse data processing tasks and multi-language environments, enabling smoother workflows and better integration between different systems. It enhances Airflow's versatility and ensures that users can execute complex tasks without being limited to just Python notebooks. ### Related issues The implementation has been verified as functional (see issue #35845). Included in Airflow version 2.8.0 ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
