its-amann commented on code in PR #76739:
URL: https://github.com/apache/airflow/pull/76739#discussion_r4512266059
##########
providers/papermill/src/airflow/providers/papermill/operators/papermill.py:
##########
@@ -142,6 +148,28 @@ def execute(self, context: Context):
**remote_kernel_kwargs,
)
+ if self.nbconvert:
Review Comment:
if output_nb is remote like s3://bucket/out.ipynb, jupyter nbconvert can't
open it, so the task fails after the notebook already ran and a retry runs the
whole notebook again. Skipping with a warning seems safer.
```suggestion
if self.nbconvert and "://" in self.output_nb.url:
self.log.warning("Skipping nbconvert, %s is not a local file",
self.output_nb.url)
elif self.nbconvert:
```
fixed it with a test in this commit, feel free to cherry pick it so you
don't have to redo it:
https://github.com/its-amann/airflow/commit/82a8f90269d3eee32efb5a7f7afee6c61ef4be1e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
