asaf400 edited a comment on issue #18874:
URL: https://github.com/apache/airflow/issues/18874#issuecomment-941906832
I have confirmed the fix I suggested works in airflow 2.0.2,
For my own use, I just overrode the 'DockerOperator' class, and it's
'_run_image' function
I'll now start to work on a PR, branching out from latest main commit so
that future versions of airflow would be fixed..
for any future refence for AWS MWAA users which are kind of stuck on airflow
2.0.2 (as of now), copy the entire '_run_image' into a custom class with
'DockerOperator' as it's super:
1. remove this line:
`lines = self.cli.attach(container=self.container['Id'], stdout=True,
stderr=True, stream=True)`
2. overwrite the entire code block that's similar to this, with this.. :) :
```python
try:
self.cli.start(self.container['Id'])
result = self.cli.wait(self.container['Id'])
lines = self.cli.logs(container=self.container['Id'], stdout=True,
stderr=True, stream=True)
line = ''
res_lines = []
for line in lines:
```
and remove the `result = self.cli.wait(self.container['Id'])` line that
comes some code lines later
As I have described before, the logs() function is presumably much more
stable as it requests the entire container log from beginning to end via
`/containers/{id}/logs` api, rather then the `/containers/{id}/attach` api, it
all relies on docker's logging mechanisms, and shouldn't have any 'missing 1st
line' issues
Edits: formatting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
