potiuk commented on issue #15952:
URL: https://github.com/apache/airflow/issues/15952#issuecomment-882094536
I think this is somewhat ambiguous whether the APIs of docker return bytes
or strings. Apparently it has been a problem before as if you look few lines
up, the conditional decode happens (and that's why we have line.encode("utf-8")
that should stay there.
```
for line in lines:
line = line.strip()
if hasattr(line, 'decode'):
# Note that lines returned can also be byte sequences so
we have to handle decode here
line = line.decode('utf-8')
res_lines.append(line)
```
Not nice.
I looked at the APIs https://docker-py.readthedocs.io/en/1.2.3/api/ and it
is mentioned in both `attach` and `logs` that it will return either `str` or
generator. It does not mention generator of what type (and we are using
stream=Yes in attach so I guess we get the generator not str). So well, it's
kinda not-specified whether the generator returns bytes or strings and it's
"OKeyish" to try to react to both cases.
Interestingly, the documentation says that `attach` method is really a
wrapper around logs() - and when you look closer at the method seems what we
are trying to do is to retrieve the logs that we already retrieved (and stored
in `res_lines` as array of `str`). So it is likely that we actually receive
again the same generator when we call logs after earlier calling "attach" with
"stream=True".
So I think the proper fix is actually:
```
ret = res_lines if self.xcom_all else line
```
However I wonder maybe there was a good reason why the line was encoded to
bytes before pushing to Xcom? I think if we fix it, we should make a major
release of docker operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
