hussein-awala commented on code in PR #32847:
URL: https://github.com/apache/airflow/pull/32847#discussion_r1284716038
##########
airflow/providers/google/cloud/hooks/dataflow.py:
##########
@@ -1014,8 +1014,9 @@ def start_sql_job(
self.log.info("Output: %s", proc.stdout.decode())
self.log.warning("Stderr: %s", proc.stderr.decode())
self.log.info("Exit code %d", proc.returncode)
+ stderr_last_20_lines =
"\n".join(proc.stderr.decode().strip().splitlines()[-20:])
if proc.returncode != 0:
- raise AirflowException(f"Process exit with non-zero exit code.
Exit code: {proc.returncode}")
+ raise AirflowException(f"Process exit with non-zero exit code.
Exit code: \ {proc.returncode} \n Error Details : {stderr_last_20_lines}")
Review Comment:
I see that we already log the stderr lines in warning level, so maybe we
could move this part to the condition and log it in error level if the return
code is not equal to 0, and if it is 0 we log it in warning level.
WDYT?
something like:
```python
if proc.returncode != 0:
self.log.error("Stderr: %s", proc.stderr.decode())
self.log.info("Exit code %d", proc.returncode)
raise AirflowException(f"Process exit with non-zero exit code.
Exit code: {proc.returncode}")
else:
self.log.warning("Stderr: %s", proc.stderr.decode())
self.log.info("Exit code %d", proc.returncode)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
