nsuthar-lumiq opened a new pull request, #26206:
URL: https://github.com/apache/airflow/pull/26206
Added Continuously logging feature for Glue jobs and fixed existing bug for
not printing Cloduwatch logs.
Following are the changes done:
- **GlueJobOperator**
- Added new input parameters name as **`continuous_logging`** for
Continuous logging.
- **GlueJobHooks**
- Added new input parameters name as **`continuous_logging`** for
Continuous logging in the method **`job_completion`**.
- Added the logic to print available logs from **Cloudwatch** to Airflow
logs.
- Added new method **`print_output_logs`** for handling continuously
logging.
- Fixed the existing bug for not executing method **`print_jobs_logs`**
- Parameterized the **boto3** cloudwatch logs client instead creating
new client in every call.
- Change the method for printing selective logs from filter mechanism
to **get log event** method.
- Added Condition for **Batch** , **Continue** and **no logging** in
**`job_completion`** method
#26203
https://github.com/apache/airflow/issues/26196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
