potiuk edited a comment on pull request #11509:
URL: https://github.com/apache/airflow/pull/11509#issuecomment-730619931
@marcusianlevine. I believe is that you have not modified your logging
configuration. The thing is that in airflow 1.10 the ElasticsearchTaskHandler
is part of the airflow 1.10 itself in `airflow/utils/log/es_task_handler.py` -
and almost by definition, those files are not touched by the new provider. What
you need to do, is to modify your logging configuration to use the new
"provider" task handler: airflow.providers.elasticsearch.log.es_task_handler
In 1.10 the configuration looked like this (this is a default, so you might
have modified it):
```
ELASTIC_REMOTE_HANDLERS = {
'task': {
'class':
'airflow.utils.log.es_task_handler.ElasticsearchTaskHandler',
'formatter': 'airflow',
'base_log_folder': os.path.expanduser(BASE_LOG_FOLDER),
'log_id_template': ELASTICSEARCH_LOG_ID_TEMPLATE,
'filename_template': FILENAME_TEMPLATE,
'end_of_log_mark': ELASTICSEARCH_END_OF_LOG_MARK,
'host': ELASTICSEARCH_HOST,
'write_stdout': ELASTICSEARCH_WRITE_STDOUT,
'json_format': ELASTICSEARCH_JSON_FORMAT,
'json_fields': ELASTICSEARCH_JSON_FIELDS
},
}
```
And when you switch it to the new provider you have to change it to this
(note changed 'class' attribute):
```
ELASTIC_REMOTE_HANDLERS: Dict[str, Dict[str, Union[str, bool]]] = {
'task': {
'class':
'airflow.providers.elasticsearch.log.es_task_handler.ElasticsearchTaskHandler',
'formatter': 'airflow',
'base_log_folder': str(os.path.expanduser(BASE_LOG_FOLDER)),
'log_id_template': ELASTICSEARCH_LOG_ID_TEMPLATE,
'filename_template': FILENAME_TEMPLATE,
'end_of_log_mark': ELASTICSEARCH_END_OF_LOG_MARK,
'host': ELASTICSEARCH_HOST,
'frontend': ELASTICSEARCH_FRONTEND,
'write_stdout': ELASTICSEARCH_WRITE_STDOUT,
'json_format': ELASTICSEARCH_JSON_FORMAT,
'json_fields': ELASTICSEARCH_JSON_FIELDS,
},
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
