nivangio opened a new issue, #39262: URL: https://github.com/apache/airflow/issues/39262
### Description Create operators and Hook to interact with Databricks' DBFS (https://docs.databricks.com/api/workspace/dbfs) ### Use case/motivation As per latest databricks plugin (https://github.com/apache/airflow/tree/main/airflow/providers/databricks) there is no possibility to interact with DBFS API. As I had to do it in my job (and I have it quite developed), I thought it'd be a good idea to share it with the community So far, I've got: - An operator that uploads files to DBFS - A hook that interacts with the DBFS API, respecting Databricks' Hooks logic and inheriting from `BaseDatabricksHook` As part of the PR, I'd add: - Some more operators (getting files, getting files metadata, deleting files) - Tests in line with Airflow's test suite Please LMK if that's sth worth adding or not ### Related issues As one of the DBFS API endpoints uses `PUT` as verb., I'd need to include a modification in `BaseDatabricksHook`, because it is not supporting PUT ATM (see https://github.com/apache/airflow/blob/main/airflow/providers/databricks/hooks/databricks_base.py#L584) ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
