Rutvijjs opened a new issue, #40076:
URL: https://github.com/apache/airflow/issues/40076
### Apache Airflow Provider(s)
databricks
### Versions of Apache Airflow Providers
6.1.0 and above
### Apache Airflow version
2.6.3
### Operating System
Mac-os
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Deployed into a kubernetes cluster through a docker image.
### What happened
We have been using the Databricks provider for quite some time with the
version 6.1.0. However, recently we started running into an issue with our
airflow install on a kubernetes cluster (through a dockerfile). Our docker file
is using the 2.6.3 image of airflow yet when we do a pip install for databricks
provider (for versions greater than 6.1.0 and even tested with just installing
6.1.0) it will automatically update the version of airflow to the latest (at
this time this is 2.9.1) causing other dependency issues installed through the
airflow image (2.6.3).
### What you think should happen instead
The airflow image should not be updated to the latest version but instead be
kept to what the version is specified in the image.
### How to reproduce
run a docker file that expands the apache-airflow 2.6.3 image and add a step
for pip install. As follows:
FROM apache/airflow:2.6.3-python3.9
USER root
RUN apt update
RUN apt install openjdk-11-jre -y
USER airflow
# required for Airflow on Azure with Kubernetes
RUN pip install --no-cache-dir \
apache-airflow-providers-databricks==6.1.0
### Anything else
Every time i install using the versions
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
