gregwhorley opened a new issue, #36118:
URL: https://github.com/apache/airflow/issues/36118
### Official Helm Chart version
1.11.0 (latest released)
### Apache Airflow version
2.7.3
### Kubernetes Version
1.24
### Helm Chart configuration
```
dags:
gitSync:
enabled: true
repo: https://github.com/myOrg/myRepo.git
branch: feature/enable-gitsync
rev: HEAD
depth: 1
# the number of consecutive failures allowed before aborting
maxFailures: 0
# subpath within the repo where dags are located
# should be "" if dags are at repo root
subPath: "dataflow/airflow/dags"
credentialsSecret: git-sync-key
```
### Docker Image customizations
```
ARG AIRFLOW_VERSION
ARG PYTHON_VERSION
FROM --platform=linux/amd64
apache/airflow:slim-${AIRFLOW_VERSION}-${PYTHON_VERSION}
ARG PIP_INDEX_URL="https://pypi.org/simple";
ARG SPARK_VERSION
USER root
RUN apt-get update \
&& ACCEPT_EULA=Y apt-get upgrade -y \
&& apt-get install -y \
pkgconf \
openjdk-17-jre-headless \
procps \
libpq-dev \
default-libmysqlclient-dev \
gcc \
&& export PIP_INDEX_URL=${PIP_INDEX_URL} \
&& su airflow -c 'pip install psycopg2 mysqlclient' \
&& apt-get autoremove gcc --purge -y \
&& export SUDO_FORCE_REMOVE=yes \
&& apt-get -y remove python3.9 sqlite3 ldap-utils openssh-client curl
rsync sudo \
&& apt -y autoremove \
&& rm -rf /var/cache/apt/* \
&& rm -rf /var/lib/apt/lists/*
# NB: We switch back to our airflow user and install the remaining pip
dependencies
USER airflow
COPY requirements.txt requirements.txt
RUN export PIP_INDEX_URL=${PIP_INDEX_URL}
RUN pip install --upgrade pip
RUN pip install --user --no-cache pyspark==${SPARK_VERSION}
apache-airflow[aws,postgres,slack,docker,mongo,mysql,spark,statsd]~=${AIRFLOW_VERSION}
RUN pip install --user --no-cache -r requirements.txt
RUN echo "JAVA_HOME=$(dirname $(dirname $(update-alternatives --list
java)))" >> ${HOME}/.bashrc \
&& echo
"SPARK_HOME=/home/airflow/.local/lib/${PYTHON_VERSION}/site-packages/pyspark"
>> ${HOME}/.bashrc
```
```
# requirements.txt
apache-airflow-providers-cncf-kubernetes~=7.5.0
boto3
docker
kubernetes
yoyo-migrations~=8.2.0
flask-appbuilder
sqlalchemy
authlib
```
### What happened
I am trying to get the gitsync feature enabled and working for my org's
Airflow instance running in Kubernetes. I tried configuring it for ssh at
first, following the steps in
https://airflow.apache.org/docs/helm-chart/stable/manage-dags-files.html#mounting-dags-from-a-private-github-repo-using-git-sync-sidecar
but I kept getting invalid key errors:
```
Cloning into '/git'...
Load key "/etc/git-secret/ssh": invalid format
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
```
So I decided to switch to HTTPS for the time being just so I could prove it
works. The airflow helm chart's values file shows the overrides to enable this:
https://github.com/apache/airflow/blob/a315165719f950c5d13493b6138c6d6eba7929ae/chart/values.yaml#L2421-L2443
When I switch to this auth method, a new error occurs:
```
Cloning into '/git'...
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see
https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls
for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/my-org/my-repo.git/
```
Not sure how this should be addressed other than removing the https auth
option from the helm chart values and docs.
### What you think should happen instead
Make the https auth option non-existent since Github does not support it
anymore.
### How to reproduce
- Spin up a k8s cluster
- Create a values file with the following overrides
```
dags:
gitSync:
enabled: true
repo: https://github.com/aRealRepo.git
branch: feature/enable-gitsync
rev: HEAD
credentialsSecret: git-sync-key
```
- Create and apply a Kubernetes secret resource with the following data
```
apiVersion: v1
kind: Secret
metadata:
name: git-sync-key
data:
GIT_SYNC_USERNAME: <base64_encoded_git_username>
GIT_SYNC_PASSWORD: <base64_encoded_git_password>
```
- Install airflow helm chart and include this values override file
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
