baugarcia opened a new issue, #42520:
URL: https://github.com/apache/airflow/issues/42520
### Apache Airflow Provider(s)
airbyte
### Versions of Apache Airflow Providers
apache-airflow-providers-airbyte | 4.0.0
### Apache Airflow version
2.8.1
### Operating System
AWS MWAA
### Deployment
Amazon (AWS) MWAA
### Deployment details
Nothing really relevant. Standard AWS MWAA installation.
### What happened
There are two issues with the version 4.0.0 of the airbyte provider:
1 - In the airbyte provider, there is no use case for disabling
authentication as an OSS instalation. You are being forced to enable
authentication and pass the client_id and client_secret
2 - It does not seem to get the authentication token. Even after enabling
auth and creating the user application and using the client_id and
client_secret, it just gets a 404 error. None of the documentes endpoints seem
to work. We have also tried multiple iterations of endpoints, such as /api,
/api/v1. /api/public/v1/.... in both the host and/or the Token URL (also tested
with v1/applications/token):
```
[2024-09-26, 15:41:37 UTC] {{taskinstance.py:2698}} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/airbyte/hooks/airbyte.py",
line 157, in submit_sync_connection
res = self.airbyte_api.jobs.create_job(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airbyte_api/jobs.py",
line 100, in create_job
raise e
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airbyte_api/jobs.py",
line 95, in create_job
req =
self.sdk_configuration.get_hooks().before_request(BeforeRequestContext(hook_ctx),
req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airbyte_api/_hooks/sdkhooks.py",
line 41, in before_request
out = hook.before_request(hook_ctx, request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py",
line 59, in before_request
sess = self.do_token_request(credentials, self.get_scopes(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py",
line 122, in do_token_request
raise Exception(
Exception: Unexpected status code 404 from token endpoint
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py",
line 433, in _execute_task
result = execute_callable(context=context, **execute_callable_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/airbyte/operators/airbyte.py",
line 83, in execute
job_object =
hook.submit_sync_connection(connection_id=self.connection_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/airbyte/hooks/airbyte.py",
line 165, in submit_sync_connection
raise AirflowException(e)
airflow.exceptions.AirflowException: Unexpected status code 404 from token
endpoint
```
### What you think should happen instead
1 - There should be a use case for an OSS instalation with disabled
authentication. No token needed to connect.
2 - Not have a 404 error. Or document clearly what you should input in the
fields, if we are doing something wrong.
### How to reproduce
1 - Newest install of OSS Airbyte with authentication disabled. You don't
have the /applications endpoint, so you cannot get a token and therefore use
the provider
2 - Install newest OSS Airbyte, use airflow's airbyte 4.0.0 provider, try
to connect and submit a job using the client_id and client_secret, as well as
the Token-URL and host fields.
### Anything else
Every time
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
