dabla commented on PR #41327:
URL: https://github.com/apache/airflow/pull/41327#issuecomment-2325860567
@eladkal and @potiuk what do you think about this implementation? I would
ideally want to use entry points to register the dialects, so that additional
dialects can be loaded through different providers, but couldn't find such
mechanism in the Airflow code base, apparently, according to ChatGPT, it is
possible to do it either via a setup.py or via pyproject.toml. So if you have
any suggestions on that case, that would be helpful, unless you guys have other
ideas or suggestions or don't like the implementation at all of course. Atm
the dialects (for the moment there is only MSSQL) are registered in a
hard-coded way in the common-sql provider to make it work, it would be nice if
those would be dynamically registered from the dedicated providers.
Bellow the answer from ChatGPT:
```
When dealing with a monolithic repository like Apache Airflow, where the
setup for each provider isn't handled through separate setup.py or
pyproject.toml files, but rather as part of a larger project, you'll need to
use a different approach to define and register entry points.
In the context of Apache Airflow, which uses a monolithic repository
structure with multiple providers, you typically interact with entry points
through the main project’s setup configuration. Here's how you can handle entry
points within such a setup:
Steps to Define Entry Points in a Monolithic Repo like Airflow
Find the Main setup.py or pyproject.toml:
In a monolithic project like Airflow, there will be a main setup.py or
pyproject.toml at the root of the repository. This is the file that defines how
the entire project is built and installed.
Locate Entry Points Definition:
Look for a section in the main setup.py or pyproject.toml dedicated to entry
points. This section typically aggregates all the entry points from different
parts of the project. For Airflow, the entry points for plugins, executors, or
other components are defined here.
Add Your Entry Point:
Modify the entry points section to include your specific entry point. This
might involve editing a Python dictionary in setup.py or adding a TOML table in
pyproject.toml.
Here’s how you might define an entry point in setup.py for an Airflow
provider:
# In setup.py of the main Airflow project
setup(
# other setup arguments...
entry_points={
'sqlalchemy.dialects': [
'mydb = myprovider.mydialect:MyDialect', # Your custom dialect
entry point
],
'airflow.plugins': [
'myplugin = myprovider.myplugin:MyAirflowPlugin', # Example of
Airflow plugin entry point
],
# other entry points...
},
# other setup arguments...
)
If you're using pyproject.toml:
[project.entry-points."sqlalchemy.dialects"]
mydb = "myprovider.mydialect:MyDialect"
[project.entry-points."airflow.plugins"]
myplugin = "myprovider.myplugin:MyAirflowPlugin"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
