[
https://issues.apache.org/jira/browse/ATTIC-218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Potiuk updated ATTIC-218:
-------------------------------
Summary: Moving one of many packages to attic (or just removing it from the
code)? (was: Moving one of mny packages to attic (or just removing it from the
code)?)
> Moving one of many packages to attic (or just removing it from the code)?
> -------------------------------------------------------------------------
>
> Key: ATTIC-218
> URL: https://issues.apache.org/jira/browse/ATTIC-218
> Project: Attic
> Issue Type: Task
> Reporter: Jarek Potiuk
> Priority: Major
>
> Hello,
> We have an interesting situation in Apache Airflow that I would like to have
> an advice on.
> When we want to remove some small (but with separate release pacakge) part of
> the PMC project, should we (somehow?) move the code we remove to attic, or is
> it just fine to remove it from sources?
> A bit of context:
> Apache Airflow is a huge and rather successful PMC (>2600 contributors, tens
> of thousands of users) but it also rather complex one. Since Airflow is an
> orchestrator and interacts with multiple eservices, instead of a single or
> few package we have 1 core, 1 python client and ~ 90 provider packages - each
> provider is a separately released convenience package, separately released
> source package. Every two weeks or so we release and vote on between few to
> all 90 provider packages (there is a well established process of bulk voting.
> Example one here:
> [https://lists.apache.org/thread/j4b5851g9lqg0jm8yb68f83vqfxzqy14]
> We released all providers that are "active" - because we bumped the minimum
> version of Airflow according to our policies.
> All those providers are managed together with Airflow in a single monorepo
> [https://github.com/apache/airflow/tree/main/airflow/providers]
> We currently have also well established process of suspending providers that
> are - for some reason holding us back (mainly due to dependencies) - we stop
> releasing them, we stop updating the code and the code in the repo gets
> efffectively ignored by tests/CI/release process. The process, voting,
> resuming is well established and being followed (we already suspended - and
> later resumed yandex, and currently we suspended qubole provider). This is
> really nice for temporary suspensions as it does not hold the development,
> but it makes it really easy (just a PR) to fix and resume such provider
> (worked very nicely in case of yandex who got resumed after upstream
> libraries fixed some dependency issues)
> https://github.com/apache/airflow/blob/main/airflow/providers/SUSPENDING_AND_RESUMING_PROVIDERS.rst
> However we eventually would like to remove qubole. It's pretty useless now -
> because the service (qubole) that it connected to had been acquired and
> discontinued
> The website is still there but see the note here:
> [https://github.com/qubole/qds-sdk-py#where-are-the-maintainers-] (note
> added 7 months ago) - there is basically no way any of their libraries will
> get updated, and the service does not work any more. Maybe someone would
> like to use the code we have there but for us it is a burden (dependencies,
> refactors etc. etc.) and we would very much like to just remove it. from
> `main` and simply mark the "apache-airflow-providers-qubole" package as
> "gone".
> So we would like to remove the code and somehow let the users know that the
> package we released in the past is "discontinued" in PyPI
> [https://pypi.org/project/apache-airflow-providers-qubole/]
> So I have a few questions:
> 1) should attic PMC be involved when we remove the code or are we fine to
> **just** remove it.
> 2) any advice on communication / informing our users?
> 3) should we do anything about the package we release in PyPI?
> Would love to hear what you think.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
