On Mon, Sep 23, 2019 at 1:22 PM Kamil Breguła <kamil.breg...@polidea.com> wrote:
> On Mon, Sep 23, 2019 at 7:04 PM Chris Palmer <ch...@crpalmer.com> wrote: > > > > Is there a reason why we can't use symlinks to have copies of the files > > show up in both subpackages? So that `gcs_to_s3.py` would be under both > > `aws/operators/` and `gcp/operators`. I could imagine there may be > > technical reasons why this is a bad idea, but just thought I would ask. > > > Symlinks is not supported by git. > > Why do you say that? This blog post <https://www.mokacoding.com/blog/symliks-in-git/> details how you can use them, and the caveats with regards to needing relative links not absolute. The example repo he links to at the end includes a symlink which worked fine for me when I cloned it. But maybe not relevant given the below: > > Likewise, someone who spends 99% of their time working in AWS and using > all > > the operators in that subpackage, might not think to look in the GCP > > package the first time they need a GCS to S3 operator. I'm admittedly > > terrible at documentation, but if duplicating the files via symlinks > isn't > > an option, then is there an easy way we could duplicate the documentation > > for those operators so they are easily findable in both doc sections? > > > > Recently, I updated the documentation: > https://airflow.readthedocs.io/en/latest/integration.html > We have list of all integration in AWS, Azure, GCP. If the operator > concerns two cloud proivders, it repeats in two places. It's good for > documentation. DRY rule is only valid for source code. > I am working on documentation for other operators. > My work is part of this ticket: > https://issues.apache.org/jira/browse/AIRFLOW-5431 > > This updated documentation looks great, definitely heading in a direction that makes it easier and addresses my concerns. (Although it took me a while to realize those tables can be scrolled horizontally!). Chris
