Re: AIP-8 Split Hooks/Operators into Separate Packages

@Max I don't see how this is doable. Consider S3ToGoogleCloudStorageOperator It users both S3Hook and GoogleCloudStorageHook. With your suggestion we have to maintain S3Hook in each separated package per operator/sensor. Which means for example if new parameter is added to any of the hooks you

[GitHub] feng-tao closed pull request #1: Create README.md

feng-tao closed pull request #1: Create README.md URL: https://github.com/apache/airflow-site/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL ab

Re: [GitHub] mik-laj opened a new issue #2: Website address in repo description

On 2019-01-09 21:20, GitBox wrote: Would it make sense to request that these notification emails go to commits@, rather than dev@? I gather an INFRA JIRA ticket should be enough to take care of that. Regards ap

[GitHub] mik-laj opened a new issue #2: Website address in repo description

mik-laj opened a new issue #2: Website address in repo description URL: https://github.com/apache/airflow-site/issues/2 Hello, I propose to add in the description of the repository the address where this website is located on the internet to make it easier to reach it. ![image]

[GitHub] mik-laj commented on issue #1: Create README.md

mik-laj commented on issue #1: Create README.md URL: https://github.com/apache/airflow-site/pull/1#issuecomment-452944297 I suggest you add an address where you can find this website on the internet. This is an automated messa

[GitHub] feng-tao opened a new pull request #1: Create README.md

feng-tao opened a new pull request #1: Create README.md URL: https://github.com/apache/airflow-site/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the

Re: AIP-8 Split Hooks/Operators into Separate Packages

If there's a strict policy of having a single hook and a single operator per package, then the hook package would be the only place where the external dependency is defined, and the operator packages would depend on hook package(s). That would follow the "micro package" philosophy and could work pr

Re: AIP-8 Split Hooks/Operators into Separate Packages

Regardless of how complex this implementation would be I am +1 on this. From the developer's point of view that the CI would run so much faster is the biggest plus for me. I think It will only become worse the more dependencies we add. From the user's point of view that I am able to choose from

Apache Airflow Announcements

Hi Folks! A few maintainers and I have been recording all milestones and events on the Airflow Announcements Wiki page since the day we started incubation. We've tried to record every announcement of value to the community here:

Airflow UI Log not avilable locally when task is in running state

I am sharing the problem that we face very frequently. When we go a particular dag and click the logs for a particular task_instance we end up with following error. The logs become available after the task finished i.e when it fetch from external source like s3. *** Log file isn't local. *** Fetc