[GitHub] [airflow] mik-laj commented on a change in pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8808: URL: https://github.com/apache/airflow/pull/8808#discussion_r426420203 ## File path: docs/howto/operator/gcp/cloud_build.rst ## @@ -80,6 +82,13 @@ It is also possible to specify it using the URL: :start-after: [START

[GitHub] [airflow] mik-laj commented on pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-18 Thread GitBox
mik-laj commented on pull request #8808: URL: https://github.com/apache/airflow/pull/8808#issuecomment-62249 Travis is sad. Do you need any help? This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] mik-laj commented on pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
mik-laj commented on pull request #8824: URL: https://github.com/apache/airflow/pull/8824#issuecomment-629976918 Hej. Can you change default values to 60s? Cheers This is an automated message from the Apache Git Service. To

[GitHub] [airflow] mschickensoup commented on pull request #8539: JIRA and Github issues explanation

2020-05-18 Thread GitBox
mschickensoup commented on pull request #8539: URL: https://github.com/apache/airflow/pull/8539#issuecomment-629993650 A :3 Thank you guys! @mik-laj @potiuk This is an automated message from the Apache Git Service. To

[GitHub] [airflow] joppevos commented on pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-18 Thread GitBox
joppevos commented on pull request #8808: URL: https://github.com/apache/airflow/pull/8808#issuecomment-630016214 @mik-laj You are helping me at every step ;) thanks. It seems that the `example_build.yaml` cannot be found. Currently, the file is in the same folder as the example. Should

[GitHub] [airflow] marclamberti commented on issue #8212: Can't read S3 remote logs using Airflow 1.10.9

2020-05-18 Thread GitBox
marclamberti commented on issue #8212: URL: https://github.com/apache/airflow/issues/8212#issuecomment-630022884 Well, in my case it works for both reading and writing logs from S3 with Airflow 1.10.10 and the Kubernetes executor :/

[GitHub] [airflow] thesuperzapper commented on pull request #8777: Add Production Helm chart support

2020-05-18 Thread GitBox
thesuperzapper commented on pull request #8777: URL: https://github.com/apache/airflow/pull/8777#issuecomment-630026860 I am also working on something similar, which is initially a rewrite of the stable/airflow chart. This

[GitHub] [airflow] marclamberti edited a comment on issue #8212: Can't read S3 remote logs using Airflow 1.10.9

2020-05-18 Thread GitBox
marclamberti edited a comment on issue #8212: URL: https://github.com/apache/airflow/issues/8212#issuecomment-630022884 Well, in my case it works for both reading and writing logs from S3 with Airflow 1.10.10 and the Kubernetes executor I agreed it could be simpler :)

[GitHub] [airflow] realradical commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical commented on a change in pull request #8875: URL: https://github.com/apache/airflow/pull/8875#discussion_r426478960 ## File path: tests/www/test_views.py ## @@ -2601,14 +2601,14 @@ def test_create_dagrun(self): dr =

[GitHub] [airflow] ashb commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
ashb commented on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-630051088 Looking better @realradical! Can you test that changing the timezone in the UI updates this field while the page is open? (If it doesn't it means it's missing an html class)

[GitHub] [airflow] mik-laj commented on pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-18 Thread GitBox
mik-laj commented on pull request #8808: URL: https://github.com/apache/airflow/pull/8808#issuecomment-630024504 @joppevos Yes. Here is example: https://github.com/apache/airflow/issues/8527 This is an automated message from

[airflow] tag master-nightly updated (1bdaca9 -> 2121f49)

2020-05-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to tag master-nightly in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag master-nightly was modified! *** from 1bdaca9 (commit) to 2121f49 (commit) from

[airflow] tag nightly-master created (now 2121f49)

2020-05-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. at 2121f49 (commit) No new revisions were added by this update.

[GitHub] [airflow] zacharya19 commented on a change in pull request #8897: Filter dags by clicking on tag

2020-05-18 Thread GitBox
zacharya19 commented on a change in pull request #8897: URL: https://github.com/apache/airflow/pull/8897#discussion_r426428010 ## File path: airflow/www/templates/airflow/dags.html ## @@ -116,7 +116,9 @@ DAGs {% for tag in dag.tags |

[GitHub] [airflow] mik-laj edited a comment on pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
mik-laj edited a comment on pull request #8824: URL: https://github.com/apache/airflow/pull/8824#issuecomment-629976918 Hi. Can you change default values to 60s? Cheers This is an automated message from the Apache Git

[GitHub] [airflow] mik-laj commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426461391 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -180,6 +178,7 @@ def create_dlp_job( timeout: Optional[float] = None,

[GitHub] [airflow] mik-laj commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426461912 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -207,6 +206,9 @@ def create_dlp_job( until it is set to DONE. :type

[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8875: URL: https://github.com/apache/airflow/pull/8875#discussion_r426477404 ## File path: tests/www/test_views.py ## @@ -2601,14 +2601,14 @@ def test_create_dagrun(self): dr = self.session.query(models.DagRun).one() -

[GitHub] [airflow] realradical commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical commented on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-630081091 > @conf_vars @ashb This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] realradical closed pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical closed pull request #8875: URL: https://github.com/apache/airflow/pull/8875 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] ashb commented on pull request #8898: Correct generation and package generation scripts

2020-05-18 Thread GitBox
ashb commented on pull request #8898: URL: https://github.com/apache/airflow/pull/8898#issuecomment-630095741 > @ashb @kaxil -> I figured out that we have really two modes of release - one for "SVN" where we only rename the files with rc but the version inside is without the suffix, and

[GitHub] [airflow] JavierLTPromofarma opened a new issue #8899: Get UI color of DAGs from airflow.cfg

2020-05-18 Thread GitBox
JavierLTPromofarma opened a new issue #8899: URL: https://github.com/apache/airflow/issues/8899 **Description** Being able to get different ui colors than the default ones for operators from airflow.cfg file **Use case / motivation** I want to have more vivid colors for

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8899: Get UI color of DAGs from airflow.cfg

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on issue #8899: URL: https://github.com/apache/airflow/issues/8899#issuecomment-630078553 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] realradical edited a comment on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical edited a comment on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-630081091 > @conf_vars @ashb I tried changing the timezone in the UI and it works! I used `@conf_vars({("core", "default_timezone"): "America/Toronto"})` in the

[GitHub] [airflow] feluelle commented on a change in pull request #8877: Add S3ToRedshift example dag and system test

2020-05-18 Thread GitBox
feluelle commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r426519274 ## File path: tests/test_utils/postgres_system_helpers.py ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts

2020-05-18 Thread GitBox
potiuk commented on pull request #8898: URL: https://github.com/apache/airflow/pull/8898#issuecomment-630097988 > > @ashb @kaxil -> I figured out that we have really two modes of release - one for "SVN" where we only rename the files with rc but the version inside is without the suffix,

[GitHub] [airflow] jeffolsi closed issue #8565: SalesforceHook missing method to return dataframe

2020-05-18 Thread GitBox
jeffolsi closed issue #8565: URL: https://github.com/apache/airflow/issues/8565 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] ashb commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
ashb commented on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-630127673 @realradical I think the backend change you have here is good! With it it should accept any datetime that the UI displays, but without it it would only accept TZ in the current

[GitHub] [airflow] ashb commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426567371 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -207,6 +206,9 @@ def create_dlp_job( until it is set to DONE. :type

[GitHub] [airflow] ashb commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426567978 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -207,6 +206,9 @@ def create_dlp_job( until it is set to DONE. :type

[GitHub] [airflow] potiuk commented on pull request #8833: Azure storage 0.37.0 is not installable any more

2020-05-18 Thread GitBox
potiuk commented on pull request #8833: URL: https://github.com/apache/airflow/pull/8833#issuecomment-630382411 It's fixed in master If you try to install released 1.10.10 version in a stable way you should follow the

[GitHub] [airflow] dimberman commented on pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman commented on pull request #8904: URL: https://github.com/apache/airflow/pull/8904#issuecomment-630396516 Many k8soperator/k8sexecutor PRs need to be essentially recreated from scratch because git can no longer merge. I'm hoping to make a few PRs like this to bring 1-10 and

[GitHub] [airflow] VijayantSoni commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
VijayantSoni commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426907487 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] VijayantSoni commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
VijayantSoni commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426907487 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] dimberman commented on a change in pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman commented on a change in pull request #8904: URL: https://github.com/apache/airflow/pull/8904#discussion_r426895969 ## File path: .github/workflows/ci.yml ## @@ -123,7 +123,7 @@ jobs: run: ./scripts/ci/ci_free_space_on_ci.sh - name: "Build CI image

[GitHub] [airflow] dimberman commented on pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman commented on pull request #8904: URL: https://github.com/apache/airflow/pull/8904#issuecomment-630442531 @potiuk do you have any idea why all of these unrelated flake tests would fail? This is an automated message

[GitHub] [airflow] athenawisdoms opened a new issue #8905: Logging with Custom Filenames

2020-05-18 Thread GitBox
athenawisdoms opened a new issue #8905: URL: https://github.com/apache/airflow/issues/8905 I am using Apache Airflow 1.10.10 to run several Python scripts in a DAG via the `BashOperator`. The logs are currently written to `/usr/airflow/logs`. Is it possible to configure Airflow

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8905: Logging with Custom Filenames

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on issue #8905: URL: https://github.com/apache/airflow/issues/8905#issuecomment-630466226 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] turbaszek commented on issue #8903: BigQueryHook refactor + deterministic BQ Job ID

2020-05-18 Thread GitBox
turbaszek commented on issue #8903: URL: https://github.com/apache/airflow/issues/8903#issuecomment-630443552 Hi @albertocalderari, I'm curently workining on refactor of BQ integration. I decided to abandon custom "run" and use `insert_job` method which will accept `job_id`:

[GitHub] [airflow] ashb commented on pull request #8836: WIP - Add 3.8 to the test matrices.

2020-05-18 Thread GitBox
ashb commented on pull request #8836: URL: https://github.com/apache/airflow/pull/8836#issuecomment-630459846 Pymssql is already "replaced" by odbc in #6850, so I'd suggest just updating the deps etc so we don't try to install mssql on Py3.8, and skip those tests on 3.8 too.

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426577191 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] JavierLopezT opened a new pull request #8901: Secrets backend

2020-05-18 Thread GitBox
JavierLopezT opened a new pull request #8901: URL: https://github.com/apache/airflow/pull/8901 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Unit tests coverage for changes (not needed for

[GitHub] [airflow] dimberman opened a new pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman opened a new pull request #8904: URL: https://github.com/apache/airflow/pull/8904 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] dimberman commented on pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman commented on pull request #8904: URL: https://github.com/apache/airflow/pull/8904#issuecomment-630438161 @ashb some point tomorrow I might need some help on this PR. I'm getting a bunch of flake8 failures that seem completely unrelated to any code changes I made so not sure if

[GitHub] [airflow] jaketf commented on issue #8673: Data Fusion Hook Start pipeline will succeed before pipeline is in RUNNING state

2020-05-18 Thread GitBox
jaketf commented on issue #8673: URL: https://github.com/apache/airflow/issues/8673#issuecomment-630444758 just an update it will be 2-3 weeks before anyone on my team has bandwidth for this. This is an automated message

[GitHub] [airflow] ashb commented on a change in pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8904: URL: https://github.com/apache/airflow/pull/8904#discussion_r426895275 ## File path: .github/workflows/ci.yml ## @@ -123,7 +123,7 @@ jobs: run: ./scripts/ci/ci_free_space_on_ci.sh - name: "Build CI image ${{

[GitHub] [airflow] ashb commented on a change in pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8904: URL: https://github.com/apache/airflow/pull/8904#discussion_r426896828 ## File path: .github/workflows/ci.yml ## @@ -123,7 +123,7 @@ jobs: run: ./scripts/ci/ci_free_space_on_ci.sh - name: "Build CI image ${{

[GitHub] [airflow] dimberman edited a comment on pull request #8904: Move k8s files out of contrib

2020-05-18 Thread GitBox
dimberman edited a comment on pull request #8904: URL: https://github.com/apache/airflow/pull/8904#issuecomment-630442531 @potiuk do you have any idea why all of these unrelated flake tests would fail? ``` tests/contrib/utils/base_gcp_system_test_case.py:89:0: F523

[GitHub] [airflow] crazy-2020 commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
crazy-2020 commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426757445 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -207,6 +206,9 @@ def create_dlp_job( until it is set to DONE.

[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts

2020-05-18 Thread GitBox
potiuk commented on pull request #8898: URL: https://github.com/apache/airflow/pull/8898#issuecomment-630361086 * Wheels are now only -py3 python type * We have now two options: ``` --version-suffix-for-svn flag to generate rc candidate packages to upload to SVN or

[GitHub] [airflow] ashb commented on a change in pull request #8836: WIP - Add 3.8 to the test matrices.

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8836: URL: https://github.com/apache/airflow/pull/8836#discussion_r426592132 ## File path: requirements/requirements-python3.8.txt ## @@ -0,0 +1,370 @@ +# Editable install with no version control (apache-airflow==2.0.0.dev0) Review

[GitHub] [airflow] ashb commented on a change in pull request #8836: WIP - Add 3.8 to the test matrices.

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8836: URL: https://github.com/apache/airflow/pull/8836#discussion_r426591912 ## File path: requirements/requirements-python3.8.txt ## @@ -0,0 +1,370 @@ +# Editable install with no version control (apache-airflow==2.0.0.dev0) Review

[GitHub] [airflow] khyurri commented on a change in pull request #8818: WIP: Add support for Microsoft Azure Blob Storage in Google Cloud Storage Transfer Service operators

2020-05-18 Thread GitBox
khyurri commented on a change in pull request #8818: URL: https://github.com/apache/airflow/pull/8818#discussion_r426614269 ## File path: airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py ## @@ -97,8 +97,15 @@ class GcpTransferOperationStatus:

[GitHub] [airflow] realradical opened a new pull request #8902: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical opened a new pull request #8902: URL: https://github.com/apache/airflow/pull/8902 This PR is to address the following bug: https://github.com/apache/airflow/issues/8842 Airflow 1.10.10 displays datetime with timezone on the web UI. Currently, when user tries to create a dag

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8903: BigQueryHook refactor + deterministic BQ Job ID

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on issue #8903: URL: https://github.com/apache/airflow/issues/8903#issuecomment-630288970 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] albertocalderari opened a new issue #8903: BigQueryHook refactor + deterministic BQ Job ID

2020-05-18 Thread GitBox
albertocalderari opened a new issue #8903: URL: https://github.com/apache/airflow/issues/8903 **Description** Looking at the code it seems like a lot of the logic in the BQ Hook is already implemented in the google API python library. This includes job polling, a nicer way

[GitHub] [airflow] Sinsin1367 commented on pull request #8734: Added optional logging for pod container statuses once a pod fails. T…

2020-05-18 Thread GitBox
Sinsin1367 commented on pull request #8734: URL: https://github.com/apache/airflow/pull/8734#issuecomment-630346426 @ashb @chrismclennon Do you mind taking a quick look and approving this if no more comments? This is an

[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts

2020-05-18 Thread GitBox
potiuk commented on pull request #8898: URL: https://github.com/apache/airflow/pull/8898#issuecomment-630352556 Hey @ashb @kaxil - I think I corrected all the issues that were mentioned in rc1 thread :). Please take a look :)

[GitHub] [airflow] crazy-2020 commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
crazy-2020 commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426760551 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -207,6 +206,9 @@ def create_dlp_job( until it is set to DONE.

[GitHub] [airflow] jblakeman commented on issue #8484: Airflow 1.10.7+ suppresses Operator logs

2020-05-18 Thread GitBox
jblakeman commented on issue #8484: URL: https://github.com/apache/airflow/issues/8484#issuecomment-630315582 Had a chance to look through #6627 and I could be mistaken but it seems like using `os._exit` in the forked child process without first flushing stdio buffers could be a factor

[GitHub] [airflow] joppevos commented on pull request #8808: Support YAML input for CloudBuildCreateOperator

2020-05-18 Thread GitBox
joppevos commented on pull request #8808: URL: https://github.com/apache/airflow/pull/8808#issuecomment-630329569 @mik-laj The absolute path did indeed the trick. Build is passing now :thumbsup: This is an automated message

[GitHub] [airflow] crazy-2020 commented on a change in pull request #8824: Make 'time_to_sleep_in_seconds' in dlp.py configurable

2020-05-18 Thread GitBox
crazy-2020 commented on a change in pull request #8824: URL: https://github.com/apache/airflow/pull/8824#discussion_r426758286 ## File path: airflow/providers/google/cloud/hooks/dlp.py ## @@ -180,6 +178,7 @@ def create_dlp_job( timeout: Optional[float] = None,

[GitHub] [airflow] jblakeman edited a comment on issue #8484: Airflow 1.10.7+ suppresses Operator logs

2020-05-18 Thread GitBox
jblakeman edited a comment on issue #8484: URL: https://github.com/apache/airflow/issues/8484#issuecomment-630315582 Had a chance to skim through #6627 and I could be mistaken but it seems like using `os._exit` in the forked child process without first flushing stdio buffers could be a

[GitHub] [airflow] josh-gree commented on pull request #7954: Pinning max pandas version to 2.0 (lesser than) to allow pandas 1.0.

2020-05-18 Thread GitBox
josh-gree commented on pull request #7954: URL: https://github.com/apache/airflow/pull/7954#issuecomment-630178641 Hey thanks for this PR great to be able to have pandas > 1.0! Can I ask when this will make its way into a release?

[GitHub] [airflow] randr97 commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-18 Thread GitBox
randr97 commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r426706026 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -664,6 +664,23 @@ def copy_object(self,

[GitHub] [airflow] ashb commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r426569097 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -664,6 +664,23 @@ def copy_object(self,

[GitHub] [airflow] felipefb commented on pull request #8833: Azure storage 0.37.0 is not installable any more

2020-05-18 Thread GitBox
felipefb commented on pull request #8833: URL: https://github.com/apache/airflow/pull/8833#issuecomment-630246885 I tried to install the azure package now and had a similar error. Is the new version already available? ``` Collecting azure-storage>=0.34.0; extra == "azure"

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426575633 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426575179 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426578034 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426577707 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] realradical commented on pull request #8902: [AIRFLOW-8902] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical commented on pull request #8902: URL: https://github.com/apache/airflow/pull/8902#issuecomment-630230910 @ashb sorry, accidentally closed https://github.com/apache/airflow/pull/8875 added the logic to parse datetime without TZ, and more test cases, please check if it is

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426578726 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[jira] [Commented] (AIRFLOW-6981) Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110271#comment-17110271 ] ASF GitHub Bot commented on AIRFLOW-6981: - ryanyuan commented on a change in pull request

[GitHub] [airflow] ryanyuan commented on a change in pull request #8575: [AIRFLOW-6981] Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread GitBox
ryanyuan commented on a change in pull request #8575: URL: https://github.com/apache/airflow/pull/8575#discussion_r426618675 ## File path: airflow/providers/google/cloud/operators/cloud_build.py ## @@ -15,186 +15,678 @@ # KIND, either express or implied. See the License for

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8900: Info about the parameter "conf" when triggering a DAG

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on issue #8900: URL: https://github.com/apache/airflow/issues/8900#issuecomment-630186616 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] AmarEL opened a new issue #8900: Info about the parameter "conf" when triggering a DAG

2020-05-18 Thread GitBox
AmarEL opened a new issue #8900: URL: https://github.com/apache/airflow/issues/8900 **Description** The information and examples about the optional JSON `conf` when triggering a dag from UI or by CLI needs to be included in the documentation. **Use case / motivation**

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426576775 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] ashb commented on a change in pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-18 Thread GitBox
ashb commented on a change in pull request #8701: URL: https://github.com/apache/airflow/pull/8701#discussion_r426576297 ## File path: airflow/providers/amazon/aws/hooks/elasticache.py ## @@ -0,0 +1,259 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] realradical commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue

2020-05-18 Thread GitBox
realradical commented on pull request #8875: URL: https://github.com/apache/airflow/pull/8875#issuecomment-630209425 @ashb i accidentally closed the pr. will create another one now. sorry about it This is an automated

[GitHub] [airflow] randr97 commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-18 Thread GitBox
randr97 commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r426587315 ## File path: airflow/providers/amazon/aws/operators/s3_bucket.py ## @@ -0,0 +1,132 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] randr97 commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-18 Thread GitBox
randr97 commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r426588128 ## File path: airflow/providers/amazon/aws/operators/s3_bucket.py ## @@ -0,0 +1,132 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] potiuk commented on a change in pull request #8836: WIP - Add 3.8 to the test matrices.

2020-05-18 Thread GitBox
potiuk commented on a change in pull request #8836: URL: https://github.com/apache/airflow/pull/8836#discussion_r426609122 ## File path: requirements/requirements-python3.8.txt ## @@ -0,0 +1,370 @@ +# Editable install with no version control (apache-airflow==2.0.0.dev0)

[GitHub] [airflow] chamcca commented on a change in pull request #8749: add AWS StepFunctions integrations to the aws provider

2020-05-18 Thread GitBox
chamcca commented on a change in pull request #8749: URL: https://github.com/apache/airflow/pull/8749#discussion_r426622398 ## File path: airflow/providers/amazon/aws/operators/step_function_start_execution.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation

[GitHub] [airflow] chamcca commented on a change in pull request #8749: add AWS StepFunctions integrations to the aws provider

2020-05-18 Thread GitBox
chamcca commented on a change in pull request #8749: URL: https://github.com/apache/airflow/pull/8749#discussion_r426622233 ## File path: airflow/providers/amazon/aws/operators/step_function_get_execution_output.py ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software

[jira] [Commented] (AIRFLOW-6290) Create guide for GKE service operator

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110786#comment-17110786 ] ASF GitHub Bot commented on AIRFLOW-6290: - mik-laj commented on a change in pull request #8883:

[jira] [Commented] (AIRFLOW-6290) Create guide for GKE service operator

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110797#comment-17110797 ] ASF GitHub Bot commented on AIRFLOW-6290: - mik-laj commented on a change in pull request #8883:

[GitHub] [airflow] listenLearning opened a new issue #8906: make success failed

2020-05-18 Thread GitBox
listenLearning opened a new issue #8906: URL: https://github.com/apache/airflow/issues/8906 I have recently installed a standalone version of airflow, version 1.10, then I created a dag, the task time is 2020-01-01-01, but maybe some tasks I want to start from the same day, instead of

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8906: make success failed

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on issue #8906: URL: https://github.com/apache/airflow/issues/8906#issuecomment-630542345 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[jira] [Commented] (AIRFLOW-6290) Create guide for GKE service operator

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110799#comment-17110799 ] ASF GitHub Bot commented on AIRFLOW-6290: - mik-laj commented on a change in pull request #8883:

[GitHub] [airflow] mik-laj commented on a change in pull request #8883: [AIRFLOW-6290] Create guide for GKE operators

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8883: URL: https://github.com/apache/airflow/pull/8883#discussion_r426999545 ## File path: docs/howto/operator/gcp/kubernetes_engine.rst ## @@ -0,0 +1,122 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

[jira] [Commented] (AIRFLOW-6981) Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110816#comment-17110816 ] ASF GitHub Bot commented on AIRFLOW-6981: - mik-laj commented on a change in pull request #8575:

[GitHub] [airflow] mik-laj commented on a change in pull request #8575: [AIRFLOW-6981] Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8575: URL: https://github.com/apache/airflow/pull/8575#discussion_r427008238 ## File path: airflow/providers/google/cloud/operators/cloud_build.py ## @@ -15,186 +15,678 @@ # KIND, either express or implied. See the License for

[airflow] branch master updated (2121f49 -> 841d816)

2020-05-18 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 2121f49 Avoid failure on transient requirements in CI image (#8892) add 841d816 Allow setting the

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8824: Allow setting the pooling time in DLPHook

2020-05-18 Thread GitBox
boring-cyborg[bot] commented on pull request #8824: URL: https://github.com/apache/airflow/pull/8824#issuecomment-630548745 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] mik-laj merged pull request #8824: Allow setting the pooling time in DLPHook

2020-05-18 Thread GitBox
mik-laj merged pull request #8824: URL: https://github.com/apache/airflow/pull/8824 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (AIRFLOW-6981) Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110821#comment-17110821 ] ASF GitHub Bot commented on AIRFLOW-6981: - mik-laj commented on pull request #8575: URL:

[GitHub] [airflow] mik-laj commented on pull request #8575: [AIRFLOW-6981] Move Google Cloud Build from Discovery API to Python Library

2020-05-18 Thread GitBox
mik-laj commented on pull request #8575: URL: https://github.com/apache/airflow/pull/8575#issuecomment-630558831 I tried to run system tests and unfortunately none of the tasks was successful. ``` [2020-05-19 03:29:06,688] {taskinstance.py:719} INFO - Dependencies all

[GitHub] [airflow] enriqueayala commented on issue #8839: "Dag Runs" UI page shows broken State after enabled DAGs serialization

2020-05-18 Thread GitBox
enriqueayala commented on issue #8839: URL: https://github.com/apache/airflow/issues/8839#issuecomment-630520178 Having same issue for versions 1.10.10/ 1.10.9 and store_serialized_dags=False/True . No solution found yet..

[GitHub] [airflow] mik-laj commented on a change in pull request #8883: [AIRFLOW-6290] Create guide for GKE operators

2020-05-18 Thread GitBox
mik-laj commented on a change in pull request #8883: URL: https://github.com/apache/airflow/pull/8883#discussion_r426994046 ## File path: docs/howto/operator/gcp/kubernetes_engine.rst ## @@ -0,0 +1,122 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

  1   2   >