[jira] [Commented] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
[
https://issues.apache.org/jira/browse/AIRFLOW-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566253#comment-16566253
]
ASF GitHub Bot commented on AIRFLOW-2839:
-
XD-DENG opened a new pull request #3678: [AIRFLOW-2839] Refine Doc
Concepts->Connections
URL: https://github.com/apache/incubator-airflow/pull/3678
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2839
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
We can define connections either in metadata database or environment
variables.
However, how about there are connections with the same conn_id in both
metadata database and environment variables? Will one of them be picked? Which
one? Or a random one will be picked?
The answer is: Airflow will check environment variables
first.https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70
The "random picking" mechanism will only be applicable for multiple
connections with the same conn_id in metadata database (if there is no
connection with the same conn_id defined in environment variables).
This should be made clear in the documentation.
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Answer a question in doc: what if I have connections with the same conn_id in
> both metadata and environment variables?
> --
>
> Key: AIRFLOW-2839
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2839
> Project: Apache Airflow
> Issue Type: Improvement
> Components: configuration, Documentation
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Critical
>
> We can define connections either in metadata database or environment
> variables.
>
> However, how about there are connections with the same conn_id in both
> metadata database and environment variables? Will one of them be picked?
> Which one? Or a random one will be picked?
>
> The answer is: Airflow will check environment variables
> first.[https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70]
> The "random picking" mechanism will only be applicable for multiple
> connections with the same conn_id in metadata database (if there is no
> connection with the same conn_id defined in environment variables).
>
> This should be made clear in the documentation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
Xiaodong DENG created AIRFLOW-2839: -- Summary: Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables? Key: AIRFLOW-2839 URL: https://issues.apache.org/jira/browse/AIRFLOW-2839 Project: Apache Airflow Issue Type: Improvement Components: configuration, Documentation Reporter: Xiaodong DENG Assignee: Xiaodong DENG We can define connections either in metadata database or environment variables. However, how about there are connections with the same conn_id in both metadata database and environment variables? Will one of them be picked? Which one? Or a random one will be picked? The answer is: Airflow will check environment variables first.[https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70] The "random picking" mechanism will only be applicable for multiple connections with the same conn_id in metadata database (if there is no connection with the same conn_id defined in environment variables). This should be made clear in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2838) Installtion issues: Could not install packages due to an EnvironmentError
Raja created AIRFLOW-2838:
-
Summary: Installtion issues: Could not install packages due to an
EnvironmentError
Key: AIRFLOW-2838
URL: https://issues.apache.org/jira/browse/AIRFLOW-2838
Project: Apache Airflow
Issue Type: Bug
Reporter: Raja
Hello,
Am trying to install airflow (sudo pip install apache-airflow) on my mac.
Getting below error. Can someone help me with that issue?
Could not install packages due to an EnvironmentError:
[('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py',
"[Errno 1] Operation not permitted:
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"),
('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py',
'/private/tmp/pip-uninstall-bXYiIZ/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py',
"[Errno 1]
[jira] [Commented] (AIRFLOW-2826) Add hook for Google Cloud KMS
[
https://issues.apache.org/jira/browse/AIRFLOW-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566064#comment-16566064
]
ASF GitHub Bot commented on AIRFLOW-2826:
-
jakahn opened a new pull request #3677: [AIRFLOW-2826] Add GoogleCloudKMSHook
URL: https://github.com/apache/incubator-airflow/pull/3677
### Jira
- [X] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title
- https://issues.apache.org/jira/browse/AIRFLOW-2826
### Description
- [X] Here are some details about my PR (no UI changes):
- Adds a hook enabling encryption and decryption through Google Cloud
KMS. Both operations also support the use of the "additionalAuthenticatedData"
field for the requests.
- This hook is also added in anticipation of integration with future work
on [AIRFLOW-2062](https://issues.apache.org/jira/browse/AIRFLOW-2062).
### Tests
- [X] My PR adds the following unit tests:
- `tests/contrib/hooks/test_gcp_kms_hook.py`
### Commits
- [X] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [X] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [X] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add hook for Google Cloud KMS
> -
>
> Key: AIRFLOW-2826
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2826
> Project: Apache Airflow
> Issue Type: Improvement
> Components: hooks
>Reporter: Jasper Kahn
>Assignee: Jasper Kahn
>Priority: Minor
> Labels: features
>
> Add a hook to support interacting with Google Cloud KMS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2829) Brush up the CI script for minikube
[
https://issues.apache.org/jira/browse/AIRFLOW-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566015#comment-16566015
]
ASF GitHub Bot commented on AIRFLOW-2829:
-
r39132 closed pull request #3676: [AIRFLOW-2829] Brush up the CI script for
minikube
URL: https://github.com/apache/incubator-airflow/pull/3676
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/scripts/ci/kubernetes/minikube/start_minikube.sh
b/scripts/ci/kubernetes/minikube/start_minikube.sh
index 5171a26313..c9388e08ff 100755
--- a/scripts/ci/kubernetes/minikube/start_minikube.sh
+++ b/scripts/ci/kubernetes/minikube/start_minikube.sh
@@ -24,19 +24,19 @@ _MY_SCRIPT="${BASH_SOURCE[0]}"
_MY_DIR=$(cd "$(dirname "$_MY_SCRIPT")" && pwd)
# Avoids 1.7.x because of https://github.com/kubernetes/minikube/issues/2240
_KUBERNETES_VERSION="${KUBERNETES_VERSION}"
+_MINIKUBE_VERSION="${MINIKUBE_VERSION:-v0.26.0}"
-echo "setting up kubernetes ${_KUBERNETES_VERSION}"
+echo "setting up kubernetes ${_KUBERNETES_VERSION}, using minikube
${_MINIKUBE_VERSION}"
-_MINIKUBE_VERSION="v0.26.0"
-_HELM_VERSION=v2.8.1
-_VM_DRIVER=none
+_VM_DRIVER="${VM_DRIVER:-none}"
USE_MINIKUBE_DRIVER_NONE=true
_UNAME_OUT=$(uname -s)
case "${_UNAME_OUT}" in
Linux*) _MY_OS=linux;;
Darwin*)_MY_OS=darwin;;
-*) _MY_OS="UNKNOWN:${unameOut}"
+*) echo "${_UNAME_OUT} is unsupported."
+exit 1;;
esac
echo "Local OS is ${_MY_OS}"
@@ -58,27 +58,25 @@ if [[ ! -x /usr/local/bin/kubectl ]]; then
curl -Lo bin/kubectl \
https://storage.googleapis.com/kubernetes-release/release/${_KUBERNETES_VERSION}/bin/${_MY_OS}/amd64/kubectl
chmod +x bin/kubectl
+ sudo mv bin/kubectl /usr/local/bin/kubectl
fi
if [[ ! -x /usr/local/bin/minikube ]]; then
echo Downloading minikube.
curl -Lo bin/minikube \
https://storage.googleapis.com/minikube/releases/${_MINIKUBE_VERSION}/minikube-${_MY_OS}-amd64
chmod +x bin/minikube
+ sudo mv bin/minikube /usr/local/bin/minikube
fi
-sudo mv bin/minikube /usr/local/bin/minikube
-sudo mv bin/kubectl /usr/local/bin/kubectl
-
export PATH="${_MY_DIR}/bin:$PATH"
if [[ "${USE_MINIKUBE_DRIVER_NONE:-}" = "true" ]]; then
# Run minikube with none driver.
# See
https://blog.travis-ci.com/2017-10-26-running-kubernetes-on-travis-ci-with-minikube
- _VM_DRIVER="--vm-driver=none"
+ _VM_DRIVER=none
if [[ ! -x /usr/local/bin/nsenter ]]; then
# From
https://engineering.bitnami.com/articles/implementing-kubernetes-integration-tests-in-travis.html
# Travis ubuntu trusty env doesn't have nsenter, needed for
--vm-driver=none
-which nsenter >/dev/null && return 0
echo "INFO: Building 'nsenter' ..."
cat <<-EOF | docker run -i --rm -v "$(pwd):/build" ubuntu:14.04 >&
nsenter.build.log
apt-get update
@@ -93,7 +91,7 @@ EOF
if [ ! -f ./nsenter ]; then
echo "ERROR: nsenter build failed, log:"
cat nsenter.build.log
-return 1
+exit 1
fi
echo "INFO: nsenter build OK"
sudo mv ./nsenter /usr/local/bin
@@ -102,10 +100,10 @@ fi
echo "your path is ${PATH}"
-_MINIKUBE="sudo PATH=$PATH minikube"
+_MINIKUBE="sudo -E PATH=$PATH minikube"
$_MINIKUBE config set bootstrapper localkube
-$_MINIKUBE start --kubernetes-version=${_KUBERNETES_VERSION} --vm-driver=none
+$_MINIKUBE start --kubernetes-version=${_KUBERNETES_VERSION}
--vm-driver=${_VM_DRIVER}
$_MINIKUBE update-context
# Wait for Kubernetes to be up and ready.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Brush up the CI script for minikube
> ---
>
> Key: AIRFLOW-2829
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2829
> Project: Apache Airflow
> Issue Type: Bug
> Components: ci
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> Ran {{scripts/ci/kubernetes/minikube/start_minikube.sh}} locally and found
> some points that can be improved:
> - minikube version is hard-coded
> - Defined but unused variables: {{$_HELM_VERSION}}, {{$_VM_DRIVER}}
> - Undefined variables: {{$unameOut}}
> - The following lines cause warnings if download is skipped:
> {code}
> 69 sudo mv bin/minikube /usr/local/bin/minikube
> 70 sudo mv bin/kubectl /usr/local/bin/kubectl
> {code}
> - {{return}} s at line 81 and 96 won't work since it's
[jira] [Commented] (AIRFLOW-2829) Brush up the CI script for minikube
[
https://issues.apache.org/jira/browse/AIRFLOW-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566016#comment-16566016
]
ASF subversion and git services commented on AIRFLOW-2829:
--
Commit 0d193ada44707ec6f50371cb1fc9cbb96d2466db in incubator-airflow's branch
refs/heads/master from [~sekikn]
[ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=0d193ad ]
[AIRFLOW-2829] Brush up the CI script for minikube
Fix scripts/ci/kubernetes/minikube/start_minikube.sh
as follows:
- Make minikube version configurable via
environment variable
- Remove unused variables for readability
- Reorder some lines to remove warnings
- Replace ineffective `return` with `exit`
- Add -E to `sudo minikube` so that non-root
users can use this script locally
> Brush up the CI script for minikube
> ---
>
> Key: AIRFLOW-2829
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2829
> Project: Apache Airflow
> Issue Type: Bug
> Components: ci
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> Ran {{scripts/ci/kubernetes/minikube/start_minikube.sh}} locally and found
> some points that can be improved:
> - minikube version is hard-coded
> - Defined but unused variables: {{$_HELM_VERSION}}, {{$_VM_DRIVER}}
> - Undefined variables: {{$unameOut}}
> - The following lines cause warnings if download is skipped:
> {code}
> 69 sudo mv bin/minikube /usr/local/bin/minikube
> 70 sudo mv bin/kubectl /usr/local/bin/kubectl
> {code}
> - {{return}} s at line 81 and 96 won't work since it's outside of a function
> - To run this script as a non-root user, {{-E}} is required for {{sudo}}. See
> https://github.com/kubernetes/minikube/issues/1883.
> {code}
> 105 _MINIKUBE="sudo PATH=$PATH minikube"
> 106
> 107 $_MINIKUBE config set bootstrapper localkube
> 108 $_MINIKUBE start --kubernetes-version=${_KUBERNETES_VERSION}
> --vm-driver=none
> 109 $_MINIKUBE update-context
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (AIRFLOW-2822) PendingDeprecationWarning Invalid arguments: HipChatAPISendRoomNotificationOperator
[
https://issues.apache.org/jira/browse/AIRFLOW-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Anand closed AIRFLOW-2822.
> PendingDeprecationWarning Invalid arguments:
> HipChatAPISendRoomNotificationOperator
> ---
>
> Key: AIRFLOW-2822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2822
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, operators
>Affects Versions: 2.0.0
>Reporter: Leo Gallucci
>Assignee: Leo Gallucci
>Priority: Trivial
> Labels: easyfix
>
> Using `HipChatAPISendRoomNotificationOperator` on Airflow master branch (2.0)
> gives:
> {code:python}
> airflow/models.py:2390: PendingDeprecationWarning:
> Invalid arguments were passed to HipChatAPISendRoomNotificationOperator.
> Support for passing such arguments will be dropped in Airflow 2.0.
> Invalid arguments were:
> *args: ()
> **kwargs: {'color': 'green'}
> category=PendingDeprecationWarning
> {code}
> I've fixed this in my fork:
> https://github.com/elgalu/apache-airflow/commit/83fc940f54e5d6531f66bff256f66765899dc055
> I will send a PR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (AIRFLOW-2820) In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature in web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand closed AIRFLOW-2820. > In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature > in web UI > --- > > Key: AIRFLOW-2820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2820 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > In documentation page *Scheduling & Triggers*, it only mentioned the CLI > method to manually trigger a DAG run. > However, the manual trigger feature in Web UI should be mentioned as well (it > may be even more frequently used by users). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2820) In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature in web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand resolved AIRFLOW-2820. -- Resolution: Fixed > In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature > in web UI > --- > > Key: AIRFLOW-2820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2820 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > In documentation page *Scheduling & Triggers*, it only mentioned the CLI > method to manually trigger a DAG run. > However, the manual trigger feature in Web UI should be mentioned as well (it > may be even more frequently used by users). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2820) In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature in web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565990#comment-16565990 ] ASF GitHub Bot commented on AIRFLOW-2820: - r39132 closed pull request #3663: [AIRFLOW-2820] Add Web UI triggger in doc "Scheduling & Triggers" URL: https://github.com/apache/incubator-airflow/pull/3663 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/scheduler.rst b/docs/scheduler.rst index 43531ca69d..3e895898fc 100644 --- a/docs/scheduler.rst +++ b/docs/scheduler.rst @@ -134,6 +134,8 @@ specific ``run_id``. The ``DAG Runs`` created externally to the scheduler get associated to the trigger's timestamp, and will be displayed in the UI alongside scheduled ``DAG runs``. +In addition, you can also manually trigger a ``DAG Run`` using the web UI (tab "DAGs" -> column "Links" -> button "Trigger Dag"). + To Keep in Mind ''' This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature > in web UI > --- > > Key: AIRFLOW-2820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2820 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > In documentation page *Scheduling & Triggers*, it only mentioned the CLI > method to manually trigger a DAG run. > However, the manual trigger feature in Web UI should be mentioned as well (it > may be even more frequently used by users). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2820) In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature in web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565991#comment-16565991 ] ASF subversion and git services commented on AIRFLOW-2820: -- Commit b120427b652dc308b8ac6fb93fc096a5b951a084 in incubator-airflow's branch refs/heads/master from Xiaodong [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=b120427 ] [AIRFLOW-2820] Add Web UI triggger in doc "Scheduling & Triggers" In documentation page "Scheduling & Triggers", it only mentioned the CLI method to manually trigger a DAG run. However, the manual trigger feature in Web UI should be mentioned as well (it may be even more frequently used by users). > In Doc "Scheduling & Triggers", missed to mention manual Dag Trigger feature > in web UI > --- > > Key: AIRFLOW-2820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2820 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > In documentation page *Scheduling & Triggers*, it only mentioned the CLI > method to manually trigger a DAG run. > However, the manual trigger feature in Web UI should be mentioned as well (it > may be even more frequently used by users). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2832) Inconsistencies and linter errors across markdown files
[ https://issues.apache.org/jira/browse/AIRFLOW-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2832. Resolution: Fixed Fix Version/s: 2.0.0 > Inconsistencies and linter errors across markdown files > --- > > Key: AIRFLOW-2832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2832 > Project: Apache Airflow > Issue Type: Improvement > Components: docs, Documentation >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > Fix For: 2.0.0 > > > There are a number of inconsistencies within and across markdown files in the > Airflow project. Most of these are simple formatting issues easily fixed by > linting (e.g., with mdl). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2670) SSHOperator's timeout parameter doesn't affect SSHook timeoot
[ https://issues.apache.org/jira/browse/AIRFLOW-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2670. Resolution: Fixed Fix Version/s: 2.0.0 > SSHOperator's timeout parameter doesn't affect SSHook timeoot > - > > Key: AIRFLOW-2670 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2670 > Project: Apache Airflow > Issue Type: Improvement > Components: contrib >Affects Versions: Airflow 2.0 >Reporter: jin zhang >Priority: Major > Fix For: 2.0.0 > > > when I use SSHOperator, SSHOperator's timeout parameter can't set in SSHHook > and it's just effect exce_command. > old version: > self.ssh_hook = SSHHook(ssh_conn_id=self.ssh_conn_id) > I change it to : > self.ssh_hook = SSHHook(ssh_conn_id=self.ssh_conn_id, timeout=self.timeout) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2822) PendingDeprecationWarning Invalid arguments: HipChatAPISendRoomNotificationOperator
[
https://issues.apache.org/jira/browse/AIRFLOW-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor updated AIRFLOW-2822:
---
Affects Version/s: (was: Airflow 2.0)
2.0.0
> PendingDeprecationWarning Invalid arguments:
> HipChatAPISendRoomNotificationOperator
> ---
>
> Key: AIRFLOW-2822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2822
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, operators
>Affects Versions: 2.0.0
>Reporter: Leo Gallucci
>Assignee: Leo Gallucci
>Priority: Trivial
> Labels: easyfix
>
> Using `HipChatAPISendRoomNotificationOperator` on Airflow master branch (2.0)
> gives:
> {code:python}
> airflow/models.py:2390: PendingDeprecationWarning:
> Invalid arguments were passed to HipChatAPISendRoomNotificationOperator.
> Support for passing such arguments will be dropped in Airflow 2.0.
> Invalid arguments were:
> *args: ()
> **kwargs: {'color': 'green'}
> category=PendingDeprecationWarning
> {code}
> I've fixed this in my fork:
> https://github.com/elgalu/apache-airflow/commit/83fc940f54e5d6531f66bff256f66765899dc055
> I will send a PR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2795) Oracle to Oracle Transfer Operator
[ https://issues.apache.org/jira/browse/AIRFLOW-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2795. Resolution: Fixed Fix Version/s: 2.0.0 > Oracle to Oracle Transfer Operator > --- > > Key: AIRFLOW-2795 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2795 > Project: Apache Airflow > Issue Type: New Feature > Components: operators >Reporter: Marcus Rehm >Assignee: Marcus Rehm >Priority: Trivial > Fix For: 2.0.0 > > > This operator should help in transfer data from one Oracle instance to > another or between tables in the same instance. t's suitable in use cases > where you don't want to or it's not allowed use dblink. > The operator needs a sql query and a destination table in order to work. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2800) Remove airflow/ low-hanging linting errors
[ https://issues.apache.org/jira/browse/AIRFLOW-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2800. Resolution: Fixed Fix Version/s: 2.0.0 > Remove airflow/ low-hanging linting errors > -- > > Key: AIRFLOW-2800 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2800 > Project: Apache Airflow > Issue Type: Bug >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > Fix For: 2.0.0 > > > Removing low hanging linting errors from airflow directory > Focuses on > * E226 > * W291 > as well as *some* E501 (line too long) where it did not risk reducing > readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2825) S3ToHiveTransfer operator may not may able to handle GZIP file with uppercase ext in S3
[ https://issues.apache.org/jira/browse/AIRFLOW-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-2825: --- Fix Version/s: 2.0.0 > S3ToHiveTransfer operator may not may able to handle GZIP file with uppercase > ext in S3 > --- > > Key: AIRFLOW-2825 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2825 > Project: Apache Airflow > Issue Type: Bug > Components: operators >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > Fix For: 2.0.0 > > > Because upper/lower case was not considered in the extension check, > S3ToHiveTransfer operator may think a GZIP file with uppercase ext `.GZ` is > not a GZIP file and raise exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2815) Notice cannot contain "onwards" and needs to be specific
[ https://issues.apache.org/jira/browse/AIRFLOW-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2815. Resolution: Fixed Fix Version/s: 1.10.0 > Notice cannot contain "onwards" and needs to be specific > > > Key: AIRFLOW-2815 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2815 > Project: Apache Airflow > Issue Type: Sub-task >Reporter: Bolke de Bruin >Priority: Major > Fix For: 1.10.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2835) Remove python-selinux
[ https://issues.apache.org/jira/browse/AIRFLOW-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2835. Resolution: Fixed Fix Version/s: 2.0.0 > Remove python-selinux > - > > Key: AIRFLOW-2835 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2835 > Project: Apache Airflow > Issue Type: Bug >Reporter: Fokko Driesprong >Priority: Major > Fix For: 2.0.0 > > > This package sometimes crashes the CI and is not required. Therefore it does > not make sense to install it since it will take ci-time and make things > brittle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2817. Resolution: Fixed Fix Version/s: 1.10.0 > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > Fix For: 1.10.0 > > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (AIRFLOW-2661) Kubernetes Executor - dags_volume_subpath and logs_volume_subpath don't do anything
[
https://issues.apache.org/jira/browse/AIRFLOW-2661?focusedWorklogId=129911=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129911
]
ASF GitHub Bot logged work on AIRFLOW-2661:
---
Author: ASF GitHub Bot
Created on: 01/Aug/18 18:59
Start Date: 01/Aug/18 18:59
Worklog Time Spent: 10m
Work Description: ashb commented on issue #3537: [AIRFLOW-2661] fix k8s
config *_volume_subpath
URL:
https://github.com/apache/incubator-airflow/pull/3537#issuecomment-409685506
@bolkedebruin Any objections to me cherry-picking this into v1-10-stable and
-test? I've tried it on my fork and Travis tests passed:
- https://travis-ci.org/ashb/incubator-airflow/builds/410828962
- https://travis-ci.org/ashb/incubator-airflow/builds/410829126
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 129911)
Time Spent: 10m
Remaining Estimate: 23h 50m (was: 24h)
> Kubernetes Executor - dags_volume_subpath and logs_volume_subpath don't do
> anything
> ---
>
> Key: AIRFLOW-2661
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2661
> Project: Apache Airflow
> Issue Type: Bug
> Components: executor
>Affects Versions: Airflow 2.0
> Environment: Kubernetes 1.10 (kops)
>Reporter: Ravi Kotecha
>Assignee: Ravi Kotecha
>Priority: Major
> Labels: Kubernetes, pull-request-available
> Fix For: 2.0.0
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> It looks like the configuration options:
> {code:java}
> dags_volume_subpath =
> logs_volume_subpath =
> {code}
> don't do anything when set. This is because the
> {{_constructVolume()}}
> function is called in
> [https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124].
> It sets subPath on the {{volume}} rather than the mount.
> This function should be changed to not take the optional parameter
> {{subPath}} and we should use {{dags_volume_subpath}} if provided in the
> construction of the {{volumeMount}}
> If you look at
> [https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can
> see that it's a property of {{volumeMount}} not a {{Volume}}. We are
> correctly handling this for the the {{configMap}} on line 167.
> My colleague has also commented on the original PR that added this
> functionality:
> [https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135]
>
> Pull Request:
> https://github.com/apache/incubator-airflow/pull/3537
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (AIRFLOW-2524) Airflow integration with AWS Sagemaker
[ https://issues.apache.org/jira/browse/AIRFLOW-2524?focusedWorklogId=129904=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129904 ] ASF GitHub Bot logged work on AIRFLOW-2524: --- Author: ASF GitHub Bot Created on: 01/Aug/18 18:39 Start Date: 01/Aug/18 18:39 Worklog Time Spent: 10m Work Description: srrajeev-aws commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r206988684 ## File path: airflow/contrib/operators/sagemaker_create_training_job_operator.py ## @@ -0,0 +1,98 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow.contrib.hooks.sagemaker_hook import SageMakerHook +from airflow.models import BaseOperator +from airflow.utils import apply_defaults +from airflow.exceptions import AirflowException + + +class SageMakerCreateTrainingJobOperator(BaseOperator): + +""" + Initiate a SageMaker training + + This operator returns The ARN of the model created in Amazon SageMaker + + :param training_job_config: + The configuration necessary to start a training job (templated) + :type training_job_config: dict + :param region_name: The AWS region_name + :type region_name: string + :param sagemaker_conn_id: The SageMaker connection ID to use. + :type aws_conn_id: string Review comment: @Fokko - To further add to Keliang explanation of separating the operator to kick off the job and sensor to monitor the job is to provide flexibility to users. Based on their use case(s), they may have the requirements to kick of multiple jobs/tasks in parallel and then monitor the completion of all Amazon Sagemaker job(s) downstream. Some these jobs may take hours and we don't want to hold the pipeline to initiate other downstream jobs hampering the users from meeting their required SLA. Since there are many other known and unknown scenarios, we are careful not to club both the initialization and monitoring of the job. The design is similar to the Amazon EMR - https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/example_dags/example_emr_job_flow_manual_steps.py This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 129904) Time Spent: 10m Remaining Estimate: 0h > Airflow integration with AWS Sagemaker > -- > > Key: AIRFLOW-2524 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2524 > Project: Apache Airflow > Issue Type: Improvement > Components: aws, contrib >Reporter: Rajeev Srinivasan >Assignee: Yang Yu >Priority: Major > Labels: AWS > Time Spent: 10m > Remaining Estimate: 0h > > Would it be possible to orchestrate an end to end AWS Sagemaker job using > Airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2837) tenacity 4.8.0 breaks with python3.7
Adrian Bridgett created AIRFLOW-2837: Summary: tenacity 4.8.0 breaks with python3.7 Key: AIRFLOW-2837 URL: https://issues.apache.org/jira/browse/AIRFLOW-2837 Project: Apache Airflow Issue Type: Bug Reporter: Adrian Bridgett Tenacity 4.8.0 (as in setup.py) uses the reserved async keyword. Tenacity seems to lack a changelog, 4.12.0 seems to fix the problem but I don't know what breaking changes may have occurred. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2658) Add GKE specific Kubernetes Pod Operator
[ https://issues.apache.org/jira/browse/AIRFLOW-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565575#comment-16565575 ] ASF GitHub Bot commented on AIRFLOW-2658: - Noremac201 commented on issue #3532: [AIRFLOW-2658] Add GCP specific k8s pod operator URL: https://github.com/apache/incubator-airflow/pull/3532#issuecomment-409633871 Looks like Travis isn't posting, here's my personal Travis build: https://travis-ci.org/Noremac201/incubator-airflow/builds/410543165 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add GKE specific Kubernetes Pod Operator > > > Key: AIRFLOW-2658 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2658 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > > Currently there is a Kubernetes Pod operator, but it is not really easy to > have it work with GCP Kubernetes Engine, it would be nice to have one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2832) Inconsistencies and linter errors across markdown files
[ https://issues.apache.org/jira/browse/AIRFLOW-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565368#comment-16565368 ] ASF GitHub Bot commented on AIRFLOW-2832: - tedmiston commented on issue #3670: [AIRFLOW-2832] Lint and resolve inconsistencies in Markdown files URL: https://github.com/apache/incubator-airflow/pull/3670#issuecomment-409585654 @Fokko Thanks for the quick merge! I'll make a note to look into linting the bash code in Airflow and see if we have enough for a PR there. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Inconsistencies and linter errors across markdown files > --- > > Key: AIRFLOW-2832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2832 > Project: Apache Airflow > Issue Type: Improvement > Components: docs, Documentation >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > > There are a number of inconsistencies within and across markdown files in the > Airflow project. Most of these are simple formatting issues easily fixed by > linting (e.g., with mdl). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2829) Brush up the CI script for minikube
[
https://issues.apache.org/jira/browse/AIRFLOW-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565363#comment-16565363
]
ASF GitHub Bot commented on AIRFLOW-2829:
-
codecov-io commented on issue #3676: [AIRFLOW-2829] Brush up the CI script for
minikube
URL:
https://github.com/apache/incubator-airflow/pull/3676#issuecomment-40958
#
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=h1)
Report
> Merging
[#3676](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=desc)
into
[master](https://codecov.io/gh/apache/incubator-airflow/commit/c37fc0b6ba19e3fe5656ae37cef9b59cef3c29e8?src=pr=desc)
will **not change** coverage.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=tree)
```diff
@@ Coverage Diff @@
## master #3676 +/- ##
==
Coverage77.5% 77.5%
==
Files 205 205
Lines 15753 15753
==
Hits12210 12210
Misses 35433543
```
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=footer).
Last update
[c37fc0b...bc5fa06](https://codecov.io/gh/apache/incubator-airflow/pull/3676?src=pr=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Brush up the CI script for minikube
> ---
>
> Key: AIRFLOW-2829
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2829
> Project: Apache Airflow
> Issue Type: Bug
> Components: ci
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> Ran {{scripts/ci/kubernetes/minikube/start_minikube.sh}} locally and found
> some points that can be improved:
> - minikube version is hard-coded
> - Defined but unused variables: {{$_HELM_VERSION}}, {{$_VM_DRIVER}}
> - Undefined variables: {{$unameOut}}
> - The following lines cause warnings if download is skipped:
> {code}
> 69 sudo mv bin/minikube /usr/local/bin/minikube
> 70 sudo mv bin/kubectl /usr/local/bin/kubectl
> {code}
> - {{return}} s at line 81 and 96 won't work since it's outside of a function
> - To run this script as a non-root user, {{-E}} is required for {{sudo}}. See
> https://github.com/kubernetes/minikube/issues/1883.
> {code}
> 105 _MINIKUBE="sudo PATH=$PATH minikube"
> 106
> 107 $_MINIKUBE config set bootstrapper localkube
> 108 $_MINIKUBE start --kubernetes-version=${_KUBERNETES_VERSION}
> --vm-driver=none
> 109 $_MINIKUBE update-context
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2829) Brush up the CI script for minikube
[
https://issues.apache.org/jira/browse/AIRFLOW-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565289#comment-16565289
]
ASF GitHub Bot commented on AIRFLOW-2829:
-
sekikn opened a new pull request #3676: [AIRFLOW-2829] Brush up the CI script
for minikube
URL: https://github.com/apache/incubator-airflow/pull/3676
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2829
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Fix scripts/ci/kubernetes/minikube/start_minikube.sh
as follows:
- Make minikube version configurable via
environment variable
- Remove unused variables for readability
- Reorder some lines to remove warnings
- Replace ineffective `return` with `exit`
- Add -E to `sudo minikube` so that non-root
users can use this script locally
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
No additional test since it's a fix on the setup script for CI.
Instead, I ran the revised script locally on Ubuntu 16.04 LTS and confirmed
that it worked for both root and non-root user, with the following
environmental variables:
```
export KUBERNETES_VERSION=v1.10.0
export MINIKUBE_VERSION=v0.28.2
```
In addition, I confirmed it worked fine on Travis:
https://travis-ci.org/sekikn/incubator-airflow/builds/410712117
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Brush up the CI script for minikube
> ---
>
> Key: AIRFLOW-2829
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2829
> Project: Apache Airflow
> Issue Type: Bug
> Components: ci
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> Ran {{scripts/ci/kubernetes/minikube/start_minikube.sh}} locally and found
> some points that can be improved:
> - minikube version is hard-coded
> - Defined but unused variables: {{$_HELM_VERSION}}, {{$_VM_DRIVER}}
> - Undefined variables: {{$unameOut}}
> - The following lines cause warnings if download is skipped:
> {code}
> 69 sudo mv bin/minikube /usr/local/bin/minikube
> 70 sudo mv bin/kubectl /usr/local/bin/kubectl
> {code}
> - {{return}} s at line 81 and 96 won't work since it's outside of a function
> - To run this script as a non-root user, {{-E}} is required for {{sudo}}. See
> https://github.com/kubernetes/minikube/issues/1883.
> {code}
> 105 _MINIKUBE="sudo PATH=$PATH minikube"
> 106
> 107 $_MINIKUBE config set bootstrapper localkube
> 108 $_MINIKUBE start --kubernetes-version=${_KUBERNETES_VERSION}
> --vm-driver=none
> 109 $_MINIKUBE update-context
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2836) Minor improvement of contrib.sensors.FileSensor
[
https://issues.apache.org/jira/browse/AIRFLOW-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565171#comment-16565171
]
ASF GitHub Bot commented on AIRFLOW-2836:
-
XD-DENG commented on issue #3674: [AIRFLOW-2836] Minor improvement of
contrib.sensors.FileSensor
URL:
https://github.com/apache/incubator-airflow/pull/3674#issuecomment-409545344
Thanks @ashb . Green now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Minor improvement of contrib.sensors.FileSensor
> ---
>
> Key: AIRFLOW-2836
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2836
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
>
> h4. *Background*
> The default *fs_conn_id* in contrib.sensors.FileSensor is '_*fs_default2*_'.
> However, when we initiate the database
> (https://github.com/apache/incubator-airflow/blob/master/airflow/utils/db.py#L88),
> there isn't such an entry. It doesn't exist anywhere else.
> h4. *Issue*
> The purpose of _contrib.sensors.FileSensor_ is mainly for checking local file
> system (of course can also be used for NAS). Then the path ("/") from default
> connection 'fs_default' would suffice.
> However, given the default value for *fs_conn_id* in
> contrib.sensors.FileSensor is "fs_default2" (a value doesn't exist), it will
> make the situation much more complex.
> When users intend to check local file system only, they should be able to
> leave *fs_conn_id* default directly, instead of going setting up another
> connection separately.
> h4. Proposal
> Change default value for *fs_conn_id* in contrib.sensors.FileSensor from
> "fs_default2" to "fs_default" (actually in the related test, the *fs_conn_id*
> are all specified to be "fs_default").
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2836) Minor improvement of contrib.sensors.FileSensor
[
https://issues.apache.org/jira/browse/AIRFLOW-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565170#comment-16565170
]
ASF GitHub Bot commented on AIRFLOW-2836:
-
codecov-io commented on issue #3674: [AIRFLOW-2836] Minor improvement of
contrib.sensors.FileSensor
URL:
https://github.com/apache/incubator-airflow/pull/3674#issuecomment-409544984
#
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=h1)
Report
> Merging
[#3674](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=desc)
into
[master](https://codecov.io/gh/apache/incubator-airflow/commit/c37fc0b6ba19e3fe5656ae37cef9b59cef3c29e8?src=pr=desc)
will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=tree)
```diff
@@Coverage Diff@@
## master #3674 +/- ##
=
- Coverage77.5% 77.5% -0.01%
=
Files 205 205
Lines 15753 15753
=
- Hits12210 12209 -1
- Misses 35433544 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=tree)
| Coverage Δ | |
|---|---|---|
|
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3674/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
| `88.54% <0%> (-0.05%)` | :arrow_down: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=footer).
Last update
[c37fc0b...4d8abd8](https://codecov.io/gh/apache/incubator-airflow/pull/3674?src=pr=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Minor improvement of contrib.sensors.FileSensor
> ---
>
> Key: AIRFLOW-2836
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2836
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
>
> h4. *Background*
> The default *fs_conn_id* in contrib.sensors.FileSensor is '_*fs_default2*_'.
> However, when we initiate the database
> (https://github.com/apache/incubator-airflow/blob/master/airflow/utils/db.py#L88),
> there isn't such an entry. It doesn't exist anywhere else.
> h4. *Issue*
> The purpose of _contrib.sensors.FileSensor_ is mainly for checking local file
> system (of course can also be used for NAS). Then the path ("/") from default
> connection 'fs_default' would suffice.
> However, given the default value for *fs_conn_id* in
> contrib.sensors.FileSensor is "fs_default2" (a value doesn't exist), it will
> make the situation much more complex.
> When users intend to check local file system only, they should be able to
> leave *fs_conn_id* default directly, instead of going setting up another
> connection separately.
> h4. Proposal
> Change default value for *fs_conn_id* in contrib.sensors.FileSensor from
> "fs_default2" to "fs_default" (actually in the related test, the *fs_conn_id*
> are all specified to be "fs_default").
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-2834 started by Rurui Ye. - > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Assignee: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[
https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565151#comment-16565151
]
ASF GitHub Bot commented on AIRFLOW-2834:
-
yeluolei opened a new pull request #3675: [AIRFLOW-2834] fix build script for
k8s docker
URL: https://github.com/apache/incubator-airflow/pull/3675
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2834
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
the kubernetes docker build airflow without rbac support, but the configmap
need rbac. so need to change the build script to build js and css files.
currently when open airflow web ui deployed in kubernetes, the webpage is
blank and will be some file missing.
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [ ] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> can not see the dag page after build from the newest code in github
> ---
>
> Key: AIRFLOW-2834
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2834
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: Airflow 2.0
>Reporter: Rurui Ye
>Assignee: Rurui Ye
>Priority: Blocker
> Attachments: image-2018-08-01-14-20-09-256.png
>
>
> after build and deploy the newest version of code from github. got the web
> server opened and the dags page blank with the following error in request
> resource.
>
> !image-2018-08-01-14-20-09-256.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565070#comment-16565070 ] ASF GitHub Bot commented on AIRFLOW-2817: - ashb commented on issue #3660: [AIRFLOW-2817] Force explicit choice on GPL dependency URL: https://github.com/apache/incubator-airflow/pull/3660#issuecomment-409521040 Something about the logic isn't right - everything on Travis is failing on the env check. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rurui Ye reopened AIRFLOW-2834: --- Assignee: Rurui Ye > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Assignee: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565085#comment-16565085 ] Rurui Ye commented on AIRFLOW-2834: --- yes, I have find the problem and need update the kubernetes build script for this. > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565082#comment-16565082 ] ASF GitHub Bot commented on AIRFLOW-2817: - verdan commented on issue #3660: [AIRFLOW-2817] Force explicit choice on GPL dependency URL: https://github.com/apache/incubator-airflow/pull/3660#issuecomment-409522600 @ashb I believe we can remove the python-nvd3 entirely and use the custom javascript to render the charts using d3 and nvd3 JS libraries, just the way we are using Graph View on DAG detail page i.e., sending all the data from python and implement charts on the frontend in templates. But as you said, it will take some time to implement on the frontend, and won't be ready for the release 1.10. P.S: Yes, it is possible to make a part of the application/page use the React. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rurui Ye closed AIRFLOW-2834. - Resolution: Not A Bug > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2770) kubernetes: add support for dag folder in the docker image
[ https://issues.apache.org/jira/browse/AIRFLOW-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rurui Ye reassigned AIRFLOW-2770: - Assignee: Rurui Ye > kubernetes: add support for dag folder in the docker image > -- > > Key: AIRFLOW-2770 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2770 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Rurui Ye >Assignee: Rurui Ye >Priority: Critical > > currently the kube executor need to provider dag_volume_chain or git repo in > the config file, but if the user has build dag into their docker image, they > doesn't need to provider these two options, and they can manager their dag > version by manager the docker image version. > So I suppose we can add the a new configuration as > kube.config.dag_folder_path along with dag_volume_chain and git repo. with > this config, we can run the worker just from the dags in docker image. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2655) Default Kubernetes worker configurations are inconsistent
[ https://issues.apache.org/jira/browse/AIRFLOW-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565049#comment-16565049 ] ASF GitHub Bot commented on AIRFLOW-2655: - johnchenghk01 commented on issue #3529: [AIRFLOW-2655] Fix inconsistency of default config of kubernetes worker URL: https://github.com/apache/incubator-airflow/pull/3529#issuecomment-409515471 It will expose the DB password when doing a kubectl describe. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Default Kubernetes worker configurations are inconsistent > - > > Key: AIRFLOW-2655 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2655 > Project: Apache Airflow > Issue Type: Bug > Components: executor >Affects Versions: 1.10.0 >Reporter: Shintaro Murakami >Priority: Minor > Fix For: 2.0.0 > > > if optional config `airflow_configmap` is not set, the worker configured with > `LocalExecutor` and sql_alchemy_conn starts with `sqlite`. > This combination is not allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565046#comment-16565046 ] ASF GitHub Bot commented on AIRFLOW-2817: - ashb edited a comment on issue #3660: [AIRFLOW-2817] Force explicit choice on GPL dependency URL: https://github.com/apache/incubator-airflow/pull/3660#issuecomment-409513668 If not I think vendoring python-nvd3 and slugify to use the non-GPL is probably the way to go. (Or perhaps replacing python-nvd3 entirely. That's a bigger job though. https://medium.com/@Elijah_Meeks/introducing-semiotic-for-data-visualization-88dc3c6b6926 looks interesting ,but uses React (which is fine from a licensing PoV now.) Edit: If we did use this I wouldn't suggest React-ifying the whole app, just the chart part of the page itself. If that's possible. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565043#comment-16565043 ] ASF GitHub Bot commented on AIRFLOW-2817: - ashb commented on issue #3660: [AIRFLOW-2817] Force explicit choice on GPL dependency URL: https://github.com/apache/incubator-airflow/pull/3660#issuecomment-409513668 If not I think vendoring python-nvd3 and slugify to use the non-GPL is probably the way to go. (Or perhaps replacing python-nvd3 entirely. That's a bigger job though. https://medium.com/@Elijah_Meeks/introducing-semiotic-for-data-visualization-88dc3c6b6926 looks interesting ,but uses React (which is fine from a licensing PoV now.) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2716) Replace new Python 3.7 keywords
[ https://issues.apache.org/jira/browse/AIRFLOW-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565044#comment-16565044 ] ASF subversion and git services commented on AIRFLOW-2716: -- Commit 0423060bb20c54035fafecda8cc0a50e65953f27 in incubator-airflow's branch refs/heads/v1-10-test from [~jacobhayes] [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=0423060 ] [AIRFLOW-2716] Replace async and await py3.7 keywords Closes #3578 from JacobHayes/py37-keywords (cherry picked from commit fcd51f362c6804cb69dee8e2dd054304a0564dde) Signed-off-by: Bolke de Bruin > Replace new Python 3.7 keywords > --- > > Key: AIRFLOW-2716 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2716 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Jacob Hayes >Assignee: Jacob Hayes >Priority: Major > Fix For: 2.0.0 > > > Python 3.7 added `async` and `await` as reserved keywords, so they need to be > replaced with alternative names. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2716) Replace new Python 3.7 keywords
[ https://issues.apache.org/jira/browse/AIRFLOW-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565042#comment-16565042 ] ASF subversion and git services commented on AIRFLOW-2716: -- Commit c720c352f0762a483fa1202e1477e740631a8184 in incubator-airflow's branch refs/heads/v1-10-stable from [~jacobhayes] [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=c720c35 ] [AIRFLOW-2716] Replace async and await py3.7 keywords Closes #3578 from JacobHayes/py37-keywords (cherry picked from commit fcd51f362c6804cb69dee8e2dd054304a0564dde) Signed-off-by: Bolke de Bruin (cherry picked from commit 0423060bb20c54035fafecda8cc0a50e65953f27) Signed-off-by: Bolke de Bruin > Replace new Python 3.7 keywords > --- > > Key: AIRFLOW-2716 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2716 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Jacob Hayes >Assignee: Jacob Hayes >Priority: Major > Fix For: 2.0.0 > > > Python 3.7 added `async` and `await` as reserved keywords, so they need to be > replaced with alternative names. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565037#comment-16565037 ] ASF subversion and git services commented on AIRFLOW-2817: -- Commit c37fc0b6ba19e3fe5656ae37cef9b59cef3c29e8 in incubator-airflow's branch refs/heads/master from bolkedebruin [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=c37fc0b ] [AIRFLOW-2817] Force explicit choice on GPL dependency (#3660) By default one of Apache Airflow's dependencies pulls in a GPL library. Airflow should not install (and upgrade) without an explicit choice. This is part of the Apache requirements as we cannot depend on Category X software. > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[
https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565036#comment-16565036
]
ASF GitHub Bot commented on AIRFLOW-2817:
-
bolkedebruin closed pull request #3660: [AIRFLOW-2817] Force explicit choice on
GPL dependency
URL: https://github.com/apache/incubator-airflow/pull/3660
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 81e43fb4b8..e078d7c9ae 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -47,6 +47,7 @@ python:
- "3.5"
env:
global:
+- SLUGIFY_USES_TEXT_UNIDECODE=yes
- TRAVIS_CACHE=$HOME/.travis_cache/
- KRB5_CONFIG=/etc/krb5.conf
- KRB5_KTNAME=/etc/airflow.keytab
diff --git a/INSTALL b/INSTALL
index 5c8f03eb66..596ce25814 100644
--- a/INSTALL
+++ b/INSTALL
@@ -1,13 +1,30 @@
-# INSTALL / BUILD instruction for Apache Airflow (incubating)
-# fetch the tarball and untar the source
+# INSTALL / BUILD instructions for Apache Airflow (incubating)
+
+# [required] fetch the tarball and untar the source
+# change into the directory that was untarred.
# [optional] run Apache RAT (release audit tool) to validate license headers
-# RAT docs here: https://creadur.apache.org/rat/
+# RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat
java -jar apache-rat.jar -E ./.rat-excludes -d .
-# [optional] by default one of Apache Airflow's dependencies pulls in a GPL
-# library. If this is a concern issue (also every upgrade):
-# export SLUGIFY_USES_TEXT_UNIDECODE=yes
+# [optional] Airflow pulls in quite a lot of dependencies in order
+# to connect to other services. You might want to test or run Airflow
+# from a virtual env to make sure those dependencies are separated
+# from your system wide versions
+python -m my_env
+source my_env/bin/activate
+
+# [required] by default one of Apache Airflow's dependencies pulls in a GPL
+# library. Airflow will not install (and upgrade) without an explicit choice.
+#
+# To make sure not to install the GPL dependency:
+# export SLUGIFY_USES_TEXT_UNIDECODE=yes
+# In case you do not mind:
+# export GPL_UNIDECODE=yes
+
+# [required] building and installing
+# by pip (preferred)
+pip install .
-# install the release
+# or directly
python setup.py install
diff --git a/UPDATING.md b/UPDATING.md
index da80f56fcb..ef29e1d3a4 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -5,6 +5,12 @@ assists users migrating to a new version.
## Airflow Master
+## Airflow 1.10
+
+Installation and upgrading requires setting `SLUGIFY_USES_TEXT_UNIDECODE=yes`
in your environment or
+`AIRFLOW_GPL_UNIDECODE=yes`. In case of the latter a GPL runtime dependency
will be installed due to a
+dependency (python-nvd3 -> python-slugify -> unidecode).
+
### Replace DataProcHook.await calls to DataProcHook.wait
The method name was changed to be compatible with the Python 3.7 async/await
keywords
diff --git a/scripts/ci/kubernetes/docker/Dockerfile
b/scripts/ci/kubernetes/docker/Dockerfile
index 498c47b21a..93b20dbcd2 100644
--- a/scripts/ci/kubernetes/docker/Dockerfile
+++ b/scripts/ci/kubernetes/docker/Dockerfile
@@ -17,6 +17,8 @@
FROM ubuntu:16.04
+ENV SLUGIFY_USES_TEXT_UNIDECODE=yes
+
# install deps
RUN apt-get update -y && apt-get install -y \
wget \
@@ -33,7 +35,6 @@ RUN apt-get update -y && apt-get install -y \
unzip \
&& apt-get clean
-
RUN pip install --upgrade pip
# Since we install vanilla Airflow, we also want to have support for Postgres
and Kubernetes
diff --git a/setup.py b/setup.py
index 50af30944e..e69572c51d 100644
--- a/setup.py
+++ b/setup.py
@@ -35,6 +35,17 @@
PY3 = sys.version_info[0] == 3
+# See LEGAL-362
+def verify_gpl_dependency():
+if (not os.getenv("AIRFLOW_GPL_UNIDECODE")
+and not os.getenv("SLUGIFY_USES_TEXT_UNIDECODE") == "yes"):
+raise RuntimeError("By default one of Airflow's dependencies installs
a GPL "
+ "dependency (unidecode). To avoid this dependency
set "
+ "SLUGIFY_USES_TEXT_UNIDECODE=yes in your
environment when you "
+ "install or upgrade Airflow. To force installing
the GPL "
+ "version set AIRFLOW_GPL_UNIDECODE")
+
+
class Tox(TestCommand):
user_options = [('tox-args=', None, "Arguments to pass to tox")]
@@ -258,6 +269,7 @@ def write_version(filename=os.path.join(*['airflow',
def do_setup():
+verify_gpl_dependency()
write_version()
setup(
name='apache-airflow',
@@ -376,6 +388,7 @@ def do_setup():
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python ::
[jira] [Commented] (AIRFLOW-2817) Force explicit choice on GPL dependency
[ https://issues.apache.org/jira/browse/AIRFLOW-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565038#comment-16565038 ] ASF GitHub Bot commented on AIRFLOW-2817: - bolkedebruin commented on issue #3660: [AIRFLOW-2817] Force explicit choice on GPL dependency URL: https://github.com/apache/incubator-airflow/pull/3660#issuecomment-409512201 Will see if we can address the issue with upstream This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Force explicit choice on GPL dependency > --- > > Key: AIRFLOW-2817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2817 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Major > > A more explicit choice on GPL dependency was required by the IPMC -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2836) Minor improvement of contrib.sensors.FileSensor
[
https://issues.apache.org/jira/browse/AIRFLOW-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565025#comment-16565025
]
ASF GitHub Bot commented on AIRFLOW-2836:
-
XD-DENG opened a new pull request #3674: [AIRFLOW-2836] Minor improvement of
contrib.sensors.FileSensor
URL: https://github.com/apache/incubator-airflow/pull/3674
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2836
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Background
The default `fs_conn_id` in `contrib.sensors.FileSensor` is 'fs_default2'.
However, when we initiate the database
(https://github.com/apache/incubator-airflow/blob/master/airflow/utils/db.py#L88),
there isn't such an entry. It doesn't exist anywhere else.
Issue
The purpose of `contrib.sensors.FileSensor` is mainly for checking local
file system (of course can also be used for NAS). Then the path ("/") from
default connection 'fs_default' would suffice.
However, given the default value for fs_conn_id in
contrib.sensors.FileSensor is "fs_default2" (a value doesn't exist), it will
make the situation much more complex.
When users intend to check local file system only, they should be able to
leave fs_conn_id default directly, instead of going setting up another
connection separately.
Proposal
Change default value for `fs_conn_id` in `contrib.sensors.FileSensor` from
"fs_default2" to "fs_default" (actually in the related test, the `fs_conn_id`
are all specified to be "fs_default").
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [ ] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Minor improvement of contrib.sensors.FileSensor
> ---
>
> Key: AIRFLOW-2836
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2836
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
>
> h4. *Background*
> The default *fs_conn_id* in contrib.sensors.FileSensor is '_*fs_default2*_'.
> However, when we initiate the database
> (https://github.com/apache/incubator-airflow/blob/master/airflow/utils/db.py#L88),
> there isn't such an entry. It doesn't exist anywhere else.
> h4. *Issue*
> The purpose of _contrib.sensors.FileSensor_ is mainly for checking local file
> system (of course can also be used for NAS). Then the path ("/") from default
> connection 'fs_default' would suffice.
> However, given the default value for *fs_conn_id* in
> contrib.sensors.FileSensor is "fs_default2" (a value doesn't exist), it will
> make the situation much more complex.
> When users intend to check local file system only, they should be able to
> leave *fs_conn_id* default directly, instead of going setting up another
> connection separately.
> h4. Proposal
> Change default value for *fs_conn_id* in contrib.sensors.FileSensor from
> "fs_default2" to "fs_default" (actually in the related test, the *fs_conn_id*
> are all specified to be "fs_default").
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-2836) Minor improvement of contrib.sensors.FileSensor
Xiaodong DENG created AIRFLOW-2836:
--
Summary: Minor improvement of contrib.sensors.FileSensor
Key: AIRFLOW-2836
URL: https://issues.apache.org/jira/browse/AIRFLOW-2836
Project: Apache Airflow
Issue Type: Improvement
Components: contrib
Reporter: Xiaodong DENG
Assignee: Xiaodong DENG
h4. *Background*
The default *fs_conn_id* in contrib.sensors.FileSensor is '_*fs_default2*_'.
However, when we initiate the database
(https://github.com/apache/incubator-airflow/blob/master/airflow/utils/db.py#L88),
there isn't such an entry. It doesn't exist anywhere else.
h4. *Issue*
The purpose of _contrib.sensors.FileSensor_ is mainly for checking local file
system (of course can also be used for NAS). Then the path ("/") from default
connection 'fs_default' would suffice.
However, given the default value for *fs_conn_id* in contrib.sensors.FileSensor
is "fs_default2" (a value doesn't exist), it will make the situation much more
complex.
When users intend to check local file system only, they should be able to leave
*fs_conn_id* default directly, instead of going setting up another connection
separately.
h4. Proposal
Change default value for *fs_conn_id* in contrib.sensors.FileSensor from
"fs_default2" to "fs_default" (actually in the related test, the *fs_conn_id*
are all specified to be "fs_default").
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565015#comment-16565015 ] Ash Berlin-Taylor commented on AIRFLOW-2834: Are you using the RBAC UI (I forget if that is default now or not?) If so you will need to follow this https://github.com/apache/incubator-airflow/blob/master/CONTRIBUTING.md#setting-up-the-node--npm-javascript-environment-only-for-www_rbac when checking out form git. The static assets are now compiled, and you will need to follow that process at least once. > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2756) Marking DAG run does not set start_time and end_time correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-2756. - Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/incubator-airflow/pull/3606 > Marking DAG run does not set start_time and end_time correctly > -- > > Key: AIRFLOW-2756 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2756 > Project: Apache Airflow > Issue Type: Bug >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > Fix For: 2.0.0 > > > Marking DAG run right now always set end_time while it should set start_time > when marking RUNNING and otherwise end_time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2756) Marking DAG run does not set start_time and end_time correctly
[
https://issues.apache.org/jira/browse/AIRFLOW-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564958#comment-16564958
]
ASF GitHub Bot commented on AIRFLOW-2756:
-
kaxil closed pull request #3606: [AIRFLOW-2756] Fix bug in set DAG run state
workflow
URL: https://github.com/apache/incubator-airflow/pull/3606
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/api/common/experimental/mark_tasks.py
b/airflow/api/common/experimental/mark_tasks.py
index 681864dfbe..88c5275f5a 100644
--- a/airflow/api/common/experimental/mark_tasks.py
+++ b/airflow/api/common/experimental/mark_tasks.py
@@ -206,7 +206,10 @@ def _set_dag_run_state(dag_id, execution_date, state,
session=None):
DR.execution_date == execution_date
).one()
dr.state = state
-dr.end_date = timezone.utcnow()
+if state == State.RUNNING:
+dr.start_date = timezone.utcnow()
+else:
+dr.end_date = timezone.utcnow()
session.commit()
diff --git a/airflow/jobs.py b/airflow/jobs.py
index 00ede5451d..70891ab4c3 100644
--- a/airflow/jobs.py
+++ b/airflow/jobs.py
@@ -1023,8 +1023,7 @@ def _change_state_for_tis_without_dagrun(self,
models.TaskInstance.dag_id == subq.c.dag_id,
models.TaskInstance.task_id == subq.c.task_id,
models.TaskInstance.execution_date ==
-subq.c.execution_date,
-models.TaskInstance.task_id == subq.c.task_id)) \
+subq.c.execution_date)) \
.update({models.TaskInstance.state: new_state},
synchronize_session=False)
session.commit()
diff --git a/airflow/www/views.py b/airflow/www/views.py
index d37c0db45d..1ee5a2df86 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -2741,7 +2741,8 @@ def after_model_change(self, form, dagrun, is_created,
session=None):
altered_tis = set_dag_run_state_to_success(
dagbag.get_dag(dagrun.dag_id),
dagrun.execution_date,
-commit=True)
+commit=True,
+session=session)
elif dagrun.state == State.FAILED:
altered_tis = set_dag_run_state_to_failed(
dagbag.get_dag(dagrun.dag_id),
diff --git a/tests/api/common/experimental/mark_tasks.py
b/tests/api/common/experimental/mark_tasks.py
index 181d10d8a1..9bba91bee0 100644
--- a/tests/api/common/experimental/mark_tasks.py
+++ b/tests/api/common/experimental/mark_tasks.py
@@ -267,11 +267,25 @@ def _create_test_dag_run(self, state, date):
def _verify_dag_run_state(self, dag, date, state):
drs = models.DagRun.find(dag_id=dag.dag_id, execution_date=date)
dr = drs[0]
+
self.assertEqual(dr.get_state(), state)
+def _verify_dag_run_dates(self, dag, date, state, middle_time):
+# When target state is RUNNING, we should set start_date,
+# otherwise we should set end_date.
+drs = models.DagRun.find(dag_id=dag.dag_id, execution_date=date)
+dr = drs[0]
+if state == State.RUNNING:
+self.assertGreater(dr.start_date, middle_time)
+self.assertIsNone(dr.end_date)
+else:
+self.assertLess(dr.start_date, middle_time)
+self.assertGreater(dr.end_date, middle_time)
+
def test_set_running_dag_run_to_success(self):
date = self.execution_dates[0]
dr = self._create_test_dag_run(State.RUNNING, date)
+middle_time = timezone.utcnow()
self._set_default_task_instance_states(dr)
altered = set_dag_run_state_to_success(self.dag1, date, commit=True)
@@ -280,10 +294,12 @@ def test_set_running_dag_run_to_success(self):
self.assertEqual(len(altered), 5)
self._verify_dag_run_state(self.dag1, date, State.SUCCESS)
self._verify_task_instance_states(self.dag1, date, State.SUCCESS)
+self._verify_dag_run_dates(self.dag1, date, State.SUCCESS, middle_time)
def test_set_running_dag_run_to_failed(self):
date = self.execution_dates[0]
dr = self._create_test_dag_run(State.RUNNING, date)
+middle_time = timezone.utcnow()
self._set_default_task_instance_states(dr)
altered = set_dag_run_state_to_failed(self.dag1, date, commit=True)
@@ -292,10 +308,12 @@ def test_set_running_dag_run_to_failed(self):
self.assertEqual(len(altered), 1)
self._verify_dag_run_state(self.dag1, date, State.FAILED)
self.assertEqual(dr.get_task_instance('run_after_loop').state,
State.FAILED)
+self._verify_dag_run_dates(self.dag1, date, State.FAILED, middle_time)
[jira] [Commented] (AIRFLOW-2756) Marking DAG run does not set start_time and end_time correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564959#comment-16564959 ] ASF subversion and git services commented on AIRFLOW-2756: -- Commit c26c56487649f8d912c20ebf7aec1677e9996ea4 in incubator-airflow's branch refs/heads/master from yrqls21 [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=c26c564 ] [AIRFLOW-2756] Fix bug in set DAG run state workflow (#3606) > Marking DAG run does not set start_time and end_time correctly > -- > > Key: AIRFLOW-2756 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2756 > Project: Apache Airflow > Issue Type: Bug >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > > Marking DAG run right now always set end_time while it should set start_time > when marking RUNNING and otherwise end_time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2835) Remove python-selinux
[ https://issues.apache.org/jira/browse/AIRFLOW-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564947#comment-16564947 ] ASF GitHub Bot commented on AIRFLOW-2835: - bolkedebruin closed pull request #3673: [AIRFLOW-2835] Remove python-selinux URL: https://github.com/apache/incubator-airflow/pull/3673 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.travis.yml b/.travis.yml index 81e43fb4b8..4e490c74e1 100644 --- a/.travis.yml +++ b/.travis.yml @@ -40,7 +40,6 @@ addons: - krb5-kdc - krb5-admin-server - oracle-java8-installer - - python-selinux postgresql: "9.2" python: - "2.7" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove python-selinux > - > > Key: AIRFLOW-2835 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2835 > Project: Apache Airflow > Issue Type: Bug >Reporter: Fokko Driesprong >Priority: Major > > This package sometimes crashes the CI and is not required. Therefore it does > not make sense to install it since it will take ci-time and make things > brittle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2803) Fix all ESLint issues
[ https://issues.apache.org/jira/browse/AIRFLOW-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564941#comment-16564941 ] ASF GitHub Bot commented on AIRFLOW-2803: - verdan commented on a change in pull request #3656: [WIP][AIRFLOW-2803] Fix all ESLint issues URL: https://github.com/apache/incubator-airflow/pull/3656#discussion_r206791865 ## File path: airflow/www_rbac/templates/airflow/circles.html ## @@ -28,117 +28,111 @@ Airflow 404 = lots of circles
[jira] [Commented] (AIRFLOW-2803) Fix all ESLint issues
[ https://issues.apache.org/jira/browse/AIRFLOW-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564938#comment-16564938 ] ASF GitHub Bot commented on AIRFLOW-2803: - verdan commented on issue #3656: [WIP][AIRFLOW-2803] Fix all ESLint issues URL: https://github.com/apache/incubator-airflow/pull/3656#issuecomment-409491349 @tedmiston please tag me once it is ready for the next review. I see you're still working on this PR. i.e., Jinja template tags, indentation and some commented out code. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix all ESLint issues > - > > Key: AIRFLOW-2803 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2803 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Verdan Mahmood >Assignee: Taylor Edmiston >Priority: Major > > Most of the JS code in Apache Airflow has linting issues which are > highlighted after the integration of ESLint. > Once AIRFLOW-2783 merged in master branch, please fix all the javascript > styling issues that we have in .js and .html files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2835) Remove python-selinux
[ https://issues.apache.org/jira/browse/AIRFLOW-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564919#comment-16564919 ] ASF GitHub Bot commented on AIRFLOW-2835: - codecov-io commented on issue #3673: [AIRFLOW-2835] Remove python-selinux URL: https://github.com/apache/incubator-airflow/pull/3673#issuecomment-409485914 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=h1) Report > Merging [#3673](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ed972042a864cd010137190e0bbb1d25a9dcfe83?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3673 +/- ## === Coverage 77.51% 77.51% === Files 205 205 Lines 1575115751 === Hits1221012210 Misses 3541 3541 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=footer). Last update [ed97204...ed2a781](https://codecov.io/gh/apache/incubator-airflow/pull/3673?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove python-selinux > - > > Key: AIRFLOW-2835 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2835 > Project: Apache Airflow > Issue Type: Bug >Reporter: Fokko Driesprong >Priority: Major > > This package sometimes crashes the CI and is not required. Therefore it does > not make sense to install it since it will take ci-time and make things > brittle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2524) Airflow integration with AWS Sagemaker
[ https://issues.apache.org/jira/browse/AIRFLOW-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564918#comment-16564918 ] ASF GitHub Bot commented on AIRFLOW-2524: - Fokko commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r206786344 ## File path: airflow/contrib/operators/sagemaker_create_training_job_operator.py ## @@ -0,0 +1,98 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow.contrib.hooks.sagemaker_hook import SageMakerHook +from airflow.models import BaseOperator +from airflow.utils import apply_defaults +from airflow.exceptions import AirflowException + + +class SageMakerCreateTrainingJobOperator(BaseOperator): + +""" + Initiate a SageMaker training + + This operator returns The ARN of the model created in Amazon SageMaker + + :param training_job_config: + The configuration necessary to start a training job (templated) + :type training_job_config: dict + :param region_name: The AWS region_name + :type region_name: string + :param sagemaker_conn_id: The SageMaker connection ID to use. + :type aws_conn_id: string Review comment: Hi Keliang, thanks for explaining the Sagemaker process. I think it is very similar to for example the Druid hook that we have: https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/druid_hook.py#L93 This hook will kick of a job using a HTTP POST of a json document to the druid cluster, and make sure that it receives a http 200. And then it will continue to poll the job by invoking the API periodically. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Airflow integration with AWS Sagemaker > -- > > Key: AIRFLOW-2524 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2524 > Project: Apache Airflow > Issue Type: Improvement > Components: aws, contrib >Reporter: Rajeev Srinivasan >Assignee: Yang Yu >Priority: Major > Labels: AWS > > Would it be possible to orchestrate an end to end AWS Sagemaker job using > Airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2832) Inconsistencies and linter errors across markdown files
[ https://issues.apache.org/jira/browse/AIRFLOW-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564911#comment-16564911 ] ASF subversion and git services commented on AIRFLOW-2832: -- Commit 6d31c9e4160ce3907e23e53023f30678485be78c in incubator-airflow's branch refs/heads/master from Taylor D. Edmiston [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=6d31c9e ] [AIRFLOW-2832] Lint and resolve inconsistencies in Markdown files (#3670) Clean up the Markdown files and make the formatting consistent > Inconsistencies and linter errors across markdown files > --- > > Key: AIRFLOW-2832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2832 > Project: Apache Airflow > Issue Type: Improvement > Components: docs, Documentation >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > > There are a number of inconsistencies within and across markdown files in the > Airflow project. Most of these are simple formatting issues easily fixed by > linting (e.g., with mdl). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2832) Inconsistencies and linter errors across markdown files
[
https://issues.apache.org/jira/browse/AIRFLOW-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564908#comment-16564908
]
ASF GitHub Bot commented on AIRFLOW-2832:
-
Fokko closed pull request #3670: [AIRFLOW-2832] Lint and resolve
inconsistencies in Markdown files
URL: https://github.com/apache/incubator-airflow/pull/3670
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
index 6000d0e5ff..90452d954b 100644
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,33 +1,34 @@
Make sure you have checked _all_ steps below.
-### JIRA
-- [ ] My PR addresses the following [Airflow
JIRA](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
-- https://issues.apache.org/jira/browse/AIRFLOW-XXX
-- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a JIRA issue.
+### Jira
+- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
+ - https://issues.apache.org/jira/browse/AIRFLOW-XXX
+ - In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
-- [ ] Here are some details about my PR, including screenshots of any UI
changes:
+- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
-- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
+- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
-- [ ] My commits all reference JIRA issues in their subject lines, and I have
squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
-1. Subject is separated from body by a blank line
-2. Subject is limited to 50 characters
-3. Subject does not end with a period
-4. Subject uses the imperative mood ("add", not "adding")
-5. Body wraps at 72 characters
-6. Body explains "what" and "why", not "how"
+- [ ] My commits all reference Jira issues in their subject lines, and I have
squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
+ 1. Subject is separated from body by a blank line
+ 1. Subject is limited to 50 characters (not including Jira issue reference)
+ 1. Subject does not end with a period
+ 1. Subject uses the imperative mood ("add", not "adding")
+ 1. Body wraps at 72 characters
+ 1. Body explains "what" and "why", not "how"
### Documentation
-- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
-- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
+- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
+ - When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
+
- [ ] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 47a1a80549..2cf8e0218e 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -3,22 +3,21 @@
Contributions are welcome and are greatly appreciated! Every
little bit helps, and credit will always be given.
-
-# Table of Contents
- * [TOC](#table-of-contents)
- * [Types of Contributions](#types-of-contributions)
- - [Report Bugs](#report-bugs)
- - [Fix Bugs](#fix-bugs)
- - [Implement Features](#implement-features)
- - [Improve Documentation](#improve-documentation)
- - [Submit Feedback](#submit-feedback)
- * [Documentation](#documentation)
- * [Development and Testing](#development-and-testing)
- - [Setting up a development
environment](#setting-up-a-development-environment)
- - [Pull requests guidelines](#pull-request-guidelines)
- - [Testing Locally](#testing-locally)
- * [Changing the Metadata Database](#changing-the-metadata-database)
-
+## Table of Contents
+
+- [TOC](#table-of-contents)
+- [Types of Contributions](#types-of-contributions)
+ - [Report Bugs](#report-bugs)
+ - [Fix Bugs](#fix-bugs)
+ - [Implement
[jira] [Commented] (AIRFLOW-2832) Inconsistencies and linter errors across markdown files
[ https://issues.apache.org/jira/browse/AIRFLOW-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564907#comment-16564907 ] ASF GitHub Bot commented on AIRFLOW-2832: - Fokko commented on a change in pull request #3670: [AIRFLOW-2832] Lint and resolve inconsistencies in Markdown files URL: https://github.com/apache/incubator-airflow/pull/3670#discussion_r206783822 ## File path: dev/README.md ## @@ -72,25 +76,33 @@ origin https://github.com//airflow (push) ``` JIRA + Users should set environment variables `JIRA_USERNAME` and `JIRA_PASSWORD` corresponding to their ASF JIRA login. This will allow the tool to automatically close issues. If they are not set, the user will be prompted every time. GitHub OAuth Token + Unauthenticated users can only make 60 requests/hour to the Github API. If you get an error about exceeding the rate, you will need to set a `GITHUB_OAUTH_KEY` environment variable that contains a token value. Users can generate tokens from their GitHub profile. ## Airflow release signing tool + The release signing tool can be used to create the SHA512/MD5 and ASC files that required for Apache releases. ### Execution -To create a release tar ball execute following command from Airflow's root. -`python setup.py compile_assets sdist --formats=gztar` +To create a release tarball execute following command from Airflow's root. -*Note: `compile_assets` command build the frontend assets (JS and CSS) files for the +```bash +python setup.py compile_assets sdist --formats=gztar +``` + +*Note: `compile_assets` command build the frontend assets (JS and CSS) files for the Web UI using webpack and npm. Please make sure you have `npm` installed on your local machine globally. Details on how to install `npm` can be found in CONTRIBUTING.md file.* After that navigate to relative directory i.e., `cd dist` and sign the release files. -`../dev/sign.sh Inconsistencies and linter errors across markdown files > --- > > Key: AIRFLOW-2832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2832 > Project: Apache Airflow > Issue Type: Improvement > Components: docs, Documentation >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > > There are a number of inconsistencies within and across markdown files in the > Airflow project. Most of these are simple formatting issues easily fixed by > linting (e.g., with mdl). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2835) Remove python-selinux
[
https://issues.apache.org/jira/browse/AIRFLOW-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564864#comment-16564864
]
ASF GitHub Bot commented on AIRFLOW-2835:
-
Fokko opened a new pull request #3673: [AIRFLOW-2835] Remove python-selinux
URL: https://github.com/apache/incubator-airflow/pull/3673
This package is not used and it sometimes breaks the CI because it is not
available. Therefore it makes sense to just remove it :-)
Example failed builds on the master branch:
https://travis-ci.org/apache/incubator-airflow/jobs/410483664
https://travis-ci.org/apache/incubator-airflow/jobs/410483665
https://travis-ci.org/apache/incubator-airflow/jobs/410484305
Make sure you have checked _all_ steps below.
### JIRA
- [x] My PR addresses the following [Airflow
JIRA](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-2835\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend
your commit with \[AIRFLOW-2835\], code changes always need a JIRA issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Remove python-selinux
> -
>
> Key: AIRFLOW-2835
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2835
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Fokko Driesprong
>Priority: Major
>
> This package sometimes crashes the CI and is not required. Therefore it does
> not make sense to install it since it will take ci-time and make things
> brittle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-2835) Remove python-selinux
Fokko Driesprong created AIRFLOW-2835: - Summary: Remove python-selinux Key: AIRFLOW-2835 URL: https://issues.apache.org/jira/browse/AIRFLOW-2835 Project: Apache Airflow Issue Type: Bug Reporter: Fokko Driesprong This package sometimes crashes the CI and is not required. Therefore it does not make sense to install it since it will take ci-time and make things brittle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2834) can not see the dag page after build from the newest code in github
[ https://issues.apache.org/jira/browse/AIRFLOW-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rurui Ye updated AIRFLOW-2834: -- Priority: Blocker (was: Major) > can not see the dag page after build from the newest code in github > --- > > Key: AIRFLOW-2834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2834 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: Airflow 2.0 >Reporter: Rurui Ye >Priority: Blocker > Attachments: image-2018-08-01-14-20-09-256.png > > > after build and deploy the newest version of code from github. got the web > server opened and the dags page blank with the following error in request > resource. > > !image-2018-08-01-14-20-09-256.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
