[jira] [Commented] (AIRFLOW-597) objects that do not support truthiness break template processing
[ https://issues.apache.org/jira/browse/AIRFLOW-597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610835#comment-15610835 ] ASF subversion and git services commented on AIRFLOW-597: - Commit 2f261266942b18a0458472c3db8daae554e6b520 in incubator-airflow's branch refs/heads/master from [~gwax] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=2f26126 ] [AIRFLOW-597] Check if content is None, not false-equivalent Closes #1856 from gwax/non_boolean_templates > objects that do not support truthiness break template processing > > > Key: AIRFLOW-597 > URL: https://issues.apache.org/jira/browse/AIRFLOW-597 > Project: Apache Airflow > Issue Type: Bug >Reporter: George Leslie-Waksman >Assignee: George Leslie-Waksman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AIRFLOW-597) objects that do not support truthiness break template processing
[ https://issues.apache.org/jira/browse/AIRFLOW-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand resolved AIRFLOW-597. - Resolution: Fixed > objects that do not support truthiness break template processing > > > Key: AIRFLOW-597 > URL: https://issues.apache.org/jira/browse/AIRFLOW-597 > Project: Apache Airflow > Issue Type: Bug >Reporter: George Leslie-Waksman >Assignee: George Leslie-Waksman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-airflow git commit: [AIRFLOW-597] Check if content is None, not false-equivalent
Repository: incubator-airflow Updated Branches: refs/heads/master 46236fa70 -> 2f2612669 [AIRFLOW-597] Check if content is None, not false-equivalent Closes #1856 from gwax/non_boolean_templates Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/2f261266 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/2f261266 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/2f261266 Branch: refs/heads/master Commit: 2f261266942b18a0458472c3db8daae554e6b520 Parents: 46236fa Author: George Leslie-Waksman Authored: Wed Oct 26 23:15:29 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 23:15:41 2016 -0700 -- airflow/models.py | 5 +++-- tests/core.py | 16 2 files changed, 19 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/2f261266/airflow/models.py -- diff --git a/airflow/models.py b/airflow/models.py index c6aa56b..44d9647 100755 --- a/airflow/models.py +++ b/airflow/models.py @@ -2164,8 +2164,9 @@ class BaseOperator(object): # Getting the content of files for template_field / template_ext for attr in self.template_fields: content = getattr(self, attr) -if (content and isinstance(content, six.string_types) and -any([content.endswith(ext) for ext in self.template_ext])): +if content is not None and \ +isinstance(content, six.string_types) and \ +any([content.endswith(ext) for ext in self.template_ext]): env = self.dag.get_template_env() try: setattr(self, attr, env.loader.get_source(env, content)[0]) http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/2f261266/tests/core.py -- diff --git a/tests/core.py b/tests/core.py index e2612f4..912ee9f 100644 --- a/tests/core.py +++ b/tests/core.py @@ -587,6 +587,22 @@ class CoreTest(unittest.TestCase): t.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, ignore_ti_state=True) assert val['success'] +def test_template_non_bool(self): +""" +Test templates can handle objects with no sense of truthiness +""" +class NonBoolObject(object): +def __len__(self): +return NotImplemented +def __bool__(self): +return NotImplemented + +t = OperatorSubclass( +task_id='test_bad_template_obj', +some_templated_field=NonBoolObject(), +dag=self.dag) +t.resolve_template_files() + def test_import_examples(self): self.assertEqual(len(self.dagbag.dags), NUM_EXAMPLE_DAGS)
[jira] [Updated] (AIRFLOW-601) Airflow's Hive integration doesn't scale up to tables with more than 32,767 partitions (and this is really easy to fix)
[
https://issues.apache.org/jira/browse/AIRFLOW-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Allman updated AIRFLOW-601:
---
Summary: Airflow's Hive integration doesn't scale up to tables with more
than 32,767 partitions (and this is really easy to fix) (was: Airflow's Hive
integration doesn't scale up to tables with more than 32,767 partitions)
> Airflow's Hive integration doesn't scale up to tables with more than 32,767
> partitions (and this is really easy to fix)
> ---
>
> Key: AIRFLOW-601
> URL: https://issues.apache.org/jira/browse/AIRFLOW-601
> Project: Apache Airflow
> Issue Type: Bug
> Components: hive_hooks
>Reporter: Michael Allman
>
> The Hive metastore API has a rather confusing method signature for
> {{listPartitions}}. The last method parameter specifies the maximum number of
> partitions to return, and its type is a Java short. So Airflow passes the
> maximum Java short value (32,767) and notes the limitation in its API docs:
> https://github.com/apache/incubator-airflow/blob/92064398c4c982a310925da376745a1713bf96e2/airflow/hooks/hive_hooks.py#L497-L499
> *However*, if you pass the magic number -1 as the "limit", then the
> metastore API will return *all* partitions. I found this documented here:
> https://issues.cloudera.org/browse/IMPALA-749
> I've also tried this myself on a Hive table with 80,000+ partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (AIRFLOW-601) Airflow's Hive integration doesn't scale up to tables with more than 32,767 partitions
Michael Allman created AIRFLOW-601:
--
Summary: Airflow's Hive integration doesn't scale up to tables
with more than 32,767 partitions
Key: AIRFLOW-601
URL: https://issues.apache.org/jira/browse/AIRFLOW-601
Project: Apache Airflow
Issue Type: Bug
Components: hive_hooks
Reporter: Michael Allman
The Hive metastore API has a rather confusing method signature for
{{listPartitions}}. The last method parameter specifies the maximum number of
partitions to return, and its type is a Java short. So Airflow passes the
maximum Java short value (32,767) and notes the limitation in its API docs:
https://github.com/apache/incubator-airflow/blob/92064398c4c982a310925da376745a1713bf96e2/airflow/hooks/hive_hooks.py#L497-L499
*However*, if you pass the magic number -1 as the "limit", then the metastore
API will return *all* partitions. I found this documented here:
https://issues.cloudera.org/browse/IMPALA-749
I've also tried this myself on a Hive table with 80,000+ partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AIRFLOW-586) Importing dags/test_dag.py fails between 0 and 3 o'clock
[
https://issues.apache.org/jira/browse/AIRFLOW-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610436#comment-15610436
]
ASF subversion and git services commented on AIRFLOW-586:
-
Commit 46236fa70702de1d25253c8972f9843e247ddc7a in incubator-airflow's branch
refs/heads/master from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=46236fa ]
[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
dags/test_dag.py tries to set START_DATE to 3
hours before using
datetime.replace, but it doesn't support minus
value as argument.
So we have to use timedelta instead of simple
numeric subtraction.
Closes #1852 from sekikn/AIRFLOW-586
> Importing dags/test_dag.py fails between 0 and 3 o'clock
>
>
> Key: AIRFLOW-586
> URL: https://issues.apache.org/jira/browse/AIRFLOW-586
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>
> It raises the following error:
> {code}
> $ airflow webserver
> (snip)
> [2016-10-23 00:07:30,613] [28693] {models.py:266} ERROR - Failed to import:
> /home/sekikn/airflow/dags/test_dag.py
> Traceback (most recent call last):
> File "/home/sekikn/incubator-airflow/airflow/models.py", line 263, in
> process_file
> m = imp.load_source(mod_name, filepath)
> File "/home/sekikn/airflow/dags/test_dag.py", line 20, in
> now_to_the_hour = now.replace(hour=now.time().hour-3 , minute=0,
> second=0, microsecond=0)
> ValueError: hour must be in 0..23
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
incubator-airflow git commit: [AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
Repository: incubator-airflow Updated Branches: refs/heads/master 2bbcdc3a9 -> 46236fa70 [AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m. dags/test_dag.py tries to set START_DATE to 3 hours before using datetime.replace, but it doesn't support minus value as argument. So we have to use timedelta instead of simple numeric subtraction. Closes #1852 from sekikn/AIRFLOW-586 Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/46236fa7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/46236fa7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/46236fa7 Branch: refs/heads/master Commit: 46236fa70702de1d25253c8972f9843e247ddc7a Parents: 2bbcdc3 Author: Kengo Seki Authored: Wed Oct 26 19:47:35 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 19:47:35 2016 -0700 -- dags/test_dag.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/46236fa7/dags/test_dag.py -- diff --git a/dags/test_dag.py b/dags/test_dag.py index 9c506b2..a1cbb74 100644 --- a/dags/test_dag.py +++ b/dags/test_dag.py @@ -14,10 +14,10 @@ from airflow import DAG from airflow.operators.dummy_operator import DummyOperator -from datetime import datetime +from datetime import datetime, timedelta now = datetime.now() -now_to_the_hour = now.replace(hour=now.time().hour-3 , minute=0, second=0, microsecond=0) +now_to_the_hour = (now - timedelta(0, 0, 0, 0, 0, 3)).replace(minute=0, second=0, microsecond=0) START_DATE = now_to_the_hour DAG_NAME = 'test_dag_v1'
[jira] [Resolved] (AIRFLOW-586) Importing dags/test_dag.py fails between 0 and 3 o'clock
[
https://issues.apache.org/jira/browse/AIRFLOW-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Anand resolved AIRFLOW-586.
-
Resolution: Fixed
Issue resolved by pull request #1852
[https://github.com/apache/incubator-airflow/pull/1852]
> Importing dags/test_dag.py fails between 0 and 3 o'clock
>
>
> Key: AIRFLOW-586
> URL: https://issues.apache.org/jira/browse/AIRFLOW-586
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>
> It raises the following error:
> {code}
> $ airflow webserver
> (snip)
> [2016-10-23 00:07:30,613] [28693] {models.py:266} ERROR - Failed to import:
> /home/sekikn/airflow/dags/test_dag.py
> Traceback (most recent call last):
> File "/home/sekikn/incubator-airflow/airflow/models.py", line 263, in
> process_file
> m = imp.load_source(mod_name, filepath)
> File "/home/sekikn/airflow/dags/test_dag.py", line 20, in
> now_to_the_hour = now.replace(hour=now.time().hour-3 , minute=0,
> second=0, microsecond=0)
> ValueError: hour must be in 0..23
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AIRFLOW-586) Importing dags/test_dag.py fails between 0 and 3 o'clock
[
https://issues.apache.org/jira/browse/AIRFLOW-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610437#comment-15610437
]
ASF subversion and git services commented on AIRFLOW-586:
-
Commit 46236fa70702de1d25253c8972f9843e247ddc7a in incubator-airflow's branch
refs/heads/master from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=46236fa ]
[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
dags/test_dag.py tries to set START_DATE to 3
hours before using
datetime.replace, but it doesn't support minus
value as argument.
So we have to use timedelta instead of simple
numeric subtraction.
Closes #1852 from sekikn/AIRFLOW-586
> Importing dags/test_dag.py fails between 0 and 3 o'clock
>
>
> Key: AIRFLOW-586
> URL: https://issues.apache.org/jira/browse/AIRFLOW-586
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>
> It raises the following error:
> {code}
> $ airflow webserver
> (snip)
> [2016-10-23 00:07:30,613] [28693] {models.py:266} ERROR - Failed to import:
> /home/sekikn/airflow/dags/test_dag.py
> Traceback (most recent call last):
> File "/home/sekikn/incubator-airflow/airflow/models.py", line 263, in
> process_file
> m = imp.load_source(mod_name, filepath)
> File "/home/sekikn/airflow/dags/test_dag.py", line 20, in
> now_to_the_hour = now.replace(hour=now.time().hour-3 , minute=0,
> second=0, microsecond=0)
> ValueError: hour must be in 0..23
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AIRFLOW-600) Add BandwidthX to Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610400#comment-15610400 ] Siddharth Anand commented on AIRFLOW-600: - Assigning to you so you get a hang of submitting a PR. Add your company and your github handle to https://github.com/apache/incubator-airflow/blob/master/README.md in alphabetic (by company name) order. -s > Add BandwidthX to Airflow users > --- > > Key: AIRFLOW-600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-600 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Dinesh Sharma >Assignee: Dinesh Sharma >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AIRFLOW-600) Add BandwidthX to Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand reassigned AIRFLOW-600: --- Assignee: Siddharth Anand > Add BandwidthX to Airflow users > --- > > Key: AIRFLOW-600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-600 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Dinesh Sharma >Assignee: Siddharth Anand >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-600) Add BandwidthX to Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand updated AIRFLOW-600: Assignee: Dinesh Sharma (was: Siddharth Anand) > Add BandwidthX to Airflow users > --- > > Key: AIRFLOW-600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-600 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Dinesh Sharma >Assignee: Dinesh Sharma >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-600) Add BandwidthX to Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610093#comment-15610093 ] Siddharth Anand commented on AIRFLOW-600: - Once you have a PR, please be sure to follow https://cwiki.apache.org/confluence/display/AIRFLOW/Contributors%27+Guide Essentially, add your company name in alphabetic order to the readme and prepend [AIRFLOW-600] to the commit message (and PR). -s > Add BandwidthX to Airflow users > --- > > Key: AIRFLOW-600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-600 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Dinesh Sharma >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-600) Add BandwidthX to Airflow users
Dinesh Sharma created AIRFLOW-600: - Summary: Add BandwidthX to Airflow users Key: AIRFLOW-600 URL: https://issues.apache.org/jira/browse/AIRFLOW-600 Project: Apache Airflow Issue Type: Task Components: Documentation Reporter: Dinesh Sharma Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AIRFLOW-453) Flask Admin page for XComs
[ https://issues.apache.org/jira/browse/AIRFLOW-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand resolved AIRFLOW-453. - Resolution: Fixed > Flask Admin page for XComs > -- > > Key: AIRFLOW-453 > URL: https://issues.apache.org/jira/browse/AIRFLOW-453 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Sumit Maheshwari >Assignee: Sumit Maheshwari > Labels: xcom > > We use XComs pretty much, but we've dig up database when we have to debug > something. Also the value stored as blob so it's difficult to understand by > just looking at it. An admin page would be very helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-453) Flask Admin page for XComs
[ https://issues.apache.org/jira/browse/AIRFLOW-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609764#comment-15609764 ] ASF subversion and git services commented on AIRFLOW-453: - Commit 2bbcdc3a9958103db49fb98727831d1aee4e12ed in incubator-airflow's branch refs/heads/master from [~msumit] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=2bbcdc3 ] [AIRFLOW-453] Add XCom Admin Page Closes #1756 from msumit/AIRFLOW-453 > Flask Admin page for XComs > -- > > Key: AIRFLOW-453 > URL: https://issues.apache.org/jira/browse/AIRFLOW-453 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Sumit Maheshwari >Assignee: Sumit Maheshwari > Labels: xcom > > We use XComs pretty much, but we've dig up database when we have to debug > something. Also the value stored as blob so it's difficult to understand by > just looking at it. An admin page would be very helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-453) Flask Admin page for XComs
[ https://issues.apache.org/jira/browse/AIRFLOW-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609765#comment-15609765 ] ASF subversion and git services commented on AIRFLOW-453: - Commit 2bbcdc3a9958103db49fb98727831d1aee4e12ed in incubator-airflow's branch refs/heads/master from [~msumit] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=2bbcdc3 ] [AIRFLOW-453] Add XCom Admin Page Closes #1756 from msumit/AIRFLOW-453 > Flask Admin page for XComs > -- > > Key: AIRFLOW-453 > URL: https://issues.apache.org/jira/browse/AIRFLOW-453 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Sumit Maheshwari >Assignee: Sumit Maheshwari > Labels: xcom > > We use XComs pretty much, but we've dig up database when we have to debug > something. Also the value stored as blob so it's difficult to understand by > just looking at it. An admin page would be very helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-airflow git commit: [AIRFLOW-453] Add XCom Admin Page
Repository: incubator-airflow
Updated Branches:
refs/heads/master 84cb7e809 -> 2bbcdc3a9
[AIRFLOW-453] Add XCom Admin Page
Closes #1756 from msumit/AIRFLOW-453
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/2bbcdc3a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/2bbcdc3a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/2bbcdc3a
Branch: refs/heads/master
Commit: 2bbcdc3a9958103db49fb98727831d1aee4e12ed
Parents: 84cb7e8
Author: Sumit Maheshwari
Authored: Wed Oct 26 14:42:18 2016 -0700
Committer: Siddharth Anand
Committed: Wed Oct 26 14:42:18 2016 -0700
--
airflow/www/app.py | 2 ++
airflow/www/views.py | 21 +
tests/core.py| 2 ++
3 files changed, 25 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/2bbcdc3a/airflow/www/app.py
--
diff --git a/airflow/www/app.py b/airflow/www/app.py
index c4dff6f..6ab4ffd 100644
--- a/airflow/www/app.py
+++ b/airflow/www/app.py
@@ -89,6 +89,8 @@ def create_app(config=None):
models.Connection, Session, name="Connections", category="Admin"))
av(vs.VariableView(
models.Variable, Session, name="Variables", category="Admin"))
+av(vs.XComView(
+models.XCom, Session, name="XComs", category="Admin"))
admin.add_link(base.MenuLink(
category='Docs', name='Documentation',
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/2bbcdc3a/airflow/www/views.py
--
diff --git a/airflow/www/views.py b/airflow/www/views.py
index b0ccfaa..36384d8 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -2161,6 +2161,27 @@ class VariableView(wwwutils.DataProfilingMixin,
AirflowModelView):
form.val.data = '*' * 8
+class XComView(wwwutils.LoginMixin, AirflowModelView):
+verbose_name = "XCom"
+verbose_name_plural = "XComs"
+page_size = 20
+
+form_columns = (
+'key',
+'value',
+'execution_date',
+'task_id',
+'dag_id',
+)
+
+form_extra_fields = {
+'value': StringField('Value'),
+}
+
+column_filters = ('key', 'timestamp', 'execution_date', 'task_id',
'dag_id')
+column_searchable_list = ('key', 'timestamp', 'execution_date', 'task_id',
'dag_id')
+
+
class JobModelView(ModelViewOnly):
verbose_name_plural = "jobs"
verbose_name = "job"
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/2bbcdc3a/tests/core.py
--
diff --git a/tests/core.py b/tests/core.py
index 9ce71d4..e2612f4 100644
--- a/tests/core.py
+++ b/tests/core.py
@@ -1399,6 +1399,8 @@ class WebUiTests(unittest.TestCase):
response = self.app.get(
"/admin/airflow/paused?"
"dag_id=example_python_operator&is_paused=false")
+response = self.app.get("/admin/xcom", follow_redirects=True)
+assert "Xcoms" in response.data.decode('utf-8')
def test_charts(self):
session = Session()
incubator-airflow git commit: closes apache/incubator-airflow#1301 *obsolete*
Repository: incubator-airflow Updated Branches: refs/heads/master 1425d7246 -> 64d7e3fde closes apache/incubator-airflow#1301 *obsolete* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/64d7e3fd Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/64d7e3fd Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/64d7e3fd Branch: refs/heads/master Commit: 64d7e3fdebec95d6f8a57a20afc28c19f373fc9e Parents: 1425d72 Author: Arthur Wiedmer Authored: Wed Oct 26 11:23:39 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:23:39 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1384 *obsolete*
Repository: incubator-airflow Updated Branches: refs/heads/master c1747b7bb -> 84cb7e809 closes apache/incubator-airflow#1384 *obsolete* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/84cb7e80 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/84cb7e80 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/84cb7e80 Branch: refs/heads/master Commit: 84cb7e80959ad2896fa9c4f67f58c274c66c0823 Parents: c1747b7 Author: Arthur Wiedmer Authored: Wed Oct 26 11:26:59 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:26:59 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1379 *obsolete*
Repository: incubator-airflow Updated Branches: refs/heads/master 64d7e3fde -> c1747b7bb closes apache/incubator-airflow#1379 *obsolete* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/c1747b7b Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/c1747b7b Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/c1747b7b Branch: refs/heads/master Commit: c1747b7bb0cdfcd3f71dc00cda0eb39a36fcbabc Parents: 64d7e3f Author: Arthur Wiedmer Authored: Wed Oct 26 11:26:09 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:26:09 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1274 *no movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master f2f1b29a1 -> 1425d7246 closes apache/incubator-airflow#1274 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/1425d724 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/1425d724 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/1425d724 Branch: refs/heads/master Commit: 1425d72468227486e002fa6406f1ee718cd73d67 Parents: f2f1b29 Author: Arthur Wiedmer Authored: Wed Oct 26 11:21:59 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:21:59 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1276 *no movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master fb4050579 -> f2f1b29a1 closes apache/incubator-airflow#1276 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/f2f1b29a Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/f2f1b29a Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/f2f1b29a Branch: refs/heads/master Commit: f2f1b29a1b16449d88ccd480048f1f18eccbbeb2 Parents: fb40505 Author: Arthur Wiedmer Authored: Wed Oct 26 11:21:28 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:21:28 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#908 *no movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master 8ec5f7f99 -> 3c5d98082 closes apache/incubator-airflow#908 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3c5d9808 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3c5d9808 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3c5d9808 Branch: refs/heads/master Commit: 3c5d980822fa966b12d8809562b94699a31c0220 Parents: 8ec5f7f Author: Arthur Wiedmer Authored: Wed Oct 26 11:14:18 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:14:18 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#989 *no movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master 3c5d98082 -> fb4050579 closes apache/incubator-airflow#989 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/fb405057 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/fb405057 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/fb405057 Branch: refs/heads/master Commit: fb40505795d3f05378f907275f740b11762ceba1 Parents: 3c5d980 Author: Arthur Wiedmer Authored: Wed Oct 26 11:15:27 2016 -0700 Committer: Arthur Wiedmer Committed: Wed Oct 26 11:15:27 2016 -0700 -- --
[3/3] incubator-airflow git commit: closes apache/incubator-airflow#746 *no movement from submitter*
closes apache/incubator-airflow#746 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/8ec5f7f9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/8ec5f7f9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/8ec5f7f9 Branch: refs/heads/master Commit: 8ec5f7f99bb73a7eb89945cdcb9a564596f615ef Parents: ad3dfa6 Author: Maxime Beauchemin Authored: Wed Oct 26 10:54:55 2016 -0700 Committer: Maxime Beauchemin Committed: Wed Oct 26 10:54:55 2016 -0700 -- --
[2/3] incubator-airflow git commit: closes apache/incubator-airflow#772 *no movement from submitter*
closes apache/incubator-airflow#772 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/ad3dfa6d Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/ad3dfa6d Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/ad3dfa6d Branch: refs/heads/master Commit: ad3dfa6daa876e836ed25ebd217eba3aa88813b3 Parents: ca343d1 Author: Maxime Beauchemin Authored: Wed Oct 26 10:54:38 2016 -0700 Committer: Maxime Beauchemin Committed: Wed Oct 26 10:54:38 2016 -0700 -- --
[1/3] incubator-airflow git commit: closes apache/incubator-airflow#746 *no movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master 44407ff1b -> 8ec5f7f99 closes apache/incubator-airflow#746 *no movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/ca343d1a Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/ca343d1a Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/ca343d1a Branch: refs/heads/master Commit: ca343d1a422cfc1c3b9ff7345225ef7611bf1ed1 Parents: 44407ff Author: Maxime Beauchemin Authored: Wed Oct 26 10:54:19 2016 -0700 Committer: Maxime Beauchemin Committed: Wed Oct 26 10:54:19 2016 -0700 -- --
[jira] [Resolved] (AIRFLOW-599) Add Spotify to list of Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand resolved AIRFLOW-599. - Resolution: Fixed Issue resolved by pull request #1855 [https://github.com/apache/incubator-airflow/pull/1855] > Add Spotify to list of Airflow users > > > Key: AIRFLOW-599 > URL: https://issues.apache.org/jira/browse/AIRFLOW-599 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Zachary Nichols >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-599) Add Spotify to list of Airflow users
[ https://issues.apache.org/jira/browse/AIRFLOW-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609079#comment-15609079 ] ASF subversion and git services commented on AIRFLOW-599: - Commit 44407ff1bce1028e7903a36ebad190a2514eb6e0 in incubator-airflow's branch refs/heads/master from [~znichols] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=44407ff ] [AIRFLOW-599] Adding spotify to Airflow users Dear Airflow Maintainers, Please accept this PR that addresses the following issues: - Adds Spotify to list of Airflow users Closes #1855 from znichols/spotify_use > Add Spotify to list of Airflow users > > > Key: AIRFLOW-599 > URL: https://issues.apache.org/jira/browse/AIRFLOW-599 > Project: Apache Airflow > Issue Type: Task > Components: Documentation >Reporter: Zachary Nichols >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-airflow git commit: [AIRFLOW-599] Adding spotify to Airflow users
Repository: incubator-airflow Updated Branches: refs/heads/master 458e7165a -> 44407ff1b [AIRFLOW-599] Adding spotify to Airflow users Dear Airflow Maintainers, Please accept this PR that addresses the following issues: - Adds Spotify to list of Airflow users Closes #1855 from znichols/spotify_use Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/44407ff1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/44407ff1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/44407ff1 Branch: refs/heads/master Commit: 44407ff1bce1028e7903a36ebad190a2514eb6e0 Parents: 458e716 Author: Zack Nichols Authored: Wed Oct 26 10:25:18 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 10:25:18 2016 -0700 -- README.md | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/44407ff1/README.md -- diff --git a/README.md b/README.md index fe34988..3e3a14a 100644 --- a/README.md +++ b/README.md @@ -125,6 +125,7 @@ Currently **officially** using Airflow: 1. [Sidecar](https://hello.getsidecar.com/) [[@getsidecar](https://github.com/getsidecar)] 1. [SimilarWeb](https://www.similarweb.com/) [[@similarweb](https://github.com/similarweb)] 1. [SmartNews](https://www.smartnews.com/) [[@takus](https://github.com/takus)] +1. [Spotify](https://github.com/spotify) [[@znichols](https://github.com/znichols)] 1. Stripe [[@jbalogh](https://github.com/jbalogh)] 1. [Thumbtack](https://www.thumbtack.com/) [[@natekupp](https://github.com/natekupp)] 1. [T2 Systems](http://t2systems.com) [[@unclaimedpants](https://github.com/unclaimedpants)]
[jira] [Created] (AIRFLOW-599) Add Spotify to list of Airflow users
Zachary created AIRFLOW-599: --- Summary: Add Spotify to list of Airflow users Key: AIRFLOW-599 URL: https://issues.apache.org/jira/browse/AIRFLOW-599 Project: Apache Airflow Issue Type: Task Components: Documentation Reporter: Zachary Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-598) Can't install Airlfow
Sadok Ben Yahya created AIRFLOW-598:
---
Summary: Can't install Airlfow
Key: AIRFLOW-598
URL: https://issues.apache.org/jira/browse/AIRFLOW-598
Project: Apache Airflow
Issue Type: Bug
Affects Versions: Airflow 1.7.1.3
Reporter: Sadok Ben Yahya
Priority: Critical
Hi,
i get this error when trying to install airflow 1.7.1.3 on ubuntu 14.04.5 with
kernel 4.4.0-45-generic and python version 2.7.6.
Some Logs:
-
Installed /tmp/pip-build-1KTFCl/pandas/.eggs/numpy-1.11.2-py2.7-linux-x86_64.egg
running egg_info
creating pip-egg-info/pandas.egg-info
writing requirements to pip-egg-info/pandas.egg-info/requires.txt
writing pip-egg-info/pandas.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pandas.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pandas.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pandas.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
package init file 'pandas/io/tests/sas/__init__.py' not found (or not a regular
file)
pandas/index.pyx: cannot find cimported module 'datetime'
pandas/index.pyx: cannot find cimported module 'util'
./pandas/hashtable.pxd: cannot find cimported module 'khash'
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip-build-1KTFCl/pandas/setup.py", line 680, in
**setuptools_kwargs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "", line 15, in replacement_run
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py",
line 306, in find_sources
mm.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py",
line 533, in run
self.add_defaults()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py",
line 562, in add_defaults
sdist.add_defaults(self)
File
"/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line
36, in add_defaults
self._add_defaults_ext()
File
"/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line
119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/usr/local/lib/python2.7/dist-packages/Cython/Distutils/build_ext.py",
line 19, in finalize_options
self.distribution.ext_modules)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 809, in cythonize
aliases=aliases)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 714, in create_extension_list
kwds = deps.distutils_info(file, aliases, base).values
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 590, in distutils_info
return (self.transitive_merge(filename, self.distutils_info0,
DistutilsInfo.merge)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 600, in transitive_merge
node, extract, merge, seen, {}, self.cimported_files)[0]
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 605, in transitive_merge_helper
deps = extract(node)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 581, in distutils_info0
externs = self.cimports_and_externs(filename)[1]
File "/usr/local/lib/python2.7/dist-packages/Cython/Utils.py", line 44, in
wrapper
res = cache[args] = f(self, *args)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 491, in cimports_and_externs
for include in self.included_files(filename):
File "/usr/local/lib/python2.7/dist-packages/Cython/Utils.py", line 44, in
wrapper
res = cache[args] = f(self, *args)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py",
line 474, in included_files
include_path = self.context.find_include_file(include, None)
File "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/Main.py", line
274, in find_include_file
error(pos, "'%s' not found" % filename)
File "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/Errors.py", line
177, in error
raise InternalError(message)
Cython.Compiler.Errors.InternalError: Internal compiler error:
'algos_common_helper.pxi' not found
Cleaning up...
Command python setup.py egg_info failed with error code 1 in
/tmp/pip-build-1KTFCl/pandas
Traceback
incubator-airflow git commit: closes apache/incubator-airflow#1758 *No movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master a9477257c -> 458e7165a closes apache/incubator-airflow#1758 *No movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/458e7165 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/458e7165 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/458e7165 Branch: refs/heads/master Commit: 458e7165af283b67c717776b6bda1df9d02c4814 Parents: a947725 Author: Siddharth Anand Authored: Wed Oct 26 00:33:02 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 00:33:02 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1572 *No movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master 3c29ed334 -> a9477257c closes apache/incubator-airflow#1572 *No movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a9477257 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/a9477257 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/a9477257 Branch: refs/heads/master Commit: a9477257cb864efc8eb9f4fdcbcd48af9ba4b459 Parents: 3c29ed3 Author: Siddharth Anand Authored: Wed Oct 26 00:31:41 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 00:31:41 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1434 *No movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master 7766ab7e9 -> 3c29ed334 closes apache/incubator-airflow#1434 *No movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3c29ed33 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3c29ed33 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3c29ed33 Branch: refs/heads/master Commit: 3c29ed3344b12240096d4070e22333b9ae319d5d Parents: 7766ab7 Author: Siddharth Anand Authored: Wed Oct 26 00:28:24 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 00:28:24 2016 -0700 -- --
incubator-airflow git commit: closes apache/incubator-airflow#1339 *No movement from submitter*
Repository: incubator-airflow Updated Branches: refs/heads/master e835cb356 -> 7766ab7e9 closes apache/incubator-airflow#1339 *No movement from submitter* Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/7766ab7e Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/7766ab7e Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/7766ab7e Branch: refs/heads/master Commit: 7766ab7e9aaf083db9e9edf753ef1dbbdcb59352 Parents: e835cb3 Author: Siddharth Anand Authored: Wed Oct 26 00:27:08 2016 -0700 Committer: Siddharth Anand Committed: Wed Oct 26 00:27:08 2016 -0700 -- --
