[jira] [Commented] (AIRFLOW-262) Simplify commands in MANIFEST.in

[Chris Riccomini (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341018#comment-15341018
 ] 

Chris Riccomini commented on AIRFLOW-262:
-

+1 thx

> Simplify commands in MANIFEST.in
> 
>
> Key: AIRFLOW-262
> URL: https://issues.apache.org/jira/browse/AIRFLOW-262
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
>Priority: Minor
> Fix For: Airflow 1.8
>
>
> `recursive-include` command with the wild card pattern can be simplified to 
> use graft command instead. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (AIRFLOW-262) Simplify commands in MANIFEST.in

[Chris Riccomini (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Riccomini closed AIRFLOW-262.
---
   Resolution: Fixed
Fix Version/s: Airflow 1.8

> Simplify commands in MANIFEST.in
> 
>
> Key: AIRFLOW-262
> URL: https://issues.apache.org/jira/browse/AIRFLOW-262
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
>Priority: Minor
> Fix For: Airflow 1.8
>
>
> `recursive-include` command with the wild card pattern can be simplified to 
> use graft command instead. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-airflow git commit: [AIRFLOW-262] Simplify commands in MANIFEST.in

[criccomini]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master 26c31d9bc -> 7992036f5


[AIRFLOW-262] Simplify commands in MANIFEST.in


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/05a1150b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/05a1150b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/05a1150b

Branch: refs/heads/master
Commit: 05a1150bd76cc65865aef870c0cb9f23fbbc2319
Parents: 45b735b
Author: Ajay Yadav 
Authored: Mon Jun 20 23:49:59 2016 +0530
Committer: Ajay Yadav 
Committed: Mon Jun 20 23:49:59 2016 +0530

--
 MANIFEST.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/05a1150b/MANIFEST.in
--
diff --git a/MANIFEST.in b/MANIFEST.in
index 284700e..67e3a77 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,4 +1,4 @@
-recursive-include airflow/www/templates *
-recursive-include airflow/www/static *
+graft airflow/www/templates
+graft airflow/www/static
 include airflow/alembic.ini
 include requirements.txt

[2/2] incubator-airflow git commit: Merge pull request #1612 from ajayyadava/262

[criccomini]

Merge pull request #1612 from ajayyadava/262


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/7992036f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/7992036f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/7992036f

Branch: refs/heads/master
Commit: 7992036f5d07204a2eab3adc996cfa0e072d0196
Parents: 26c31d9 05a1150
Author: Chris Riccomini 
Authored: Mon Jun 20 20:53:58 2016 -0700
Committer: Chris Riccomini 
Committed: Mon Jun 20 20:53:58 2016 -0700

--
 MANIFEST.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--

[jira] [Closed] (AIRFLOW-6) Remove high-charts

[Chris Riccomini (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Riccomini closed AIRFLOW-6.
-
   Resolution: Fixed
Fix Version/s: Airflow 1.8

> Remove high-charts
> --
>
> Key: AIRFLOW-6
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Chris Riccomini
>Assignee: Maxime Beauchemin
> Fix For: Airflow 1.8
>
>
> According to LEGAL-246, we can't use High Charts in Airflow. The license is 
> incompatible with Apache. We should replace it with another library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-258) Always load plugin executors

[Chris Riccomini (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341003#comment-15341003
 ] 

Chris Riccomini commented on AIRFLOW-258:
-

Makes sense to me. Want to send a PR?

> Always load plugin executors
> 
>
> Key: AIRFLOW-258
> URL: https://issues.apache.org/jira/browse/AIRFLOW-258
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Alexandr Nikitin
>
> At the moment we load plugin executors only when the specified `EXECUTOR` 
> field in config isn't from existing executors: 'LocalExecutor', 
> 'CeleryExecutor', 'SequentialExecutor', 'MesosExecutor'.
> If the default executor belongs to existing executors we won't load plugins 
> executors at all. This breaks if the plugin executor used in SubDAG.
> We should load plugin executors always.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] incubator-airflow git commit: Merge pull request #1547 from artwr/artwr_fix_sensor_timeout

[arthur]

Merge pull request #1547 from artwr/artwr_fix_sensor_timeout


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/26c31d9b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/26c31d9b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/26c31d9b

Branch: refs/heads/master
Commit: 26c31d9bca16a88c915d0d501aaa58915056a2a9
Parents: 45b735b c38a5c2
Author: Arthur Wiedmer 
Authored: Mon Jun 20 16:02:14 2016 -0700
Committer: Arthur Wiedmer 
Committed: Mon Jun 20 16:02:14 2016 -0700

--
 airflow/operators/sensors.py |  4 +-
 tests/operators/sensors.py   | 77 ++-
 2 files changed, 77 insertions(+), 4 deletions(-)
--

[1/2] incubator-airflow git commit: [AIRFLOW-180] Fix timeout behavior for sensors

[arthur]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master 45b735bae -> 26c31d9bc


[AIRFLOW-180] Fix timeout behavior for sensors

In the previous state of the code, datetime.now was compared to
started_at and seconds was pulled out. It turns out that the seconds
attribute of a timedelta has a maximum of 86400 and the rolls up to 1 day.
The unintended consequence is that timeout larger than 86400 are
ignored, with sensors running forever.

To fix this we use the total_seconds method to get at the real
timedelta in seconds.


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/c38a5c2a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/c38a5c2a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/c38a5c2a

Branch: refs/heads/master
Commit: c38a5c2a8b227194ec52d81e8a5a85c97751ecd9
Parents: 45b735b
Author: Arthur Wiedmer 
Authored: Thu May 26 10:27:55 2016 -0700
Committer: Arthur Wiedmer 
Committed: Mon Jun 20 15:45:44 2016 -0700

--
 airflow/operators/sensors.py |  4 +-
 tests/operators/sensors.py   | 77 ++-
 2 files changed, 77 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/c38a5c2a/airflow/operators/sensors.py
--
diff --git a/airflow/operators/sensors.py b/airflow/operators/sensors.py
index 5276f6e..4e4cb3b 100644
--- a/airflow/operators/sensors.py
+++ b/airflow/operators/sensors.py
@@ -69,12 +69,12 @@ class BaseSensorOperator(BaseOperator):
 def execute(self, context):
 started_at = datetime.now()
 while not self.poke(context):
-sleep(self.poke_interval)
-if (datetime.now() - started_at).seconds > self.timeout:
+if (datetime.now() - started_at).total_seconds() > self.timeout:
 if self.soft_fail:
 raise AirflowSkipException('Snap. Time is OUT.')
 else:
 raise AirflowSensorTimeout('Snap. Time is OUT.')
+sleep(self.poke_interval)
 logging.info("Success criteria met. Exiting.")
 
 

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/c38a5c2a/tests/operators/sensors.py
--
diff --git a/tests/operators/sensors.py b/tests/operators/sensors.py
index 025790e..325ee8d 100644
--- a/tests/operators/sensors.py
+++ b/tests/operators/sensors.py
@@ -12,11 +12,84 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+
+import logging
 import os
+import time
 import unittest
 
-from airflow.operators.sensors import HttpSensor
-from airflow.exceptions import AirflowException
+from datetime import datetime, timedelta
+
+from airflow import DAG, configuration
+from airflow.operators.sensors import HttpSensor, BaseSensorOperator
+from airflow.utils.decorators import apply_defaults
+from airflow.exceptions import (AirflowException,
+AirflowSensorTimeout,
+AirflowSkipException)
+configuration.test_mode()
+
+DEFAULT_DATE = datetime(2015, 1, 1)
+TEST_DAG_ID = 'unit_test_dag'
+
+
+class TimeoutTestSensor(BaseSensorOperator):
+"""
+Sensor that always returns the return_value provided
+
+:param return_value: Set to true to mark the task as SKIPPED on failure
+:type return_value: any
+"""
+
+@apply_defaults
+def __init__(
+self,
+return_value=False,
+*args,
+**kwargs):
+self.return_value = return_value
+super(TimeoutTestSensor, self).__init__(*args, **kwargs)
+
+def poke(self, context):
+return self.return_value
+
+def execute(self, context):
+started_at = datetime.now()
+time_jump = self.params.get('time_jump')
+while not self.poke(context):
+if time_jump:
+started_at -= time_jump
+if (datetime.now() - started_at).total_seconds() > self.timeout:
+if self.soft_fail:
+raise AirflowSkipException('Snap. Time is OUT.')
+else:
+raise AirflowSensorTimeout('Snap. Time is OUT.')
+time.sleep(self.poke_interval)
+logging.info("Success criteria met. Exiting.")
+
+
+class SensorTimeoutTest(unittest.TestCase):
+def setUp(self):
+configuration.test_mode()
+args = {
+'owner': 'airflow',
+'start_date': DEFAULT_DATE
+}
+dag = DAG(TEST_DAG_ID, default_args=args)
+self.dag = dag
+
+def test_timeout(self):
+t = 

[jira] [Commented] (AIRFLOW-263) Backtick file introduced by Highcharts refactor

[Arthur Wiedmer (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340606#comment-15340606
 ] 

Arthur Wiedmer commented on AIRFLOW-263:


PR here : https://github.com/apache/incubator-airflow/pull/1613

> Backtick file introduced by Highcharts refactor
> ---
>
> Key: AIRFLOW-263
> URL: https://issues.apache.org/jira/browse/AIRFLOW-263
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Arthur Wiedmer
>Assignee: Arthur Wiedmer
>Priority: Minor
>
> A file named "`" was introduced during the Highcharts removal. See
> https://github.com/apache/incubator-airflow/commit/0a460081bc7cba2d05434148f092b87d35aa8cd3
> My best assessment, is that this was a temporary file created by mistake.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-247) EMR Hook, Operators, Sensor

[Greg Reda (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340593#comment-15340593
 ] 

Greg Reda commented on AIRFLOW-247:
---

Awesome! Looking forward to this.

> EMR Hook, Operators, Sensor
> ---
>
> Key: AIRFLOW-247
> URL: https://issues.apache.org/jira/browse/AIRFLOW-247
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Rob Froetscher
>Assignee: Rob Froetscher
>Priority: Minor
>
> Substory of https://issues.apache.org/jira/browse/AIRFLOW-115. It would be 
> nice to have an EMR hook and operators.
> Hook to generally interact with EMR.
> Operators to:
> * setup and start a job flow
> * add steps to an existing jobflow 
> A sensor to:
> * monitor completion and status of EMR jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (AIRFLOW-247) EMR Hook, Operators, Sensor

[Rob Froetscher (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on AIRFLOW-247 started by Rob Froetscher.
--
> EMR Hook, Operators, Sensor
> ---
>
> Key: AIRFLOW-247
> URL: https://issues.apache.org/jira/browse/AIRFLOW-247
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Rob Froetscher
>Assignee: Rob Froetscher
>Priority: Minor
>
> Substory of https://issues.apache.org/jira/browse/AIRFLOW-115. It would be 
> nice to have an EMR hook and operators.
> Hook to generally interact with EMR.
> Operators to:
> * setup and start a job flow
> * add steps to an existing jobflow 
> A sensor to:
> * monitor completion and status of EMR jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-247) EMR Hook, Operators, Sensor

[Chris Riccomini (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340558#comment-15340558
 ] 

Chris Riccomini commented on AIRFLOW-247:
-

[~rfroetscher], are you still planning to work on this? If not, we can 
re-assign to [~gjreda].

> EMR Hook, Operators, Sensor
> ---
>
> Key: AIRFLOW-247
> URL: https://issues.apache.org/jira/browse/AIRFLOW-247
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Rob Froetscher
>Assignee: Rob Froetscher
>Priority: Minor
>
> Substory of https://issues.apache.org/jira/browse/AIRFLOW-115. It would be 
> nice to have an EMR hook and operators.
> Hook to generally interact with EMR.
> Operators to:
> * setup and start a job flow
> * add steps to an existing jobflow 
> A sensor to:
> * monitor completion and status of EMR jobs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AIRFLOW-249) Refactor the SLA mecanism

[Chris Riccomini (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Riccomini updated AIRFLOW-249:

  Assignee: dud
External issue URL: https://github.com/apache/incubator-airflow/pull/1601

> Refactor the SLA mecanism
> -
>
> Key: AIRFLOW-249
> URL: https://issues.apache.org/jira/browse/AIRFLOW-249
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: dud
>Assignee: dud
>
> Hello
> I've noticed the SLA feature is currently behaving as follow :
> - it doesn't work on DAG scheduled @once or None because they have no 
> dag.followwing_schedule property
> - it keeps endlessly checking for SLA misses without ever worrying about any 
> end_date. Worse I noticed that emails are still being sent for runs that are 
> never happening because of end_date
> - it keeps checking for recent TIs even if SLA notification has been already 
> been sent for them
> - the SLA logic is only being fired after following_schedule + sla has 
> elapsed, in other words one has to wait for the next TI before having a 
> chance of getting any email. Also the email reports dag.following_schedule 
> time (I guess because it is close of TI.start_date), but unfortunately that 
> doesn't match what the task instances shows nor the log filename
> - the SLA logic is based on max(TI.execution_date) for the starting point of 
> its checks, that means that for a DAG whose SLA is longer than its schedule 
> period if half of the TIs are running longer than expected it will go 
> unnoticed. This could be demonstrated with a DAG like this one :
> {code}
> from airflow import DAG
> from airflow.operators import *
> from datetime import datetime, timedelta
> from time import sleep
> default_args = {
> 'owner': 'airflow',
> 'depends_on_past': False,
> 'start_date': datetime(2016, 6, 16, 12, 20),
> 'email': my_email
> 'sla': timedelta(minutes=2),
> }
> dag = DAG('unnoticed_sla', default_args=default_args, 
> schedule_interval=timedelta(minutes=1))
> def alternating_sleep(**kwargs):
> minute = kwargs['execution_date'].strftime("%M")
> is_odd = int(minute) % 2
> if is_odd:
> sleep(300)
> else:
> sleep(10)
> return True
> PythonOperator(
> task_id='sla_miss',
> python_callable=alternating_sleep,
> provide_context=True,
> dag=dag)
> {code}
> I've tried to rework the SLA triggering mechanism by addressing the above 
> points., please [have a look on 
> it|https://github.com/dud225/incubator-airflow/commit/972260354075683a8d55a1c960d839c37e629e7d]
> I made some tests with this patch :
> - the fluctuent DAG shown above no longer make Airflow skip any SLA event :
> {code}
>  task_id  |dag_id |   execution_date| email_sent | 
> timestamp  | description | notification_sent 
> --+---+-+++-+---
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:05:00 | t  | 2016-06-16 
> 15:08:26.058631 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:07:00 | t  | 2016-06-16 
> 15:10:06.093253 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:09:00 | t  | 2016-06-16 
> 15:12:06.241773 | | t
> {code}
> - on a normal DAG, the SLA is being triggred more quickly :
> {code}
> // start_date = 2016-06-16 15:55:00
> // end_date = 2016-06-16 16:00:00
> // schedule_interval =  timedelta(minutes=1)
> // sla = timedelta(minutes=2)
>  task_id  |dag_id |   execution_date| email_sent | 
> timestamp  | description | notification_sent 
> --+---+-+++-+---
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:55:00 | t  | 2016-06-16 
> 15:58:11.832299 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:56:00 | t  | 2016-06-16 
> 15:59:09.663778 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:57:00 | t  | 2016-06-16 
> 16:00:13.651422 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:58:00 | t  | 2016-06-16 
> 16:01:08.576399 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 15:59:00 | t  | 2016-06-16 
> 16:02:08.523486 | | t
>  sla_miss | dag_sla_miss1 | 2016-06-16 16:00:00 | t  | 2016-06-16 
> 16:03:08.538593 | | t
> (6 rows)
> {code}
> than before (current master branch) :
> {code}
> // start_date = 2016-06-16 15:40:00
> // end_date = 2016-06-16 15:45:00
> // schedule_interval =  timedelta(minutes=1)
> // sla = timedelta(minutes=2)
>  task_id  |dag_id |   execution_date| email_sent | 
> timestamp  | description | notification_sent 
> 

[jira] [Closed] (AIRFLOW-234) make task that aren't `running` self-terminate

[Chris Riccomini (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Riccomini closed AIRFLOW-234.
---
   Resolution: Fixed
 Assignee: Maxime Beauchemin
Fix Version/s: Airflow 1.8

> make task that aren't `running` self-terminate
> --
>
> Key: AIRFLOW-234
> URL: https://issues.apache.org/jira/browse/AIRFLOW-234
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Maxime Beauchemin
>Assignee: Maxime Beauchemin
> Fix For: Airflow 1.8
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (AIRFLOW-256) test_scheduler_reschedule fails due to heartrate check

[Chris Riccomini (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Riccomini closed AIRFLOW-256.
---
   Resolution: Fixed
 Assignee: Bolke de Bruin
Fix Version/s: Airflow 1.8

> test_scheduler_reschedule fails due to heartrate check
> --
>
> Key: AIRFLOW-256
> URL: https://issues.apache.org/jira/browse/AIRFLOW-256
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Bolke de Bruin
>Assignee: Bolke de Bruin
> Fix For: Airflow 1.8
>
>
> test_scheduler_reschedule can fail due to the heartrate check



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-263) Backtick file introduced by Highcharts refactor

[Arthur Wiedmer (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340552#comment-15340552
 ] 

Arthur Wiedmer commented on AIRFLOW-263:


[~bolke], you are the best to assess if this file is needed, but it looks like 
a temp file to me.

> Backtick file introduced by Highcharts refactor
> ---
>
> Key: AIRFLOW-263
> URL: https://issues.apache.org/jira/browse/AIRFLOW-263
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Arthur Wiedmer
>Assignee: Arthur Wiedmer
>Priority: Minor
>
> A file named "`" was introduced during the Highcharts removal. See
> https://github.com/apache/incubator-airflow/commit/0a460081bc7cba2d05434148f092b87d35aa8cd3
> My best assessment, is that this was a temporary file created by mistake.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AIRFLOW-263) Backtick file introduced by Highcharts refactor

[Arthur Wiedmer (JIRA)]

Arthur Wiedmer created AIRFLOW-263:
--

 Summary: Backtick file introduced by Highcharts refactor
 Key: AIRFLOW-263
 URL: https://issues.apache.org/jira/browse/AIRFLOW-263
 Project: Apache Airflow
  Issue Type: Bug
Reporter: Arthur Wiedmer
Assignee: Arthur Wiedmer
Priority: Minor


A file named "`" was introduced during the Highcharts removal. See
https://github.com/apache/incubator-airflow/commit/0a460081bc7cba2d05434148f092b87d35aa8cd3

My best assessment, is that this was a temporary file created by mistake.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AIRFLOW-262) Simplify commands in MANIFEST.in

[Ajay Yadava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajay Yadava updated AIRFLOW-262:

Priority: Minor  (was: Major)

> Simplify commands in MANIFEST.in
> 
>
> Key: AIRFLOW-262
> URL: https://issues.apache.org/jira/browse/AIRFLOW-262
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
>Priority: Minor
>
> `recursive-include` command with the wild card pattern can be simplified to 
> use graft command instead. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AIRFLOW-262) Simplify commands in MANIFEST.in

[Ajay Yadava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajay Yadava updated AIRFLOW-262:

External issue URL: https://github.com/apache/incubator-airflow/pull/1612/

> Simplify commands in MANIFEST.in
> 
>
> Key: AIRFLOW-262
> URL: https://issues.apache.org/jira/browse/AIRFLOW-262
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
>
> `recursive-include` command with the wild card pattern can be simplified to 
> use graft command instead. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-262) Simplify commands in MANIFEST.in

[Ajay Yadava (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340123#comment-15340123
 ] 

Ajay Yadava commented on AIRFLOW-262:
-

Complete reference for the MANIFEST.in can be found at 
https://docs.python.org/3/distutils/commandref.html

> Simplify commands in MANIFEST.in
> 
>
> Key: AIRFLOW-262
> URL: https://issues.apache.org/jira/browse/AIRFLOW-262
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
>
> `recursive-include` command with the wild card pattern can be simplified to 
> use graft command instead. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AIRFLOW-261) Add more fields for EmailOperator

[Ajay Yadava (JIRA)]

Ajay Yadava created AIRFLOW-261:
---

 Summary: Add more fields for EmailOperator
 Key: AIRFLOW-261
 URL: https://issues.apache.org/jira/browse/AIRFLOW-261
 Project: Apache Airflow
  Issue Type: Improvement
Reporter: Ajay Yadava
Assignee: Ajay Yadava


It will be nice to be able to specify cc, bcc recipients for the EmailOperator. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AIRFLOW-244) Expose task/dag id/run data for ingestion by performance monitoring tools

[Paul Bramsen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Bramsen updated AIRFLOW-244:
-
External issue URL: https://github.com/apache/incubator-airflow/pull/1607  
(was: https://github.com/apache/incubator-airflow/pull/1594)

> Expose task/dag id/run data for ingestion by performance monitoring tools
> -
>
> Key: AIRFLOW-244
> URL: https://issues.apache.org/jira/browse/AIRFLOW-244
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Paul Bramsen
>Assignee: Paul Bramsen
>
> Expose task/dag id/run data for ingestion by performance analysis tools like 
> Dr. Elephant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-245) Access to task instance from custom Executor

[Jeremiah Lowin (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339888#comment-15339888
 ] 

Jeremiah Lowin commented on AIRFLOW-245:


I also posted this on PR 1596 but perhaps it's more relevant here:

Something important to keep in mind -- the current version of execute_async() 
just requires the task key (a bunch of strings) and the command that runs the 
task (another string, something like airflow run task). That's because we make 
no assumptions about how the execution is implemented, or on what 
machine/environment, or even when it will be run. Passing a task object to the 
execute_async() method is contrary to that assumption. In particular, there is 
no guarantee that task objects are serializable.

The key should give enough information to load the task inside the executor by 
querying the database for the matching taskinstance -- I think that would be 
the far preferable way to implement the desired behavior in a custom executor.

> Access to task instance from custom Executor
> 
>
> Key: AIRFLOW-245
> URL: https://issues.apache.org/jira/browse/AIRFLOW-245
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: executor
>Reporter: Alexandr Nikitin
>
> I'm writing a custom executor that executes tasks on mesos and I want to have 
> access to task instances from it. So that I can reuse all existing operators 
> e.g. DockerOperator and access its fields like image, command, volumes and 
> transform them to mesos.
> This can be done by changing `def execute_async(self, key, command, 
> queue=None):` in `BaseExecutor`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AIRFLOW-260) PR tool edge cases with JIRA issues

[Jeremiah Lowin (JIRA)]

Jeremiah Lowin created AIRFLOW-260:
--

 Summary: PR tool edge cases with JIRA issues
 Key: AIRFLOW-260
 URL: https://issues.apache.org/jira/browse/AIRFLOW-260
 Project: Apache Airflow
  Issue Type: Bug
  Components: PR tool
Reporter: Jeremiah Lowin
Assignee: Jeremiah Lowin


Observed two edge cases:
1. If multiple issues are referenced in a PR (AIRFLOW-X and AIRFLOW-Y), and 
AIRFLOW-X is already closed, the PR tool won't close AIRFLOW-Y (but doesn't 
raise an error)

2. An error was raised on an issue that has a fix version set -- it looks like 
the expected format of the fix version doesn't match the format we're using in 
JIRA. Need to look into this further.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (AIRFLOW-257) airflow command fails with "ImportError: No module named zope.deprecation"

[Jeremiah Lowin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Lowin closed AIRFLOW-257.
--
Resolution: Fixed
  Assignee: Jeremiah Lowin

> airflow command fails with "ImportError: No module named zope.deprecation"
> --
>
> Key: AIRFLOW-257
> URL: https://issues.apache.org/jira/browse/AIRFLOW-257
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Kengo Seki
>Assignee: Jeremiah Lowin
>Priority: Critical
>
> After AIRFLOW-31 has been merged, airflow command fails as follows:
> {code}
> $ airflow webserver
> [2016-06-18 00:56:50,367] {__init__.py:36} INFO - Using executor 
> SequentialExecutor
> [2016-06-18 00:56:50,492] {driver.py:120} INFO - Generating grammar tables 
> from /usr/lib/python2.7/lib2to3/Grammar.txt
> [2016-06-18 00:56:50,529] {driver.py:120} INFO - Generating grammar tables 
> from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
> Traceback (most recent call last):
>   File "/home/sekikn/.virtualenvs/e/bin/airflow", line 6, in 
> exec(compile(open(__file__).read(), __file__, 'exec'))
>   File "/home/sekikn/dev/incubator-airflow/airflow/bin/airflow", line 4, in 
> 
> from airflow import configuration
>   File "/home/sekikn/dev/incubator-airflow/airflow/__init__.py", line 76, in 
> 
> from airflow import operators
>   File "/home/sekikn/dev/incubator-airflow/airflow/operators/__init__.py", 
> line 24, in 
> from .check_operator import (
>   File 
> "/home/sekikn/dev/incubator-airflow/airflow/operators/check_operator.py", 
> line 20, in 
> from airflow.hooks import BaseHook
>   File "/home/sekikn/dev/incubator-airflow/airflow/hooks/__init__.py", line 
> 66, in 
> from zope.deprecation import deprecated as _deprecated
> ImportError: No module named zope.deprecation
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-257) airflow command fails with "ImportError: No module named zope.deprecation"

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339871#comment-15339871
 ] 

ASF subversion and git services commented on AIRFLOW-257:
-

Commit 45b735baeac794b54dd89ced2f43eec54adf13f7 in incubator-airflow's branch 
refs/heads/master from jlowin
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=45b735b ]

[AIRFLOW-31] Add zope dependency

Closes #1608 from jlowin/standard-imports-2.
Also closes AIRFLOW-257.


> airflow command fails with "ImportError: No module named zope.deprecation"
> --
>
> Key: AIRFLOW-257
> URL: https://issues.apache.org/jira/browse/AIRFLOW-257
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Kengo Seki
>Priority: Critical
>
> After AIRFLOW-31 has been merged, airflow command fails as follows:
> {code}
> $ airflow webserver
> [2016-06-18 00:56:50,367] {__init__.py:36} INFO - Using executor 
> SequentialExecutor
> [2016-06-18 00:56:50,492] {driver.py:120} INFO - Generating grammar tables 
> from /usr/lib/python2.7/lib2to3/Grammar.txt
> [2016-06-18 00:56:50,529] {driver.py:120} INFO - Generating grammar tables 
> from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
> Traceback (most recent call last):
>   File "/home/sekikn/.virtualenvs/e/bin/airflow", line 6, in 
> exec(compile(open(__file__).read(), __file__, 'exec'))
>   File "/home/sekikn/dev/incubator-airflow/airflow/bin/airflow", line 4, in 
> 
> from airflow import configuration
>   File "/home/sekikn/dev/incubator-airflow/airflow/__init__.py", line 76, in 
> 
> from airflow import operators
>   File "/home/sekikn/dev/incubator-airflow/airflow/operators/__init__.py", 
> line 24, in 
> from .check_operator import (
>   File 
> "/home/sekikn/dev/incubator-airflow/airflow/operators/check_operator.py", 
> line 20, in 
> from airflow.hooks import BaseHook
>   File "/home/sekikn/dev/incubator-airflow/airflow/hooks/__init__.py", line 
> 66, in 
> from zope.deprecation import deprecated as _deprecated
> ImportError: No module named zope.deprecation
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AIRFLOW-31) Use standard imports for hooks/operators

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339870#comment-15339870
 ] 

ASF subversion and git services commented on AIRFLOW-31:


Commit 45b735baeac794b54dd89ced2f43eec54adf13f7 in incubator-airflow's branch 
refs/heads/master from jlowin
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=45b735b ]

[AIRFLOW-31] Add zope dependency

Closes #1608 from jlowin/standard-imports-2.
Also closes AIRFLOW-257.


> Use standard imports for hooks/operators
> 
>
> Key: AIRFLOW-31
> URL: https://issues.apache.org/jira/browse/AIRFLOW-31
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: operators
>Reporter: Jeremiah Lowin
>Assignee: Jeremiah Lowin
>  Labels: enhancement
> Fix For: Airflow 1.8
>
>
> (Migrated from https://github.com/airbnb/airflow/issues/1238)
> Currently, Airflow uses a relatively complex import mechanism to import hooks 
> and operators without polluting the namespace with submodules. I would like 
> to propose that Airflow abandon that system and use standard Python importing.
> Here are a few major reasons why I think the current system has run its 
> course.
> h3. Polluting namespace
> The biggest advantage of the current system, as I understand it, is that only 
> Operators appear in the `airflow.operators` namespace.  The submodules that 
> actually contain the operators do not.
> So for example while `airflow.operators.python_operator.PythonOperator` is a 
> thing, `PythonOperator` is in the `airflow.operators` namespace but 
> `python_operator` is not.
> I think this sort of namespace pollution was helpful when Airflow was a 
> smaller project, but as the number of hooks/operators grows -- and especially 
> as the `contrib` hooks/operators grow -- I'd argue that namespacing is a 
> *good thing*. It provides structure and organization, and opportunities for 
> documentation (through module docstrings).
> In fact, I'd argue that the current namespace is itself getting quite 
> polluted -- the only way to know what's available is to use something like 
> Ipython tab-completion to browse an alphabetical list of Operator names, or 
> to load the source file and grok the import definition (which no one 
> installing from pypi is likely to do).
> h3. Conditional imports
> There's a second advantage to the current system that any module that fails 
> to import is silently ignored. It makes it easy to have optional 
> dependencies. For example, if someone doesn't have `boto` installed, then 
> they don't have an `S3Hook` either. Same for a HiveOperator
> Again, as Airflow grows and matures, I think this is a little too magic. If 
> my environment is missing a dependency, I want to hear about it.
> On the other hand, the `contrib` namespace sort of depends on this -- we 
> don't want users to have to install every single dependency. So I propose 
> that contrib modules all live in their submodules: `from 
> airflow.contrib.operators.my_operator import MyOperator`. As mentioned 
> previously, having structure and namespacing is a good thing as the project 
> gets more complex.
> Other ways to handle this include putting "non-standard" dependencies inside 
> the operator/hook rather than the module (see `HiveOperator`/`HiveHook`), so 
> it can be imported but not used. Another is judicious use of `try`/`except 
> ImportError`. The simplest is to make people import things explicitly from 
> submodules.
> h3. Operator dependencies
> Right now, operators can't depend on each other if they aren't in the same 
> file. This is for the simple reason that there is no guarantee on what order 
> the operators will be loaded. It all comes down to which dictionary key gets 
> loaded first. One day Operator B could be loaded after Operator A; the next 
> day it might be loaded before. Consequently, A and B can't depend on each 
> other. Worse, if a user makes two operators that do depend on each other, 
> they won't get an error message when one fails to import.
> For contrib modules in particular, this is sort of killer.
> h3. Ease of use
> It's *hard* to set up imports for a new operator. The dictionary-based import 
> instructions aren't obvious for new users, and errors are silently dismissed 
> which makes debugging difficult.
> h3. Identity
> Surprisingly, `airflow.operators.SubDagOperator != 
> airflow.operators.subdag_operator.SubDagOperator`. See #1168.
> h2. Proposal
> Use standard python importing for hooks/operators/etc.
> - `__init__.py` files use straightforward, standard Python imports
> - major operators are available at `airflow.operators.OperatorName` or 
> `airflow.operators.operator_module.OperatorName`.
> - contrib operators are only available at 
> 

incubator-airflow git commit: [AIRFLOW-31] Add zope dependency

[jlowin]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master 004272b15 -> 45b735bae


[AIRFLOW-31] Add zope dependency

Closes #1608 from jlowin/standard-imports-2.
Also closes AIRFLOW-257.


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/45b735ba
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/45b735ba
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/45b735ba

Branch: refs/heads/master
Commit: 45b735baeac794b54dd89ced2f43eec54adf13f7
Parents: 004272b
Author: jlowin 
Authored: Mon Jun 20 12:40:07 2016 -0400
Committer: jlowin 
Committed: Mon Jun 20 12:40:27 2016 -0400

--
 setup.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/45b735ba/setup.py
--
diff --git a/setup.py b/setup.py
index 978c484..3fc77ff 100644
--- a/setup.py
+++ b/setup.py
@@ -199,6 +199,7 @@ def do_setup():
 'setproctitle>=1.1.8, <2',
 'sqlalchemy>=0.9.8',
 'thrift>=0.9.2, <0.10',
+'zope.deprecation>=4.0, <5.0',
 ],
 extras_require={
 'all': devel_all,
@@ -257,4 +258,3 @@ def do_setup():
 
 if __name__ == "__main__":
 do_setup()
-

[jira] [Updated] (AIRFLOW-98) Using Flask extensions from a plugin

[Timo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timo updated AIRFLOW-98:

Priority: Major  (was: Minor)

> Using Flask extensions from a plugin
> 
>
> Key: AIRFLOW-98
> URL: https://issues.apache.org/jira/browse/AIRFLOW-98
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Timo
>
> I am creating a plugin which should be able to use some Flask extensions. Is 
> there any way to do this?
> Basically I need to import the airflow.www.app.app object in the plugin to 
> initialise the extension with.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AIRFLOW-98) Using Flask extensions from a plugin

[Timo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timo updated AIRFLOW-98:

Issue Type: Bug  (was: Wish)

> Using Flask extensions from a plugin
> 
>
> Key: AIRFLOW-98
> URL: https://issues.apache.org/jira/browse/AIRFLOW-98
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Timo
>Priority: Minor
>
> I am creating a plugin which should be able to use some Flask extensions. Is 
> there any way to do this?
> Basically I need to import the airflow.www.app.app object in the plugin to 
> initialise the extension with.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/static/gantt-chart-d3v2.js
--
diff --git a/airflow/www/static/gantt-chart-d3v2.js 
b/airflow/www/static/gantt-chart-d3v2.js
new file mode 100644
index 000..b8924f9
--- /dev/null
+++ b/airflow/www/static/gantt-chart-d3v2.js
@@ -0,0 +1,247 @@
+/**
+ * @author Dimitry Kudrayvtsev
+ * @version 2.1
+ * @modifiedby Maxime Beauchemin
+ */
+
+d3.gantt = function() {
+  var FIT_TIME_DOMAIN_MODE = "fit";
+  var FIXED_TIME_DOMAIN_MODE = "fixed";
+  var tip = d3.tip()
+.attr('class', 'd3-tip')
+.offset([-10, 0])
+.html(function(d) {
+  var s = ""
+  s += "";
+  s += "start:end:duration:"
+  s += " "
+  s += d.isoStart + "";
+  s += d.isoEnd + "";
+  s += d.duration + "";
+  s += "";
+  s += "";
+  return s;
+  })
+
+  var margin = {
+top : 20,
+right : 40,
+bottom : 20,
+left : 150
+  };
+  var selector = 'body';
+  var timeDomainStart = d3.time.day.offset(new Date(),-3);
+  var timeDomainEnd = d3.time.hour.offset(new Date(),+3);
+  var timeDomainMode = FIT_TIME_DOMAIN_MODE;// fixed or fit
+  var taskTypes = [];
+  var taskStatus = [];
+  var height = document.body.clientHeight - margin.top - margin.bottom-5;
+  var width = $('.gantt').width() - margin.right - margin.left-5;
+
+  var tickFormat = "%H:%M";
+
+  var keyFunction = function(d) {
+return d.startDate + d.taskName + d.endDate;
+  };
+
+  var rectTransform = function(d) {
+return "translate(" + x(d.startDate) + "," + y(d.taskName) + ")";
+  };
+
+  var x = d3.time.scale().domain([ timeDomainStart, timeDomainEnd ]).range([ 
0, width ]).clamp(true);
+
+  var y = d3.scale.ordinal().domain(taskTypes).rangeRoundBands([ 0, height - 
margin.top - margin.bottom ], .1);
+
+  var xAxis = 
d3.svg.axis().scale(x).orient("bottom").tickFormat(d3.time.format(tickFormat)).tickSubdivide(true)
+  .tickSize(8).tickPadding(8);
+
+  var yAxis = d3.svg.axis().scale(y).orient("left").tickSize(0);
+
+  var initTimeDomain = function(tasks) {
+if (timeDomainMode === FIT_TIME_DOMAIN_MODE) {
+  if (tasks === undefined || tasks.length < 1) {
+timeDomainStart = d3.time.day.offset(new Date(), -3);
+timeDomainEnd = d3.time.hour.offset(new Date(), +3);
+return;
+  }
+  tasks.sort(function(a, b) {
+return a.endDate - b.endDate;
+  });
+  timeDomainEnd = tasks[tasks.length - 1].endDate;
+  tasks.sort(function(a, b) {
+return a.startDate - b.startDate;
+  });
+  timeDomainStart = tasks[0].startDate;
+}
+  };
+
+  var initAxis = function() {
+x = d3.time.scale().domain([ timeDomainStart, timeDomainEnd ]).range([ 0, 
width ]).clamp(true);
+y = d3.scale.ordinal().domain(taskTypes).rangeRoundBands([ 0, height - 
margin.top - margin.bottom ], .1);
+xAxis = 
d3.svg.axis().scale(x).orient("bottom").tickFormat(d3.time.format(tickFormat)).tickSubdivide(true)
+.tickSize(8).tickPadding(8);
+
+yAxis = d3.svg.axis().scale(y).orient("left").tickSize(0);
+  };
+
+  function gantt(tasks) {
+
+initTimeDomain(tasks);
+initAxis();
+
+var svg = d3.select(selector)
+.append("svg")
+.attr("class", "chart")
+.attr("width", width + margin.left + margin.right)
+.attr("height", height + margin.top + margin.bottom)
+.append("g")
+.attr("class", "gantt-chart")
+.attr("width", width + margin.left + margin.right)
+.attr("height", height + margin.top + margin.bottom)
+.attr("transform", "translate(" + margin.left + ", " + margin.top + ")");
+
+svg.selectAll(".chart")
+.data(tasks, keyFunction).enter()
+.append("rect")
+.on('mouseover', tip.show)
+.on('mouseout', tip.hide)
+.on('click', function(d) {
+  call_modal(d.taskName, d.executionDate);
+})
+.attr("class", function(d){
+  if(taskStatus[d.status] == null){ return "bar";}
+  return taskStatus[d.status];
+})
+.attr("y", 0)
+.attr("transform", rectTransform)
+.attr("height", function(d) { return y.rangeBand(); })
+.attr("width", function(d) {
+  return d3.max([x(d.endDate) - x(d.startDate), 1]);
+});
+
+
+svg.append("g")
+.attr("class", "x axis")
+.attr("transform", "translate(0, " + (height - margin.top - margin.bottom) 
+ ")")
+.transition()
+.call(xAxis);
+
+svg.append("g").attr("class", "y axis").transition().call(yAxis);
+svg.call(tip);
+
+return gantt;
+
+  };
+
+  gantt.redraw = function(tasks) {
+
+initTimeDomain(tasks);
+initAxis();
+
+var svg = d3.select(".chart");
+
+var ganttChartGroup = svg.select(".gantt-chart");
+var rect = ganttChartGroup.selectAll("rect").data(tasks, keyFunction);
+
+rect.enter()
+.insert("rect",":first-child")
+.attr("rx", 5)
+.attr("ry", 5)
+.attr("class", function(d){
+  if(taskStatus[d.status] == null){ return "bar";}

[2/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/static/nv.d3.js
--
diff --git a/airflow/www/static/nv.d3.js b/airflow/www/static/nv.d3.js
new file mode 100644
index 000..f00492a
--- /dev/null
+++ b/airflow/www/static/nv.d3.js
@@ -0,0 +1,14241 @@
+(function(){
+
+var nv = window.nv || {};
+
+
+nv.version = '1.1.11b';
+nv.dev = true //set false when in production
+
+window.nv = nv;
+
+nv.tooltip = {}; // For the tooltip system
+nv.utils = {}; // Utility subsystem
+nv.models = {}; //stores all the possible models/components
+nv.charts = {}; //stores all the ready to use charts
+nv.graphs = []; //stores all the graphs currently on the page
+nv.logs = {}; //stores some statistics and potential error messages
+
+nv.dispatch = d3.dispatch('render_start', 'render_end');
+
+// *
+//  Development render timers - disabled if dev = false
+
+if (nv.dev) {
+  nv.dispatch.on('render_start', function(e) {
+nv.logs.startTime = +new Date();
+  });
+
+  nv.dispatch.on('render_end', function(e) {
+nv.logs.endTime = +new Date();
+nv.logs.totalTime = nv.logs.endTime - nv.logs.startTime;
+nv.log('total', nv.logs.totalTime); // used for development, to keep track 
of graph generation times
+  });
+}
+
+// 
+//  Public Core NV functions
+
+// Logs all arguments, and returns the last so you can test things in place
+// Note: in IE8 console.log is an object not a function, and if modernizr is 
used
+// then calling Function.prototype.bind with with anything other than a 
function
+// causes a TypeError to be thrown.
+nv.log = function() {
+  if (nv.dev && console.log && console.log.apply)
+console.log.apply(console, arguments)
+  else if (nv.dev && typeof console.log == "function" && 
Function.prototype.bind) {
+var log = Function.prototype.bind.call(console.log, console);
+log.apply(console, arguments);
+  }
+  return arguments[arguments.length - 1];
+};
+
+
+nv.render = function render(step) {
+  step = step || 1; // number of graphs to generate in each timeout loop
+
+  nv.render.active = true;
+  nv.dispatch.render_start();
+
+  setTimeout(function() {
+var chart, graph;
+
+for (var i = 0; i < step && (graph = nv.render.queue[i]); i++) {
+  chart = graph.generate();
+  if (typeof graph.callback == typeof(Function)) graph.callback(chart);
+  nv.graphs.push(chart);
+}
+
+nv.render.queue.splice(0, i);
+
+if (nv.render.queue.length) setTimeout(arguments.callee, 0);
+else { nv.render.active = false; nv.dispatch.render_end(); }
+  }, 0);
+};
+
+nv.render.active = false;
+nv.render.queue = [];
+
+nv.addGraph = function(obj) {
+  if (typeof arguments[0] === typeof(Function))
+obj = {generate: arguments[0], callback: arguments[1]};
+
+  nv.render.queue.push(obj);
+
+  if (!nv.render.active) nv.render();
+};
+
+nv.identity = function(d) { return d; };
+
+nv.strip = function(s) { return s.replace(/(\s|&)/g,''); };
+
+function daysInMonth(month,year) {
+  return (new Date(year, month+1, 0)).getDate();
+}
+
+function d3_time_range(floor, step, number) {
+  return function(t0, t1, dt) {
+var time = floor(t0), times = [];
+if (time < t0) step(time);
+if (dt > 1) {
+  while (time < t1) {
+var date = new Date(+time);
+if ((number(date) % dt === 0)) times.push(date);
+step(time);
+  }
+} else {
+  while (time < t1) { times.push(new Date(+time)); step(time); }
+}
+return times;
+  };
+}
+
+d3.time.monthEnd = function(date) {
+  return new Date(date.getFullYear(), date.getMonth(), 0);
+};
+
+d3.time.monthEnds = d3_time_range(d3.time.monthEnd, function(date) {
+date.setUTCDate(date.getUTCDate() + 1);
+date.setDate(daysInMonth(date.getMonth() + 1, date.getFullYear()));
+  }, function(date) {
+return date.getMonth();
+  }
+);
+
+/* Utility class to handle creation of an interactive layer.
+This places a rectangle on top of the chart. When you mouse move over it, it 
sends a dispatch
+containing the X-coordinate. It can also render a vertical line where the 
mouse is located.
+
+dispatch.elementMousemove is the important event to latch onto.  It is fired 
whenever the mouse moves over
+the rectangle. The dispatch is given one object which contains the mouseX/Y 
location.
+It also has 'pointXValue', which is the conversion of mouseX to the x-axis 
scale.
+*/
+nv.interactiveGuideline = function() {
+   "use strict";
+   var tooltip = nv.models.tooltip();
+   //Public settings
+   var width = null
+   , height = null
+//Please pass in the bounding chart's top and left margins
+//This is important for calculating the correct mouseX/Y positions.
+   , margin = {left: 0, top: 0}
+   , xScale = d3.scale.linear()
+   , yScale = d3.scale.linear()
+   , dispatch 

[8/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

[AIRFLOW-6] Remove dependency on Highcharts

Highcharts' license is not compatible with the Apache 2.0
license. This patch removes Highcharts in favor of d3,
however some charts are not supported anymore.

* This brings Maxime Beauchemin's work to master


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/0a460081
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/0a460081
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/0a460081

Branch: refs/heads/master
Commit: 0a460081bc7cba2d05434148f092b87d35aa8cd3
Parents: d243c00
Author: Bolke de Bruin 
Authored: Mon Jun 20 14:19:34 2016 +0200
Committer: Bolke de Bruin 
Committed: Mon Jun 20 14:53:30 2016 +0200

--
 `|  2347 
 airflow/bin/airflow  |13 +
 airflow/executors/base_executor.py   |14 +
 airflow/executors/celery_executor.py |14 +
 airflow/hooks/__init__.py| 2 +-
 airflow/hooks/dbapi_hook.py  | 1 -
 airflow/hooks/jdbc_hook.py   | 3 +-
 airflow/hooks/oracle_hook.py | 1 +
 airflow/www/app.py   | 4 +-
 airflow/www/blueprints.py| 6 -
 airflow/www/static/d3.tip.v0.6.3.js  |   280 +
 airflow/www/static/d3.v3.min.js  |10 +-
 airflow/www/static/gantt-chart-d3v2.js   |   247 +
 airflow/www/static/gantt.css |38 +
 airflow/www/static/highcharts-more.js|53 -
 airflow/www/static/highcharts.js |   308 -
 airflow/www/static/nv.d3.css |   769 ++
 airflow/www/static/nv.d3.js  | 14241 
 airflow/www/static/nvd3.tar.gz   |   Bin 0 -> 328377 bytes
 airflow/www/templates/airflow/chart.html |37 +-
 airflow/www/templates/airflow/dag.html   | 4 +-
 airflow/www/templates/airflow/gantt.html |86 +-
 airflow/www/templates/airflow/highchart.html |   183 -
 airflow/www/templates/airflow/nvd3.html  |   175 +
 airflow/www/utils.py |11 +-
 airflow/www/views.py |   352 +-
 setup.py |23 +-
 tests/core.py|19 -
 28 files changed, 18315 insertions(+), 926 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/`
--
diff --git a/` b/`
new file mode 100644
index 000..6331805
--- /dev/null
+++ b/`
@@ -0,0 +1,2347 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import sys
+
+import os
+import socket
+import importlib
+
+from functools import wraps
+from datetime import datetime, timedelta
+import dateutil.parser
+import copy
+from itertools import chain, product
+
+from past.utils import old_div
+from past.builtins import basestring
+
+import inspect
+import traceback
+
+import sqlalchemy as sqla
+from sqlalchemy import or_, desc, and_
+
+
+from flask import redirect, url_for, request, Markup, Response, current_app, 
render_template
+from flask_admin import BaseView, expose, AdminIndexView
+from flask_admin.contrib.sqla import ModelView
+from flask_admin.actions import action
+from flask_login import flash
+from flask._compat import PY2
+
+import jinja2
+import markdown
+import json
+
+from wtforms import (
+Form, SelectField, TextAreaField, PasswordField, StringField)
+
+from pygments import highlight, lexers
+from pygments.formatters import HtmlFormatter
+
+import airflow
+from airflow import configuration as conf
+from airflow import models
+from airflow import settings
+from airflow.exceptions import AirflowException
+from airflow.settings import Session
+from airflow.models import XCom
+
+from airflow.utils.json import json_ser
+from airflow.utils.state import State
+from airflow.utils.db import provide_session
+from airflow.utils.helpers import alchemy_to_dict
+from airflow.utils import logging as log_utils
+from airflow.www import utils as wwwutils
+from airflow.www.forms import DateTimeForm, DateTimeWithNumRunsForm
+
+QUERY_LIMIT = 10

[jira] [Commented] (AIRFLOW-6) Remove high-charts

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339493#comment-15339493
 ] 

ASF subversion and git services commented on AIRFLOW-6:
---

Commit 0a460081bc7cba2d05434148f092b87d35aa8cd3 in incubator-airflow's branch 
refs/heads/master from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=0a46008 ]

[AIRFLOW-6] Remove dependency on Highcharts

Highcharts' license is not compatible with the Apache 2.0
license. This patch removes Highcharts in favor of d3,
however some charts are not supported anymore.

* This brings Maxime Beauchemin's work to master


> Remove high-charts
> --
>
> Key: AIRFLOW-6
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Chris Riccomini
>Assignee: Maxime Beauchemin
>
> According to LEGAL-246, we can't use High Charts in Airflow. The license is 
> incompatible with Apache. We should replace it with another library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[7/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/hooks/__init__.py
--
diff --git a/airflow/hooks/__init__.py b/airflow/hooks/__init__.py
index 10fb5a7..5209a51 100644
--- a/airflow/hooks/__init__.py
+++ b/airflow/hooks/__init__.py
@@ -12,7 +12,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-
 # Only import Core Airflow Operators that don't have extra requirements.
 # All other operators must be imported directly.
 from .base_hook import BaseHook
@@ -34,6 +33,7 @@ from .sqlite_hook import SqliteHook
 # Imports the hooks dynamically while keeping the package API clean,
 # abstracting the underlying modules
 
+
 from airflow.utils.helpers import import_module_attrs as _import_module_attrs
 from airflow.hooks.base_hook import BaseHook  # noqa to expose in package
 

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/hooks/dbapi_hook.py
--
diff --git a/airflow/hooks/dbapi_hook.py b/airflow/hooks/dbapi_hook.py
index e48e138..f6e86b6 100644
--- a/airflow/hooks/dbapi_hook.py
+++ b/airflow/hooks/dbapi_hook.py
@@ -12,7 +12,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-
 from builtins import str
 from past.builtins import basestring
 from datetime import datetime

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/hooks/jdbc_hook.py
--
diff --git a/airflow/hooks/jdbc_hook.py b/airflow/hooks/jdbc_hook.py
index 510f386..d0091e1 100644
--- a/airflow/hooks/jdbc_hook.py
+++ b/airflow/hooks/jdbc_hook.py
@@ -13,13 +13,12 @@
 # limitations under the License.
 
 from builtins import str
-__author__ = 'janomar'
 
-import logging
 import jaydebeapi
 
 from airflow.hooks.dbapi_hook import DbApiHook
 
+
 class JdbcHook(DbApiHook):
 """
 General hook for jdbc db access.

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/hooks/oracle_hook.py
--
diff --git a/airflow/hooks/oracle_hook.py b/airflow/hooks/oracle_hook.py
index aa70a56..3ff50bf 100644
--- a/airflow/hooks/oracle_hook.py
+++ b/airflow/hooks/oracle_hook.py
@@ -21,6 +21,7 @@ from datetime import datetime
 import numpy
 import logging
 
+
 class OracleHook(DbApiHook):
 """
 Interact with Oracle SQL.

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/app.py
--
diff --git a/airflow/www/app.py b/airflow/www/app.py
index 5f772a9..a947f37 100644
--- a/airflow/www/app.py
+++ b/airflow/www/app.py
@@ -23,7 +23,7 @@ import airflow
 from airflow import models
 from airflow.settings import Session
 
-from airflow.www.blueprints import ck, routes
+from airflow.www.blueprints import routes
 from airflow import jobs
 from airflow import settings
 from airflow import configuration
@@ -45,9 +45,7 @@ def create_app(config=None):
 cache = Cache(
 app=app, config={'CACHE_TYPE': 'filesystem', 'CACHE_DIR': '/tmp'})
 
-app.register_blueprint(ck, url_prefix='/ck')
 app.register_blueprint(routes)
-app.jinja_env.add_extension("chartkick.ext.charts")
 
 with app.app_context():
 from airflow.www import views

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/blueprints.py
--
diff --git a/airflow/www/blueprints.py b/airflow/www/blueprints.py
index fb54306..88f9628 100644
--- a/airflow/www/blueprints.py
+++ b/airflow/www/blueprints.py
@@ -15,14 +15,8 @@
 from flask import (
 url_for, Markup, Blueprint, redirect,
 )
-import chartkick
 import markdown
 
-# Init for chartkick, the python wrapper for highcharts
-ck = Blueprint(
-'ck_page', __name__,
-static_folder=chartkick.js(), static_url_path='/static')
-
 routes = Blueprint('routes', __name__)
 
 

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/static/d3.tip.v0.6.3.js
--
diff --git a/airflow/www/static/d3.tip.v0.6.3.js 
b/airflow/www/static/d3.tip.v0.6.3.js
new file mode 100644
index 000..6cfa3cb
--- /dev/null
+++ b/airflow/www/static/d3.tip.v0.6.3.js
@@ -0,0 +1,280 @@
+// d3.tip
+// Copyright (c) 2013 Justin Palmer
+//
+// Tooltips for d3.js SVG visualizations
+
+// Public - contructs a new tooltip
+//
+// Returns a tip
+d3.tip = function() {
+  var direction = d3_tip_direction,
+  offset= d3_tip_offset,
+  html  = d3_tip_html,
+  node  = initNode(),
+  svg   = null,
+  point = null,
+  target= null
+
+  function tip(vis) {
+

[6/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/static/d3.v3.min.js
--
diff --git a/airflow/www/static/d3.v3.min.js b/airflow/www/static/d3.v3.min.js
index 88550ae..77615c0 100644
--- a/airflow/www/static/d3.v3.min.js
+++ b/airflow/www/static/d3.v3.min.js
@@ -1,5 +1,5 @@
-!function(){function n(n,t){return t>n?-1:n>t?1:n>=t?0:0/0}function 
t(n){return null!=n&&!isNaN(n)}function 
e(n){return{left:function(t,e,r,u){for(arguments.length<3&&(r=0),arguments.length<4&&(u=t.length);u>r;){var
 i=r+u>>>1;n(t[i],e)<0?r=i+1:u=i}return 
r},right:function(t,e,r,u){for(arguments.length<3&&(r=0),arguments.length<4&&(u=t.length);u>r;){var
 i=r+u>>>1;n(t[i],e)>0?u=i:r=i+1}return r}}}function r(n){return 
n.length}function u(n){for(var t=1;n*t%1;)t*=10;return t}function 
i(n,t){try{for(var e in 
t)Object.defineProperty(n.prototype,e,{value:t[e],enumerable:!1})}catch(r){n.prototype=t}}function
 o(){}function a(n){return ia+n in this}function c(n){return n=ia+n,n in 
this& this[n]}function s(){var n=[];return 
this.forEach(function(t){n.push(t)}),n}function l(){var n=0;for(var t in 
this)t.charCodeAt(0)===oa&&++n;return n}function f(){for(var n in 
this)if(n.charCodeAt(0)===oa)return!1;return!0}function h(){}function 
g(n,t,e){return function(){var r=e.apply(t,arguments);retu
 rn r===t?n:r}}function p(n,t){if(t in n)return 
t;t=t.charAt(0).toUpperCase()+t.substring(1);for(var 
e=0,r=aa.length;r>e;++e){var u=aa[e]+t;if(u in n)return u}}function 
v(){}function d(){}function m(n){function t(){for(var 
t,r=e,u=-1,i=r.length;++ue;e++)for(var 
u,i=n[e],o=0,a=i.length;a>o;o++)(u=i[o])&(u,o,e);return n}function 
U(n){return sa(n,da),n}function j(n){var t,e;return function(r,u,i){var 
o,a=n[i].update,c=a.length;for(i!=e&&(e=i,t=0),u>=t&&(t=u+1);!(o=a[t])&&++t

[3/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/static/nv.d3.css
--
diff --git a/airflow/www/static/nv.d3.css b/airflow/www/static/nv.d3.css
new file mode 100644
index 000..cae8348
--- /dev/null
+++ b/airflow/www/static/nv.d3.css
@@ -0,0 +1,769 @@
+
+/
+ * HTML CSS
+ */
+
+
+.chartWrap {
+  margin: 0;
+  padding: 0;
+  overflow: hidden;
+}
+
+/
+  Box shadow and border radius styling
+*/
+.nvtooltip.with-3d-shadow, .with-3d-shadow .nvtooltip {
+  -moz-box-shadow: 0 5px 10px rgba(0,0,0,.2);
+  -webkit-box-shadow: 0 5px 10px rgba(0,0,0,.2);
+  box-shadow: 0 5px 10px rgba(0,0,0,.2);
+
+  -webkit-border-radius: 6px;
+  -moz-border-radius: 6px;
+  border-radius: 6px;
+}
+
+/
+ * TOOLTIP CSS
+ */
+
+.nvtooltip {
+  position: absolute;
+  background-color: rgba(255,255,255,1.0);
+  padding: 1px;
+  border: 1px solid rgba(0,0,0,.2);
+  z-index: 1;
+
+  font-family: Arial;
+  font-size: 13px;
+  text-align: left;
+  pointer-events: none;
+
+  white-space: nowrap;
+
+  -webkit-touch-callout: none;
+  -webkit-user-select: none;
+  -khtml-user-select: none;
+  -moz-user-select: none;
+  -ms-user-select: none;
+  user-select: none;
+}
+
+/*Give tooltips that old fade in transition by
+putting a "with-transitions" class on the container div.
+*/
+.nvtooltip.with-transitions, .with-transitions .nvtooltip {
+  transition: opacity 250ms linear;
+  -moz-transition: opacity 250ms linear;
+  -webkit-transition: opacity 250ms linear;
+
+  transition-delay: 250ms;
+  -moz-transition-delay: 250ms;
+  -webkit-transition-delay: 250ms;
+}
+
+.nvtooltip.x-nvtooltip,
+.nvtooltip.y-nvtooltip {
+  padding: 8px;
+}
+
+.nvtooltip h3 {
+  margin: 0;
+  padding: 4px 14px;
+  line-height: 18px;
+  font-weight: normal;
+  background-color: rgba(247,247,247,0.75);
+  text-align: center;
+
+  border-bottom: 1px solid #ebebeb;
+
+  -webkit-border-radius: 5px 5px 0 0;
+  -moz-border-radius: 5px 5px 0 0;
+  border-radius: 5px 5px 0 0;
+}
+
+.nvtooltip p {
+  margin: 0;
+  padding: 5px 14px;
+  text-align: center;
+}
+
+.nvtooltip span {
+  display: inline-block;
+  margin: 2px 0;
+}
+
+.nvtooltip table {
+  margin: 6px;
+  border-spacing:0;
+}
+
+
+.nvtooltip table td {
+  padding: 2px 9px 2px 0;
+  vertical-align: middle;
+}
+
+.nvtooltip table td.key {
+  font-weight:normal;
+}
+.nvtooltip table td.value {
+  text-align: right;
+  font-weight: bold;
+}
+
+.nvtooltip table tr.highlight td {
+  padding: 1px 9px 1px 0;
+  border-bottom-style: solid;
+  border-bottom-width: 1px;
+  border-top-style: solid;
+  border-top-width: 1px;
+}
+
+.nvtooltip table td.legend-color-guide div {
+  width: 8px;
+  height: 8px;
+  vertical-align: middle;
+}
+
+.nvtooltip .footer {
+  padding: 3px;
+  text-align: center;
+}
+
+
+.nvtooltip-pending-removal {
+  position: absolute;
+  pointer-events: none;
+}
+
+
+/
+ * SVG CSS
+ */
+
+
+svg {
+  -webkit-touch-callout: none;
+  -webkit-user-select: none;
+  -khtml-user-select: none;
+  -moz-user-select: none;
+  -ms-user-select: none;
+  user-select: none;
+  /* Trying to get SVG to act like a greedy block in all browsers */
+  display: block;
+  width:100%;
+  height:100%;
+}
+
+
+svg text {
+  font: normal 12px Arial;
+}
+
+svg .title {
+ font: bold 14px Arial;
+}
+
+.nvd3 .nv-background {
+  fill: white;
+  fill-opacity: 0;
+  /*
+  pointer-events: none;
+  */
+}
+
+.nvd3.nv-noData {
+  font-size: 18px;
+  font-weight: bold;
+}
+
+
+/**
+*  Brush
+*/
+
+.nv-brush .extent {
+  fill-opacity: .125;
+  shape-rendering: crispEdges;
+}
+
+
+
+/**
+*  Legend
+*/
+
+.nvd3 .nv-legend .nv-series {
+  cursor: pointer;
+}
+
+.nvd3 .nv-legend .disabled circle {
+  fill-opacity: 0;
+}
+
+
+
+/**
+*  Axes
+*/
+.nvd3 .nv-axis {
+  pointer-events:none;
+}
+
+.nvd3 .nv-axis path {
+  fill: none;
+  stroke: #000;
+  stroke-opacity: .75;
+  shape-rendering: crispEdges;
+}
+
+.nvd3 .nv-axis path.domain {
+  stroke-opacity: .75;
+}
+
+.nvd3 .nv-axis.nv-x path.domain {
+  stroke-opacity: 0;
+}
+
+.nvd3 .nv-axis line {
+  fill: none;
+  stroke: #e5e5e5;
+  shape-rendering: crispEdges;
+}
+
+.nvd3 .nv-axis .zero line,
+/*this selector may not be necessary*/ .nvd3 .nv-axis line.zero {
+  stroke-opacity: .75;
+}
+
+.nvd3 .nv-axis .nv-axisMaxMin text {
+  font-weight: bold;
+}
+
+.nvd3 .x  .nv-axis .nv-axisMaxMin text,
+.nvd3 .x2 .nv-axis .nv-axisMaxMin text,
+.nvd3 .x3 .nv-axis .nv-axisMaxMin text {
+  text-anchor: middle
+}
+
+
+
+/**
+*  Brush
+*/
+
+.nv-brush .resize path {
+  fill: #eee;
+  stroke: #666;
+}
+
+
+
+/**
+*  Bars
+*/
+
+.nvd3 .nv-bars .negative rect {
+zfill: brown;
+}
+
+.nvd3 .nv-bars rect {
+  zfill: steelblue;
+  fill-opacity: .75;
+
+  transition: fill-opacity 250ms linear;
+  -moz-transition: fill-opacity 250ms linear;
+  -webkit-transition: fill-opacity 250ms linear;
+}
+
+.nvd3 

[1/9] incubator-airflow git commit: [AIRFLOW-6] Remove dependency on Highcharts

[bolke]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master 7c0f8373f -> 004272b15


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/templates/airflow/chart.html
--
diff --git a/airflow/www/templates/airflow/chart.html 
b/airflow/www/templates/airflow/chart.html
index 1901af5..91b8be1 100644
--- a/airflow/www/templates/airflow/chart.html
+++ b/airflow/www/templates/airflow/chart.html
@@ -21,54 +21,21 @@
 
 
 
-
+{{ chart |safe }}
 
 {% endblock %}
 
 {% block tail %}
 {{ super() }}
-
-
-
 
   // We blur task_ids in demo mode
 $( document ).ready(function() {
-  if ("{{ demo_mode }}" == "True") {
-  d3.select("svg")
-.append("filter")
-.attr("id", "blur-effect-1")
-.append("feGaussianBlur")
-.attr("stdDeviation", 3);
-  d3.selectAll("g.highcharts-legend-item text").style("filter", 
"url(#blur-effect-1)");
-  }
-  $('#uncheck').click(function(){
-  chart = Highcharts.charts[0];
-  $(chart.series).each(function(){
-  this.setVisible(false, false);
-  });
-  chart.redraw();
-  });
-  $('#check').click(function(){
-  chart = Highcharts.charts[0];
-  $(chart.series).each(function(){
-  this.setVisible(true, false);
-  });
-  chart.redraw();
-  });
 });
 
 
-
-  {% line_chart data with height=height library=chart_options%}
-  
-Hide all series
-Show all series
-  
-
+
 
-
   
-  
 
 {% endblock %}

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/0a460081/airflow/www/templates/airflow/dag.html
--
diff --git a/airflow/www/templates/airflow/dag.html 
b/airflow/www/templates/airflow/dag.html
index 22b312e..89ce2d6 100644
--- a/airflow/www/templates/airflow/dag.html
+++ b/airflow/www/templates/airflow/dag.html
@@ -4,8 +4,8 @@
 {% block title %}Airflow - DAG {{ dag.dag_id }}{% endblock %}
 
 {% block head_css %}
-  {{ super() }}
   {{ lib.form_css() }}
+  {{ super() }}
 {% endblock %}
 
 {% block body %}
@@ -186,8 +186,8 @@