And one more - on Python3 we can't use S3Hook.load_string due to bytes vs string issue: https://issues.apache.org/jira/browse/AIRFLOW-1796 <https://issues.apache.org/jira/browse/AIRFLOW-1796>
I'll try and work on fixes for some/all of these today, including adding and expanding on the tests for S3Hook which it looks like was kind of lacking. -ash > On 9 Nov 2017, at 10:54, Ash Berlin-Taylor <[email protected]> > wrote: > > Thanks for picking this up. Your fix should stop the 500 error, but there's > another problem (which is ultimately user misconfiguration about) > https://issues.apache.org/jira/browse/AIRFLOW-1796 - the fix for that to > update a doc somewhere, and probably validate this settings is correct at > start time. > > > I've found another issue related to arg names of S3Hook. In 1.8.2 it was > `s3_conn_id` but the move to boto3/basing off AWSHook now expects > `aws_conn_id`, and various places in Airflow code base (and a few places in > our dags/operators code base) still pass it as s3_conn_id. I've created > https://issues.apache.org/jira/browse/AIRFLOW-1795 for that issue. > > -ash > > >> On 8 Nov 2017, at 18:54, Daniel Huang <[email protected]> wrote: >> >> Still testing this out. >> >> Put up a small fix for Ash's second exception >> https://github.com/apache/incubator-airflow/pull/2766 >> >> On Wed, Nov 8, 2017 at 10:48 AM, Bolke de Bruin <[email protected]> wrote: >> >>> Hi Chris, >>> >>> Actively testing here: we found an issue in the SSHOperator introduced in >>> 1.9.0 (fix already merged for RC2, but blocking I as it stops us from >>> running SSH properly), some minor fixes by Airbnb should also be in RC2. >>> There is some logging “weirdness”, that might warrant a small patch here in >>> there and could be squeezed into RC2, but I don’t consider them blocking. >>> >>> So almost there, but we need an RC2 imho. >>> >>> -1, binding. >>> >>> Bolke >>> >>>> On 8 Nov 2017, at 19:00, Ash Berlin-Taylor <ash_airflowlist@firemirror. >>> com> wrote: >>>> >>>> -1 (for now. Non binding. Is that how this process works?) >>>> >>>> We've built a test env for this RC and are testing, but have run into an >>> issue reading task logs. (See below) >>>> >>>> We haven't gotten very far with this yet, we will dig more tomorrow >>> (it's the end of the UK work day now). I suspect this might be how we've >>> misconfigured our logging. We will see tomorrow. >>>> >>>> -ash >>>> >>>> >>>> >>>> >>>> File "/usr/local/lib/python3.5/dist-packages/airflow/www/views.py", >>> line 712, in log >>>> logs = handler.read(ti) >>>> AttributeError: 'NoneType' object has no attribute 'read' >>>> >>>> During handling of the above exception, another exception occurred: >>>> >>>> Traceback (most recent call last): >>>> File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1988, >>> in wsgi_app >>>> response = self.full_dispatch_request() >>>> File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1641, >>> in full_dispatch_request >>>> rv = self.handle_user_exception(e) >>>> File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1544, >>> in handle_user_exception >>>> reraise(exc_type, exc_value, tb) >>>> File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line >>> 33, in reraise >>>> raise value >>>> File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1639, >>> in full_dispatch_request >>>> rv = self.dispatch_request() >>>> File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1625, >>> in dispatch_request >>>> return self.view_functions[rule.endpoint](**req.view_args) >>>> File "/usr/local/lib/python3.5/dist-packages/flask_admin/base.py", >>> line 69, in inner >>>> return self._run_view(f, *args, **kwargs) >>>> File "/usr/local/lib/python3.5/dist-packages/flask_admin/base.py", >>> line 368, in _run_view >>>> return fn(self, *args, **kwargs) >>>> File "/usr/local/lib/python3.5/dist-packages/flask_login.py", line >>> 758, in decorated_view >>>> return func(*args, **kwargs) >>>> File "/usr/local/lib/python3.5/dist-packages/airflow/www/utils.py", >>> line 262, in wrapper >>>> return f(*args, **kwargs) >>>> File "/usr/local/lib/python3.5/dist-packages/airflow/www/views.py", >>> line 715, in log >>>> .format(task_log_reader, e.message)] >>>> AttributeError: 'AttributeError' object has no attribute 'message' >>>> >>>> >>>>> On 8 Nov 2017, at 17:46, Chris Riccomini <[email protected]> wrote: >>>>> >>>>> Anyone? :/ >>>>> >>>>> On Mon, Nov 6, 2017 at 1:22 PM, Chris Riccomini <[email protected]> >>>>> wrote: >>>>> >>>>>> Hey all, >>>>>> >>>>>> I have cut Airflow 1.9.0 RC1. This email is calling a vote on the >>> release, >>>>>> which will last fo 72 hours. Consider this my (binding) +1. >>>>>> >>>>>> Airflow 1.9.0 RC1 is available at: >>>>>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0rc1/ >>>>>> >>>>>> apache-airflow-1.9.0rc1+incubating-source.tar.gz is a source release >>> that >>>>>> comes with INSTALL instructions. >>>>>> apache-airflow-1.9.0rc1+incubating-bin.tar.gz is the binary Python >>>>>> "sdist" release. >>>>>> >>>>>> Public keys are available at: >>>>>> >>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/ >>>>>> >>>>>> The release contains the following JIRAs: >>>>>> >>>>>> ISSUE ID |DESCRIPTION |PR >>>>>> |COMMIT >>>>>> AIRFLOW-1779|Add keepalive packets to ssh hook |#2749 >>>>>> |d2f9d1 >>>>>> AIRFLOW-1776|stdout/stderr logging not captured |#2745 >>>>>> |590d9f >>>>>> AIRFLOW-1771|Change heartbeat text from boom to heartbeat |- >>> |- >>>>>> >>>>>> AIRFLOW-1767|Airflow Scheduler no longer schedules DAGs |- >>> |- >>>>>> >>>>>> AIRFLOW-1765|Default API auth backed should deny all. |#2737 >>>>>> |6ecdac >>>>>> AIRFLOW-1764|Web Interface should not use experimental api |#2738 >>>>>> |6bed1d >>>>>> AIRFLOW-1757|Contrib.SparkSubmitOperator should allow --package|#2725 >>>>>> |4e06ee >>>>>> AIRFLOW-1745|BashOperator ignores SIGPIPE in subprocess |#2714 >>>>>> |e021c9 >>>>>> AIRFLOW-1744|task.retries can be False |#2713 >>>>>> |6144c6 >>>>>> AIRFLOW-1743|Default config template should not contain ldap fi|#2712 >>>>>> |270684 >>>>>> AIRFLOW-1741|Task Duration shows two charts on first page load.|#2711 >>>>>> |974b49 >>>>>> AIRFLOW-1734|Sqoop Operator contains logic errors & needs optio|#2703 >>>>>> |f6810c >>>>>> AIRFLOW-1731|Import custom config on PYTHONPATH |#2721 >>>>>> |f07eb3 >>>>>> AIRFLOW-1726|Copy Expert command for Postgres Hook |#2698 >>>>>> |8a4ad3 >>>>>> AIRFLOW-1719|Fix small typo - your vs you |- >>> |- >>>>>> >>>>>> AIRFLOW-1712|Log SSHOperator output |- >>> |- >>>>>> >>>>>> AIRFLOW-1711|Ldap Attributes not always a "list" part 2 |#2731 >>>>>> |40a936 >>>>>> AIRFLOW-1706|Scheduler is failed on startup with MS SQL Server |#2733 >>>>>> |9e209b >>>>>> AIRFLOW-1698|Remove confusing SCHEDULER_RUNS env var from syste|#2677 >>>>>> |00dd06 >>>>>> AIRFLOW-1695|Redshift Hook using boto3 & AWS Hook |#2717 >>>>>> |bfddae >>>>>> AIRFLOW-1694|Hive Hooks: Python 3 does not have an `itertools.i|#2674 >>>>>> |c6e5ae >>>>>> AIRFLOW-1692|Master cannot be checked out on windows |#2673 >>>>>> |31805e >>>>>> AIRFLOW-1691|Add better documentation for Google cloud storage |#2671 >>>>>> |ace2b1 >>>>>> AIRFLOW-1690|Error messages regarding gcs log commits are spars|#2670 >>>>>> |5fb5cd >>>>>> AIRFLOW-1682|S3 task handler never writes to S3 |#2664 >>>>>> |0080f0 >>>>>> AIRFLOW-1678|Fix docstring errors for `set_upstream` and `set_d|- >>> |- >>>>>> >>>>>> AIRFLOW-1677|Fix typo in example_qubole_operator |- >>> |- >>>>>> >>>>>> AIRFLOW-1676|GCS task handler never writes to GCS |#2659 >>>>>> |781fa4 >>>>>> AIRFLOW-1675|Fix API docstrings to be properly rendered |#2667 >>>>>> |f12381 >>>>>> AIRFLOW-1671|Missing @apply_defaults annotation for gcs downloa|#2655 >>>>>> |97666b >>>>>> AIRFLOW-1669|Fix Docker import in Master |#na >>>>>> |f7f2a8 >>>>>> AIRFLOW-1668|Redhsift requires a keep alive of < 300s |#2650 >>>>>> |f2bb77 >>>>>> AIRFLOW-1664|Make MySqlToGoogleCloudStorageOperator support bin|#2649 >>>>>> |95813d >>>>>> AIRFLOW-1660|Change webpage width to full-width |#2646 >>>>>> |8ee3d9 >>>>>> AIRFLOW-1659|Fix invalid attribute bug in FileTaskHandler |#2645 >>>>>> |bee823 >>>>>> AIRFLOW-1658|Kill (possibly) still running Druid indexing job a|#2644 >>>>>> |cbf7ad >>>>>> AIRFLOW-1657|Handle failure of Qubole Operator for s3distcp had|- >>> |- >>>>>> >>>>>> AIRFLOW-1654|Show tooltips for link icons in DAGs view |#2642 >>>>>> |ada7b2 >>>>>> AIRFLOW-1647|Fix Spark-sql hook |#2637 >>>>>> |b1e5c6 >>>>>> AIRFLOW-1641|Task gets stuck in queued state |#2715 >>>>>> |735497 >>>>>> AIRFLOW-1640|Add Qubole default connection in connection table |- >>> |- >>>>>> >>>>>> AIRFLOW-1639|ValueError does not have .message attribute |#2629 >>>>>> |87df67 >>>>>> AIRFLOW-1637|readme not tracking master branch for travis |- >>> |- >>>>>> >>>>>> AIRFLOW-1636|aws and emr connection types get cleared |#2626 >>>>>> |540e04 >>>>>> AIRFLOW-1635|Allow creating Google Cloud Platform connection wi|#2640 >>>>>> |6dec7a >>>>>> AIRFLOW-1629|make extra a textarea in edit connections form |#2623 >>>>>> |f5d46f >>>>>> AIRFLOW-1628|Docstring of sqlsensor is incorrect |#2621 >>>>>> |9ba73d >>>>>> AIRFLOW-1627|SubDagOperator initialization should only query po|#2620 >>>>>> |516ace >>>>>> AIRFLOW-1621|Add tests for logic added on server side dag list |#2614 >>>>>> |8de9fd >>>>>> AIRFLOW-1614|Improve performance of DAG parsing when there are |#2610 >>>>>> |a95adb >>>>>> AIRFLOW-1611|Customize logging in Airflow |#2631 >>>>>> |8b4a50 >>>>>> AIRFLOW-1609|Ignore all venvs in gitignore |#2608 >>>>>> |f1f9b4 >>>>>> AIRFLOW-1608|GCP Dataflow hook missing pending job state |#2607 >>>>>> |653562 >>>>>> AIRFLOW-1606|DAG.sync_to_db is static, but takes a DAG as first|#2606 >>>>>> |6ac296 >>>>>> AIRFLOW-1605|Fix log source of local loggers |- >>> |- >>>>>> >>>>>> AIRFLOW-1604|Rename the logger to log |#2604 >>>>>> |af4050 >>>>>> AIRFLOW-1602|Use LoggingMixin for the DAG class |#2602 >>>>>> |956699 >>>>>> AIRFLOW-1601|Add configurable time between SIGTERM and SIGKILL |#2601 >>>>>> |48a95e >>>>>> AIRFLOW-1600|Uncaught exceptions in get_fernet if cryptography |#2600 >>>>>> |ad963e >>>>>> AIRFLOW-1597|Add GameWisp as Airflow user |#2599 >>>>>> |26b747 >>>>>> AIRFLOW-1594|Installing via pip copies test files into python l|#2597 >>>>>> |a6b23a >>>>>> AIRFLOW-1593|Expose load_string in WasbHook |#2596 >>>>>> |7ece95 >>>>>> AIRFLOW-1591|Exception: 'TaskInstance' object has no attribute |#2578 >>>>>> |f4653e >>>>>> AIRFLOW-1590|Small fix for dates util |#2652 >>>>>> |31946e >>>>>> AIRFLOW-1587|fix `ImportError: cannot import name 'CeleryExecut|#2590 >>>>>> |34c73b >>>>>> AIRFLOW-1586|MySQL to GCS to BigQuery fails for tables with dat|#2589 >>>>>> |e83012 >>>>>> AIRFLOW-1584|Remove the insecure /headers endpoints |#2588 >>>>>> |17ac07 >>>>>> AIRFLOW-1582|Improve logging structure of Airflow |#2592 >>>>>> |a7a518 >>>>>> AIRFLOW-1580|Error in string formatter when throwing an excepti|#2583 >>>>>> |ea9ab9 >>>>>> AIRFLOW-1579|Allow jagged rows in BQ Hook. |#2582 >>>>>> |5b978b >>>>>> AIRFLOW-1577|Add token support to DatabricksHook |#2579 >>>>>> |c2c515 >>>>>> AIRFLOW-1573|Remove `thrift < 0.10.0` requirement |#2574 >>>>>> |aa95f2 >>>>>> AIRFLOW-1571|Add AWS Lambda Hook for invoking Lambda Function |#2718 >>>>>> |017f18 >>>>>> AIRFLOW-1568|Add datastore import/export operator |#2568 >>>>>> |86063b >>>>>> AIRFLOW-1567|Clean up ML Engine operators |#2567 >>>>>> |af91e2 >>>>>> AIRFLOW-1564|Default logging filename contains a colon |#2565 >>>>>> |4c674c >>>>>> AIRFLOW-1560|Add AWS DynamoDB hook for inserting batch items |#2587 >>>>>> |71400b >>>>>> AIRFLOW-1556|BigQueryBaseCursor should support SQL parameters |#2557 >>>>>> |9df0ac >>>>>> AIRFLOW-1546| add Zymergen to org list in README |#2512 >>>>>> |7cc346 >>>>>> AIRFLOW-1535|Add support for Dataproc serviceAccountScopes in D|#2546 >>>>>> |b1f902 >>>>>> AIRFLOW-1529|Support quoted newlines in Google BigQuery load jo|#2545 >>>>>> |4a4b02 >>>>>> AIRFLOW-1527|Refactor celery config to make use of template |#2542 >>>>>> |f4437b >>>>>> AIRFLOW-1522|Increase size of val column for variable table in |#2535 >>>>>> |8a2d24 >>>>>> AIRFLOW-1521|Template fields definition for bigquery_table_dele|#2534 >>>>>> |f1a7c0 >>>>>> AIRFLOW-1520|S3Hook uses boto2 |#2532 >>>>>> |386583 >>>>>> AIRFLOW-1519|Main DAG list page does not scale using client sid|#2531 >>>>>> |d7d7ce >>>>>> AIRFLOW-1512|Add operator for running Python functions in a vir|#2446 >>>>>> |14e6d7 >>>>>> AIRFLOW-1507|Make src, dst and bucket parameters as templated i|#2516 >>>>>> |d295cf >>>>>> AIRFLOW-1505|Document when Jinja substitution occurs |#2523 >>>>>> |984a87 >>>>>> AIRFLOW-1504|Log Cluster Name on Dataproc Operator When Execute|#2517 >>>>>> |1cd6c4 >>>>>> AIRFLOW-1499s|Eliminate duplicate and unneeded code |- >>> |- >>>>>> >>>>>> AIRFLOW-1497|Hidden fields in connection form aren't reset when|#2507 >>>>>> |d8da8b >>>>>> AIRFLOW-1493|Fix race condition with airflow run |#2505 >>>>>> |b2e175 >>>>>> AIRFLOW-1492|Add metric for task success/failure |#2504 >>>>>> |fa84d4 >>>>>> AIRFLOW-1489|Docs: Typo in BigQueryCheckOperator |#2501 >>>>>> |111ce5 >>>>>> AIRFLOW-1483|Page size on model views is to large to render qui|#2497 >>>>>> |04bfba >>>>>> AIRFLOW-1478|Chart -> Owner column should be sortable |#2493 >>>>>> |651e60 >>>>>> AIRFLOW-1476|Add INSTALL file for source releases |#2492 >>>>>> |da76ac >>>>>> AIRFLOW-1474|Add dag_id regex for 'airflow clear' CLI command |#2486 >>>>>> |18f849 >>>>>> AIRFLOW-1470s|BashSensor Implementation |- >>> |- >>>>>> >>>>>> AIRFLOW-1459|integration rst doc is broken in github view |#2481 >>>>>> |322ec9 >>>>>> AIRFLOW-1438|Scheduler batch queries should have a limit |#2462 >>>>>> |3547cb >>>>>> AIRFLOW-1437|BigQueryTableDeleteOperator should define deletion|#2459 >>>>>> |b87903 >>>>>> AIRFLOW-1432|NVD3 Charts do not have labeled axes and units cha|#2710 >>>>>> |70ffa4 >>>>>> AIRFLOW-1402|Cleanup SafeConfigParser DeprecationWarning |#2435 >>>>>> |38c86b >>>>>> AIRFLOW-1401|Standardize GCP project, region, and zone argument|#2439 >>>>>> |b6d363 >>>>>> AIRFLOW-1397|Airflow 1.8.1 - No data displays in Last Run Colum|- >>> |- >>>>>> >>>>>> AIRFLOW-1394|Add quote_character parameter to GoogleCloudStorag|#2428 >>>>>> |9fd0be >>>>>> AIRFLOW-1389|BigQueryOperator should support `createDisposition|#2470 >>>>>> |6e2640 >>>>>> AIRFLOW-1384|Add ARGO/CaDC |#2434 >>>>>> |715947 >>>>>> AIRFLOW-1368|Automatically remove the container when it exits |#2653 >>>>>> |d42d23 >>>>>> AIRFLOW-1359|Provide GoogleCloudML operator for model evaluatio|#2407 >>>>>> |194d1d >>>>>> AIRFLOW-1356|add `--celery_hostname` to `airflow worker` |#2405 >>>>>> |b9d7d1 >>>>>> AIRFLOW-1352|Revert bad logging Handler |- >>> |- >>>>>> >>>>>> AIRFLOW-1350|Add "query_uri" parameter for Google DataProc oper|#2402 >>>>>> |d32c72 >>>>>> AIRFLOW-1348|Paginated UI has broken toggles after first page |- >>> |- >>>>>> >>>>>> AIRFLOW-1345|Don't commit on each loop |#2397 >>>>>> |0dd002 >>>>>> AIRFLOW-1344|Builds failing on Python 3.5 with AttributeError |#2394 >>>>>> |2a5883 >>>>>> AIRFLOW-1343|Add airflow default label to the dataproc operator|#2396 >>>>>> |e4b240 >>>>>> AIRFLOW-1338|gcp_dataflow_hook is incompatible with the recent |#2388 >>>>>> |cf2605 >>>>>> AIRFLOW-1337|Customize log format via config file |#2392 >>>>>> |4841e3 >>>>>> AIRFLOW-1335|Use buffered logger |#2386 >>>>>> |0d23d3 >>>>>> AIRFLOW-1333|Enable copy function for Google Cloud Storage Hook|#2385 >>>>>> |e2c383 >>>>>> AIRFLOW-1331|Contrib.SparkSubmitOperator should allow --package|#2622 >>>>>> |fbca8f >>>>>> AIRFLOW-1330|Connection.parse_from_uri doesn't work for google_|#2525 >>>>>> |6e5e9d >>>>>> AIRFLOW-1324|Make the Druid operator/hook more general |#2378 >>>>>> |de99aa >>>>>> AIRFLOW-1323|Operators related to Dataproc should keep some par|#2636 >>>>>> |ed248d >>>>>> AIRFLOW-1315|Add Qubole File and Partition Sensors |- >>> |- >>>>>> >>>>>> AIRFLOW-1309|Add optional hive_tblproperties in HiveToDruidTran|- >>> |- >>>>>> >>>>>> AIRFLOW-1301|Add New Relic to Airflow user list |#2359 >>>>>> |355fc9 >>>>>> AIRFLOW-1299|Google Dataproc cluster creation operator should s|#2358 >>>>>> |c2b80e >>>>>> AIRFLOW-1289|Don't restrict scheduler threads to CPU cores |#2353 >>>>>> |8e23d2 >>>>>> AIRFLOW-1286|BaseTaskRunner - Exception TypeError: a bytes-like|#2363 >>>>>> |d8891d >>>>>> AIRFLOW-1277|Forbid creation of a known event with empty fields|#na >>>>>> |65184a >>>>>> AIRFLOW-1276|Forbid event creation with end_data earlier than s|#na >>>>>> |d5d02f >>>>>> AIRFLOW-1275|Fix `airflow pool` command exception |#2346 >>>>>> |9958aa >>>>>> AIRFLOW-1273|Google Cloud ML Version and Model CRUD Operator |#2379 >>>>>> |534a0e >>>>>> AIRFLOW-1272|Google Cloud ML Batch Prediction Operator |#2390 >>>>>> |e92d6b >>>>>> AIRFLOW-1271|Google Cloud ML Training Operator |#2408 >>>>>> |0fc450 >>>>>> AIRFLOW-1256|Add United Airlines as Airflow user |#2332 >>>>>> |d3484a >>>>>> AIRFLOW-1251|Add eRevalue as an Airflow user |#2331 >>>>>> |8d5160 >>>>>> AIRFLOW-1248|Fix inconsistent configuration name for worker tim|#2328 >>>>>> |92314f >>>>>> AIRFLOW-1247|CLI: ignore all dependencies argument ignored |#2441 >>>>>> |e88ecf >>>>>> AIRFLOW-1245|Fix random failure of test_trigger_dag_for_date un|#2325 >>>>>> |cef01b >>>>>> AIRFLOW-1244|Forbid creation of a pool with empty name |#2324 >>>>>> |df9a10 >>>>>> AIRFLOW-1242|BigQueryHook assumes that a valid project_id can't|#2335 >>>>>> |ffe616 >>>>>> AIRFLOW-1237|Fix IN-predicate sqlalchemy warning |#2320 >>>>>> |a1f422 >>>>>> AIRFLOW-1234|Cover utils.operator_helpers with unit tests |#2317 >>>>>> |d16537 >>>>>> AIRFLOW-1233|Cover utils.json with unit tests |#2316 >>>>>> |502410 >>>>>> AIRFLOW-1232|Remove deprecated readfp warning |#2315 >>>>>> |6ffaaf >>>>>> AIRFLOW-1231|Use flask_wtf.CSRFProtect instead of flask_wtf.Csr|#2313 >>>>>> |cac49e >>>>>> AIRFLOW-1221|Fix DatabricksSubmitRunOperator Templating |#2308 >>>>>> |0fa104 >>>>>> AIRFLOW-1217|Enable logging in Sqoop hook |#2307 >>>>>> |4f459b >>>>>> AIRFLOW-1213|Add hcatalog parameters to the sqoop operator/hook|#2305 >>>>>> |857850 >>>>>> AIRFLOW-1208|Speed-up cli tests |#2301 >>>>>> |21c142 >>>>>> AIRFLOW-1207|Enable utils.helpers unit tests |#2300 >>>>>> |8ac87b >>>>>> AIRFLOW-1203|Tests failing after oauth upgrade |#2296 >>>>>> |3e9c66 >>>>>> AIRFLOW-1201|Update deprecated 'nose-parameterized' library to |#2298 >>>>>> |d2d3e4 >>>>>> AIRFLOW-1193|Add Checkr to Airflow user list |#2276 >>>>>> |707238 >>>>>> AIRFLOW-1189|Get pandas DataFrame using BigQueryHook fails |#2287 >>>>>> |93666f >>>>>> AIRFLOW-1188|Add max_bad_records param to GoogleCloudStorageToB|#2286 >>>>>> |443e6b >>>>>> AIRFLOW-1187|Obsolete package names in documentation |- >>> |- >>>>>> >>>>>> AIRFLOW-1185|Incorrect url to PyPi |#2283 >>>>>> |829755 >>>>>> AIRFLOW-1181|Enable delete and list function for Google Cloud S|#2281 >>>>>> |24f73c >>>>>> AIRFLOW-1179|Pandas 0.20 broke Google BigQuery hook |#2279 >>>>>> |ac9ccb >>>>>> AIRFLOW-1177|variable json deserialize does not work at set def|#2540 >>>>>> |65319a >>>>>> AIRFLOW-1175|Add Pronto Tools to Airflow user list |#2277 >>>>>> |86aafa >>>>>> AIRFLOW-1173|Add Robinhood to list of Airflow users |#2271 >>>>>> |379115 >>>>>> AIRFLOW-1165|airflow webservice crashes on ubuntu16 - python3 |- >>> |- >>>>>> >>>>>> AIRFLOW-1160|Upadte SparkSubmitOperator parameters |#2265 >>>>>> |2e3f07 >>>>>> AIRFLOW-1155|Add Tails.com to community |#2261 >>>>>> |2fa690 >>>>>> AIRFLOW-1149|Allow custom filters to be added to jinja2 |#2258 >>>>>> |48135a >>>>>> AIRFLOW-1141|Remove DAG.crawl_for_tasks method |#2275 >>>>>> |a30fee >>>>>> AIRFLOW-1140|DatabricksSubmitRunOperator should template the "j|#2255 >>>>>> |e6d316 >>>>>> AIRFLOW-1136|Invalid parameters are not captured for Sqoop oper|#2252 >>>>>> |2ef4db >>>>>> AIRFLOW-1125|Clarify documentation regarding fernet_key |#2251 >>>>>> |831f8d >>>>>> AIRFLOW-1122|Node strokes are too thin for people with color vi|#2246 >>>>>> |a08761 >>>>>> AIRFLOW-1121|airflow webserver --pid no longer write out pid fi|- >>> |- >>>>>> >>>>>> AIRFLOW-1118|Add evo.company to Airflow users |#2243 >>>>>> |f16914 >>>>>> AIRFLOW-1112|Log which pool is full in scheduler when pool slot|#2242 >>>>>> |74c1ce >>>>>> AIRFLOW-1107|Add support for ftps non-default port |#2240 >>>>>> |4d0c2f >>>>>> AIRFLOW-1106|Add Groupalia/Letsbonus |#2239 >>>>>> |945b42 >>>>>> AIRFLOW-1095|ldap_auth memberOf should come from configuration |#2232 >>>>>> |6b1c32 >>>>>> AIRFLOW-1094|Invalid unit tests under `contrib/` |#2234 >>>>>> |219c50 >>>>>> AIRFLOW-1091|As a release manager I want to be able to compare |#2231 >>>>>> |bfae42 >>>>>> AIRFLOW-1090|Add HBO |#2230 >>>>>> |177d34 >>>>>> AIRFLOW-1089|Add Spark application arguments to SparkSubmitOper|#2229 >>>>>> |e5b914 >>>>>> AIRFLOW-1081|Task duration page is slow |#2226 >>>>>> |0da512 >>>>>> AIRFLOW-1075|Cleanup security docs |#2222 >>>>>> |5a6f18 >>>>>> AIRFLOW-1065|Add functionality for Azure Blob Storage |#2216 >>>>>> |f1bc5f >>>>>> AIRFLOW-1059|Reset_state_for_orphaned_task should operate in ba|#2205 >>>>>> |e05d3b >>>>>> AIRFLOW-1058|Improvements for SparkSubmitOperator |- >>> |- >>>>>> >>>>>> AIRFLOW-1051|Add a test for resetdb to CliTests |#2198 >>>>>> |15aee0 >>>>>> AIRFLOW-1047|Airflow logs vulnerable to XSS |#2193 >>>>>> |fe9ebe >>>>>> AIRFLOW-1045|Make log level configurable via airflow.cfg |#2191 >>>>>> |e739a5 >>>>>> AIRFLOW-1043|Documentation issues for operators |#2188 >>>>>> |b55f41 >>>>>> AIRFLOW-1041|DockerOperator replaces its xcom_push method with |#2274 >>>>>> |03704c >>>>>> AIRFLOW-1040|Fix typos in comments/docstrings in models.py |#2174 >>>>>> |d8c0f5 >>>>>> AIRFLOW-1036|Exponential backoff should use randomization |#2262 >>>>>> |66168e >>>>>> AIRFLOW-1035|Exponential backoff retry logic should use 2 as ba|#2196 >>>>>> |4ec932 >>>>>> AIRFLOW-1034|Make it possible to connect to S3 in sigv4 regions|#2181 >>>>>> |4c0905 >>>>>> AIRFLOW-1031|'scheduled__' may replace with DagRun.ID_PREFIX in|#2613 >>>>>> |aa3844 >>>>>> AIRFLOW-1030|HttpHook error when creating HttpSensor |- >>> |- >>>>>> >>>>>> AIRFLOW-1028|Databricks Operator for Airflow |#2202 >>>>>> |53ca50 >>>>>> AIRFLOW-1024|Handle CeleryExecutor errors gracefully |#2355 >>>>>> |7af20f >>>>>> AIRFLOW-1018|Scheduler DAG processes can not log to stdout |#2728 >>>>>> |ef775d >>>>>> AIRFLOW-1016|Allow HTTP HEAD request method on HTTPSensor |#2175 >>>>>> |4c41f6 >>>>>> AIRFLOW-1010|Add a convenience script for signing |#2169 >>>>>> |a2b65a >>>>>> AIRFLOW-1009|Remove SQLOperator from Concepts page |#2168 >>>>>> |7d1144 >>>>>> AIRFLOW-1007|Jinja sandbox is vulnerable to RCE |#2184 >>>>>> |daa281 >>>>>> AIRFLOW-1005|Speed up Airflow startup time |#na >>>>>> |996dd3 >>>>>> AIRFLOW-999 |Support for Redis database |#2165 >>>>>> |8de850 >>>>>> AIRFLOW-997 |Change setup.cfg to point to Apache instead of Max|#na >>>>>> |75cd46 >>>>>> AIRFLOW-995 |Update Github PR template |#2163 >>>>>> |b62485 >>>>>> AIRFLOW-994 |Add MiNODES to the AIRFLOW Active Users List |#2159 >>>>>> |ca1623 >>>>>> AIRFLOW-991 |Mark_success while a task is running leads to fail|- >>> |- >>>>>> >>>>>> AIRFLOW-990 |DockerOperator fails when logging unicode string |#2155 >>>>>> |6bbf54 >>>>>> AIRFLOW-988 |SLA Miss Callbacks Are Repeated if Email is Not be|#2415 >>>>>> |6e74d4 >>>>>> AIRFLOW-985 |Extend the sqoop operator/hook with additional par|#2177 >>>>>> |82eb20 >>>>>> AIRFLOW-984 |Subdags unrecognized when subclassing SubDagOperat|#2152 >>>>>> |a8bd16 >>>>>> AIRFLOW-979 |Add GovTech GDS |#2149 >>>>>> |b17bd3 >>>>>> AIRFLOW-976 |Mark success running task causes it to fail |- >>> |- >>>>>> >>>>>> AIRFLOW-969 |Catch bad python_callable argument at DAG construc|#2142 >>>>>> |12901d >>>>>> AIRFLOW-963 |Some code examples are not rendered in the airflow|#2139 >>>>>> |f69c1b >>>>>> AIRFLOW-960 |Add support for .editorconfig |#na >>>>>> |f5cacc >>>>>> AIRFLOW-959 |.gitignore file is disorganized and incomplete |#na >>>>>> |3d3c14 >>>>>> AIRFLOW-958 |Improve tooltip readability |#2134 >>>>>> |b3c3eb >>>>>> AIRFLOW-950 |Missing AWS integrations on documentation::integra|#2552 >>>>>> |01be02 >>>>>> AIRFLOW-947 |Make PrestoHook surface better messages when the P|#na >>>>>> |6dd4b3 >>>>>> AIRFLOW-945 |Revert psycopg2 workaround when psycopg2 2.7.1 is |- >>> |- >>>>>> >>>>>> AIRFLOW-943 |Add Digital First Media to the Airflow users list |#2115 >>>>>> |2cfe28 >>>>>> AIRFLOW-942 |Add mytaxi to Airflow Users |#2111 >>>>>> |d579e6 >>>>>> AIRFLOW-935 |Impossible to use plugin executors |#2120 >>>>>> |08a784 >>>>>> AIRFLOW-926 |jdbc connector is broken due to jaydebeapi api upd|#2651 >>>>>> |07ed29 >>>>>> AIRFLOW-917 |Incorrectly formatted failure status message |#2109 >>>>>> |b8164c >>>>>> AIRFLOW-916 |Fix ConfigParser deprecation warning |#2108 >>>>>> |ef6dd1 >>>>>> AIRFLOW-911 |Add colouring and profiling info on tests |#2106 >>>>>> |4f52db >>>>>> AIRFLOW-903 |Add configuration setting for default DAG view. |#2103 >>>>>> |cadfae >>>>>> AIRFLOW-896 |BigQueryOperator fails to execute with certain inp|#2097 >>>>>> |2bceee >>>>>> AIRFLOW-891 |Webserver Clock Should Include Day |- >>> |- >>>>>> >>>>>> AIRFLOW-889 |Minor error in the docstrings for BaseOperator. |#2084 >>>>>> |50702d >>>>>> AIRFLOW-887 |Add compatibility with future v0.16 |#na >>>>>> |50902d >>>>>> AIRFLOW-886 |Pass Operator result to post_execute hook |#na >>>>>> |4da361 >>>>>> AIRFLOW-885 |Add Change.org to the list of Airflow users |#2089 >>>>>> |a279be >>>>>> AIRFLOW-882 |Code example in docs has unnecessary DAG>>Operator|#2088 >>>>>> |baa4cd >>>>>> AIRFLOW-881 |Create SubDagOperator within DAG context manager w|#2087 >>>>>> |0ed608 >>>>>> AIRFLOW-880 |Fix remote log functionality inconsistencies for W|#2086 >>>>>> |974b75 >>>>>> AIRFLOW-877 |GoogleCloudStorageDownloadOperator: template_ext c|#2083 >>>>>> |debc69 >>>>>> AIRFLOW-875 |Allow HttpSensor params to be templated |#2080 >>>>>> |62f503 >>>>>> AIRFLOW-871 |multiple places use logging.warn() instead of warn|#2082 >>>>>> |21d775 >>>>>> AIRFLOW-866 |Add FTPSensor |#2070 >>>>>> |5f87f8 >>>>>> AIRFLOW-863 |Example DAG start dates should be recent to avoid |#2068 >>>>>> |bbfd43 >>>>>> AIRFLOW-862 |Add DaskExecutor |#2067 >>>>>> |6e2210 >>>>>> AIRFLOW-860 |Circular module dependency prevents loading of cus|- >>> |- >>>>>> >>>>>> AIRFLOW-854 |Add Open Knowledge International to Airflow users |#2061 >>>>>> |51a311 >>>>>> AIRFLOW-842 |scheduler.clean_dirty raises warning: SAWarning: T|#2072 >>>>>> |485280 >>>>>> AIRFLOW-840 |Python3 encoding issue in Kerberos |#2158 >>>>>> |639336 >>>>>> AIRFLOW-836 |The paused and queryview endpoints are vulnerable |#2054 >>>>>> |6aca2c >>>>>> AIRFLOW-831 |Fix broken unit tests |#2050 >>>>>> |b86194 >>>>>> AIRFLOW-830 |Plugin manager should log to debug, not info |- >>> |- >>>>>> >>>>>> AIRFLOW-829 |Reduce verbosity of successful Travis unit tests |- >>> |- >>>>>> >>>>>> AIRFLOW-826 |Add Zendesk Hook |#2066 >>>>>> |a09762 >>>>>> AIRFLOW-823 |Make task instance details available via API |#2045 >>>>>> |3f546e >>>>>> AIRFLOW-822 |Close the connection before throwing exception in |#2038 >>>>>> |4b6c38 >>>>>> AIRFLOW-821 |Scheduler dagbag importing not Py3 compatible |#2039 >>>>>> |fbb59b >>>>>> AIRFLOW-809 |SqlAlchemy is_ ColumnOperator Causing Errors in MS|- >>> |- >>>>>> >>>>>> AIRFLOW-802 |Integration of spark-submit |- >>> |- >>>>>> >>>>>> AIRFLOW-781 |Allow DataFlowJavaOperator to accept jar file stor|#2037 >>>>>> |259c86 >>>>>> AIRFLOW-770 |HDFS hooks should support alternative ways of gett|#2056 >>>>>> |261b65 >>>>>> AIRFLOW-756 |Refactor ssh_hook and ssh_operator |- >>> |- >>>>>> >>>>>> AIRFLOW-751 |SFTP file transfer functionality |#1999 >>>>>> |fe0ede >>>>>> AIRFLOW-725 |Make merge tool use OS' keyring for password stora|#1966 >>>>>> |8c1695 >>>>>> AIRFLOW-706 |Configuration shell commands are not split properl|#2053 >>>>>> |0bb6f2 >>>>>> AIRFLOW-705 |airflow.configuration.run_command output does not |- >>> |- >>>>>> >>>>>> AIRFLOW-681 |homepage doc link should pointing to apache's repo|#2164 >>>>>> |a8027a >>>>>> AIRFLOW-654 |SSL for AMQP w/ Celery(Executor) |#2333 >>>>>> |868bfe >>>>>> AIRFLOW-645 |HttpHook ignores https |#2311 >>>>>> |fd381a >>>>>> AIRFLOW-365 |Code view in subdag trigger exception |#2043 >>>>>> |cf102c >>>>>> AIRFLOW-300 |Add Google Pubsub hook and operator |#2036 >>>>>> |d231dc >>>>>> AIRFLOW-289 |Use datetime.utcnow() to keep airflow system indep|#2618 >>>>>> |20c83e >>>>>> AIRFLOW-71 |docker_operator - pulling from private repositorie|#na >>>>>> |d4406c >>>>>> >>>>>> Cheers, >>>>>> Chris >>>>>> >>>> >>> >>> >
