Hey all, Figured out the issue with the JIRA list at the top of this email. There's a bug in the dev/airflow-jira script that limits the results to a max of 50. Here's the total list of everything that's slated for 1.9.0:
ISSUE ID |DESCRIPTION |MERGED AIRFLOW-1608 |GCP Dataflow hook missing pending job state |1 AIRFLOW-1606 |DAG.sync_to_db is static, but takes a DAG as first|1 AIRFLOW-1605 |Fix log source of local loggers |0 AIRFLOW-1602 |Use LoggingMixin for the DAG class |1 AIRFLOW-1597 |Add GameWisp as Airflow user |1 AIRFLOW-1594 |Installing via pip copies test files into python l|1 AIRFLOW-1593 |Expose load_string in WasbHook |1 AIRFLOW-1586 |MySQL to GCS to BigQuery fails for tables with dat|1 AIRFLOW-1584 |Remove the insecure /headers endpoints |1 AIRFLOW-1582 |Improve logging structure of Airflow |1 AIRFLOW-1580 |Error in string formatter when throwing an excepti|1 AIRFLOW-1579 |Allow jagged rows in BQ Hook. |1 AIRFLOW-1577 |Add token support to DatabricksHook |1 AIRFLOW-1573 |Remove `thrift < 0.10.0` requirement |1 AIRFLOW-1568 |Add datastore import/export operator |1 AIRFLOW-1567 |Clean up ML Engine operators |1 AIRFLOW-1564 |Default logging filename contains a colon |1 AIRFLOW-1556 |BigQueryBaseCursor should support SQL parameters |1 AIRFLOW-1546 | add Zymergen to org list in README |1 AIRFLOW-1535 |Add support for Dataproc serviceAccountScopes in D|1 AIRFLOW-1529 |Support quoted newlines in Google BigQuery load jo|1 AIRFLOW-1525 |Fix minor LICENSE & NOTICE issue |0 AIRFLOW-1522 |Increase size of val column for variable table in |1 AIRFLOW-1521 |Template fields definition for bigquery_table_dele|1 AIRFLOW-1507 |Make src, dst and bucket parameters as templated i|1 AIRFLOW-1505 |Document when Jinja substitution occurs |1 AIRFLOW-1504 |Log Cluster Name on Dataproc Operator When Execute|1 AIRFLOW-1499 s|Eliminate duplicate and unneeded code |0 AIRFLOW-1493 |Fix race condition with airflow run |1 AIRFLOW-1492 |Add metric for task success/failure |1 AIRFLOW-1489 |Docs: Typo in BigQueryCheckOperator |1 AIRFLOW-1478 |Chart -> Owner column should be sortable |1 AIRFLOW-1476 |Add INSTALL file for source releases |1 AIRFLOW-1474 |Add dag_id regex for 'airflow clear' CLI command |1 AIRFLOW-1459 |integration rst doc is broken in github view |1 AIRFLOW-1438 |Scheduler batch queries should have a limit |1 AIRFLOW-1437 |BigQueryTableDeleteOperator should define deletion|1 AIRFLOW-1402 |Cleanup SafeConfigParser DeprecationWarning |1 AIRFLOW-1401 |Standardize GCP project, region, and zone argument|1 AIRFLOW-1394 |Add quote_character parameter to GoogleCloudStorag|1 AIRFLOW-1389 |BigQueryOperator should support `createDisposition|1 AIRFLOW-1384 |Add ARGO/CaDC |1 AIRFLOW-1359 |Provide GoogleCloudML operator for model evaluatio|1 AIRFLOW-1352 |Revert bad logging Handler |0 AIRFLOW-1350 |Add "query_uri" parameter for Google DataProc oper|1 AIRFLOW-1345 |Don't commit on each loop |1 AIRFLOW-1344 |Builds failing on Python 3.5 with AttributeError |1 AIRFLOW-1343 |Add airflow default label to the dataproc operator|1 AIRFLOW-1338 |gcp_dataflow_hook is incompatible with the recent |1 AIRFLOW-1337 |Customize log format via config file |1 AIRFLOW-1335 |Use buffered logger |1 AIRFLOW-1333 |Enable copy function for Google Cloud Storage Hook|1 AIRFLOW-1324 |Make the Druid operator/hook more general |1 AIRFLOW-1301 |Add New Relic to Airflow user list |1 AIRFLOW-1299 |Google Dataproc cluster creation operator should s|1 AIRFLOW-1289 |Don't restrict scheduler threads to CPU cores |1 AIRFLOW-1286 |BaseTaskRunner - Exception TypeError: a bytes-like|1 AIRFLOW-1277 |Forbid creation of a known event with empty fields|1 AIRFLOW-1276 |Forbid event creation with end_data earlier than s|1 AIRFLOW-1275 |Fix `airflow pool` command exception |1 AIRFLOW-1274 |HttpSensor parameter params is overriding BaseOper|1 AIRFLOW-1273 |Google Cloud ML Version and Model CRUD Operator |1 AIRFLOW-1272 |Google Cloud ML Batch Prediction Operator |1 AIRFLOW-1271 |Google Cloud ML Training Operator |1 AIRFLOW-1256 |Add United Airlines as Airflow user |1 AIRFLOW-1251 |Add eRevalue as an Airflow user |1 AIRFLOW-1248 |Fix inconsistent configuration name for worker tim|1 AIRFLOW-1247 |CLI: ignore all dependencies argument ignored |1 AIRFLOW-1245 |Fix random failure of test_trigger_dag_for_date un|1 AIRFLOW-1244 |Forbid creation of a pool with empty name |1 AIRFLOW-1242 |BigQueryHook assumes that a valid project_id can't|1 AIRFLOW-1237 |Fix IN-predicate sqlalchemy warning |1 AIRFLOW-1234 |Cover utils.operator_helpers with unit tests |1 AIRFLOW-1233 |Cover utils.json with unit tests |1 AIRFLOW-1232 |Remove deprecated readfp warning |1 AIRFLOW-1231 |Use flask_wtf.CSRFProtect instead of flask_wtf.Csr|1 AIRFLOW-1221 |Fix DatabricksSubmitRunOperator Templating |1 AIRFLOW-1217 |Enable logging in Sqoop hook |1 AIRFLOW-1213 |Add hcatalog parameters to the sqoop operator/hook|1 AIRFLOW-1208 |Speed-up cli tests |1 AIRFLOW-1207 |Enable utils.helpers unit tests |1 AIRFLOW-1203 |Tests failing after oauth upgrade |1 AIRFLOW-1201 |Update deprecated 'nose-parameterized' library to |1 AIRFLOW-1198 |HDFSOperator to operate HDFS |0 AIRFLOW-1197 |Contrib Spark Submit Hook on_kill error |1 AIRFLOW-1193 |Add Checkr to Airflow user list |1 AIRFLOW-1191 |Contrib Spark Submit hook should permit override o|1 AIRFLOW-1189 |Get pandas DataFrame using BigQueryHook fails |1 AIRFLOW-1188 |Add max_bad_records param to GoogleCloudStorageToB|1 AIRFLOW-1187 |Obsolete package names in documentation |0 AIRFLOW-1185 |Incorrect url to PyPi |1 AIRFLOW-1182 |Contrib Spark Submit operator should template fiel|1 AIRFLOW-1181 |Enable delete and list function for Google Cloud S|1 AIRFLOW-1179 |Pandas 0.20 broke Google BigQuery hook |1 AIRFLOW-1175 |Add Pronto Tools to Airflow user list |1 AIRFLOW-1173 |Add Robinhood to list of Airflow users |1 AIRFLOW-1165 |airflow webservice crashes on ubuntu16 - python3 |0 AIRFLOW-1160 |Upadte SparkSubmitOperator parameters |1 AIRFLOW-1155 |Add Tails.com to community |1 AIRFLOW-1149 |Allow custom filters to be added to jinja2 |1 AIRFLOW-1141 |Remove DAG.crawl_for_tasks method |1 AIRFLOW-1140 |DatabricksSubmitRunOperator should template the "j|1 AIRFLOW-1136 |Invalid parameters are not captured for Sqoop oper|1 AIRFLOW-1125 |Clarify documentation regarding fernet_key |1 AIRFLOW-1122 |Node strokes are too thin for people with color vi|1 AIRFLOW-1121 |airflow webserver --pid no longer write out pid fi|0 AIRFLOW-1118 |Add evo.company to Airflow users |1 AIRFLOW-1112 |Log which pool is full in scheduler when pool slot|1 AIRFLOW-1107 |Add support for ftps non-default port |1 AIRFLOW-1106 |Add Groupalia/Letsbonus |1 AIRFLOW-1095 |ldap_auth memberOf should come from configuration |1 AIRFLOW-1094 |Invalid unit tests under `contrib/` |1 AIRFLOW-1091 |As a release manager I want to be able to compare |1 AIRFLOW-1090 |Add HBO |1 AIRFLOW-1089 |Add Spark application arguments to SparkSubmitOper|1 AIRFLOW-1081 |Task duration page is slow |1 AIRFLOW-1075 |Cleanup security docs |1 AIRFLOW-1065 |Add functionality for Azure Blob Storage |1 AIRFLOW-1059 |Reset_state_for_orphaned_task should operate in ba|1 AIRFLOW-1058 |Improvements for SparkSubmitOperator |0 AIRFLOW-1055 |airflow/jobs.py:create_dag_run() exception for @on|0 AIRFLOW-1051 |Add a test for resetdb to CliTests |1 AIRFLOW-1047 |Airflow logs vulnerable to XSS |1 AIRFLOW-1045 |Make log level configurable via airflow.cfg |1 AIRFLOW-1043 |Documentation issues for operators |1 AIRFLOW-1041 |DockerOperator replaces its xcom_push method with |1 AIRFLOW-1040 |Fix typos in comments/docstrings in models.py |1 AIRFLOW-1036 |Exponential backoff should use randomization |1 AIRFLOW-1035 |Exponential backoff retry logic should use 2 as ba|1 AIRFLOW-1034 |Make it possible to connect to S3 in sigv4 regions|1 AIRFLOW-1030 |HttpHook error when creating HttpSensor |0 AIRFLOW-1028 |Databricks Operator for Airflow |1 AIRFLOW-1024 |Handle CeleryExecutor errors gracefully |1 AIRFLOW-1019 |active_dagruns shouldn't include paused DAGs |0 AIRFLOW-1018 |Scheduler DAG processes can not log to stdout |0 AIRFLOW-1016 |Allow HTTP HEAD request method on HTTPSensor |1 AIRFLOW-1015 |TreeView displayed over task instances |0 AIRFLOW-1013 |airflow/jobs.py:manage_slas() exception for @once |0 AIRFLOW-1010 |Add a convenience script for signing |1 AIRFLOW-1009 |Remove SQLOperator from Concepts page |1 AIRFLOW-1007 |Jinja sandbox is vulnerable to RCE |1 AIRFLOW-1006 |Move configuration templates to separate files |1 AIRFLOW-1005 |Speed up Airflow startup time |1 AIRFLOW-999 |Support for Redis database |1 AIRFLOW-997 |Change setup.cfg to point to Apache instead of Max|1 AIRFLOW-995 |Update Github PR template |1 AIRFLOW-994 |Add MiNODES to the AIRFLOW Active Users List |1 AIRFLOW-991 |Mark_success while a task is running leads to fail|0 AIRFLOW-990 |DockerOperator fails when logging unicode string |1 AIRFLOW-985 |Extend the sqoop operator/hook with additional par|1 AIRFLOW-984 |Subdags unrecognized when subclassing SubDagOperat|1 AIRFLOW-979 |Add GovTech GDS |1 AIRFLOW-976 |Mark success running task causes it to fail |0 AIRFLOW-969 |Catch bad python_callable argument at DAG construc|1 AIRFLOW-963 |Some code examples are not rendered in the airflow|1 AIRFLOW-960 |Add support for .editorconfig |1 AIRFLOW-959 |.gitignore file is disorganized and incomplete |1 AIRFLOW-958 |Improve tooltip readability |1 AIRFLOW-950 |Missing AWS integrations on documentation::integra|1 AIRFLOW-947 |Make PrestoHook surface better messages when the P|1 AIRFLOW-945 |Revert psycopg2 workaround when psycopg2 2.7.1 is |0 AIRFLOW-943 |Add Digital First Media to the Airflow users list |1 AIRFLOW-942 |Add mytaxi to Airflow Users |1 AIRFLOW-935 |Impossible to use plugin executors |1 AIRFLOW-917 |Incorrectly formatted failure status message |1 AIRFLOW-916 |Fix ConfigParser deprecation warning |1 AIRFLOW-914 |Refactor BackfillJobTest.test_backfill_examples to|0 AIRFLOW-913 |Refactor tests.CoreTest.test_scheduler_job to real|0 AIRFLOW-912 |Refactor tests and build matrix |0 AIRFLOW-911 |Add colouring and profiling info on tests |1 AIRFLOW-903 |Add configuration setting for default DAG view. |1 AIRFLOW-896 |BigQueryOperator fails to execute with certain inp|1 AIRFLOW-889 |Minor error in the docstrings for BaseOperator. |1 AIRFLOW-888 |Operators should not push XComs by default |0 AIRFLOW-887 |Add compatibility with future v0.16 |1 AIRFLOW-886 |Pass Operator result to post_execute hook |1 AIRFLOW-885 |Add Change.org to the list of Airflow users |1 AIRFLOW-882 |Code example in docs has unnecessary DAG>>Operator|1 AIRFLOW-881 |Create SubDagOperator within DAG context manager w|1 AIRFLOW-880 |Fix remote log functionality inconsistencies for W|1 AIRFLOW-877 |GoogleCloudStorageDownloadOperator: template_ext c|1 AIRFLOW-875 |Allow HttpSensor params to be templated |1 AIRFLOW-871 |multiple places use logging.warn() instead of warn|1 AIRFLOW-866 |Add FTPSensor |1 AIRFLOW-863 |Example DAG start dates should be recent to avoid |1 AIRFLOW-862 |Add DaskExecutor |1 AIRFLOW-860 |Circular module dependency prevents loading of cus|0 AIRFLOW-854 |Add Open Knowledge International to Airflow users |1 AIRFLOW-842 |scheduler.clean_dirty raises warning: SAWarning: T|1 AIRFLOW-840 |Python3 encoding issue in Kerberos |1 AIRFLOW-836 |The paused and queryview endpoints are vulnerable |1 AIRFLOW-831 |Fix broken unit tests |1 AIRFLOW-830 |Plugin manager should log to debug, not info |0 AIRFLOW-829 |Reduce verbosity of successful Travis unit tests |0 AIRFLOW-828 |Add maximum size for XComs |0 AIRFLOW-826 |Add Zendesk Hook |1 AIRFLOW-825 |Add Dataflow semantics |0 AIRFLOW-823 |Make task instance details available via API |1 AIRFLOW-822 |Close the connection before throwing exception in |1 AIRFLOW-821 |Scheduler dagbag importing not Py3 compatible |1 AIRFLOW-809 |SqlAlchemy is_ ColumnOperator Causing Errors in MS|0 AIRFLOW-802 |Integration of spark-submit |0 AIRFLOW-788 |Context unexpectedly added to hive conf |0 AIRFLOW-781 |Allow DataFlowJavaOperator to accept jar file stor|1 AIRFLOW-770 |HDFS hooks should support alternative ways of gett|1 AIRFLOW-756 |Refactor ssh_hook and ssh_operator |0 AIRFLOW-751 |SFTP file transfer functionality |1 AIRFLOW-725 |Make merge tool use OS' keyring for password stora|1 AIRFLOW-706 |Configuration shell commands are not split properl|1 AIRFLOW-705 |airflow.configuration.run_command output does not |0 AIRFLOW-681 |homepage doc link should pointing to apache's repo|1 AIRFLOW-654 |SSL for AMQP w/ Celery(Executor) |1 AIRFLOW-645 |HttpHook ignores https |1 AIRFLOW-365 |Code view in subdag trigger exception |1 AIRFLOW-300 |Add Google Pubsub hook and operator |1 Cheers, Chris On Wed, Sep 13, 2017 at 2:01 PM, Chris Riccomini <[email protected]> wrote: > Note: Everything on master up to the point I cut the branch is included in > 1.9.0. I'm not sure what's up with the `dev/airflow-jira compare 1.9.0` > script.. I'm looking into that now. > > On Wed, Sep 13, 2017 at 1:55 PM, Alex Guziel <[email protected]. > invalid> wrote: > >> Nevermind, I misunderstood what you meant. (I thought you meant you were >> only including things with a fix version of 1.9.0, when you meant master >> cut + 1.9.0 fix versions) >> >> On Wed, Sep 13, 2017 at 1:19 PM, Alex Guziel <[email protected]> >> wrote: >> >> > Shouldn't we include everything on master? >> > >> > On Wed, Sep 13, 2017 at 12:45 PM, Chris Riccomini < >> [email protected]> >> > wrote: >> > >> >> Hey all, >> >> >> >> I've cut a 1.9.0 test branch. >> >> >> >> https://github.com/apache/incubator-airflow/tree/v1-9-test >> >> >> >> Here are the tickets that are being tracked on 1.9.0. >> >> >> >> ISSUE ID |DESCRIPTION |MERGED >> >> AIRFLOW-1608 |GCP Dataflow hook missing pending job state |1 >> >> AIRFLOW-1606 |DAG.sync_to_db is static, but takes a DAG as first|1 >> >> AIRFLOW-1605 |Fix log source of local loggers |0 >> >> AIRFLOW-1602 |Use LoggingMixin for the DAG class |1 >> >> AIRFLOW-1597 |Add GameWisp as Airflow user |1 >> >> AIRFLOW-1594 |Installing via pip copies test files into python l|1 >> >> AIRFLOW-1593 |Expose load_string in WasbHook |1 >> >> AIRFLOW-1586 |MySQL to GCS to BigQuery fails for tables with dat|1 >> >> AIRFLOW-1584 |Remove the insecure /headers endpoints |1 >> >> AIRFLOW-1582 |Improve logging structure of Airflow |1 >> >> AIRFLOW-1580 |Error in string formatter when throwing an excepti|1 >> >> AIRFLOW-1579 |Allow jagged rows in BQ Hook. |1 >> >> AIRFLOW-1577 |Add token support to DatabricksHook |1 >> >> AIRFLOW-1573 |Remove `thrift < 0.10.0` requirement |1 >> >> AIRFLOW-1568 |Add datastore import/export operator |1 >> >> AIRFLOW-1567 |Clean up ML Engine operators |1 >> >> AIRFLOW-1564 |Default logging filename contains a colon |1 >> >> AIRFLOW-1556 |BigQueryBaseCursor should support SQL parameters |1 >> >> AIRFLOW-1546 | add Zymergen to org list in README |1 >> >> AIRFLOW-1535 |Add support for Dataproc serviceAccountScopes in D|1 >> >> AIRFLOW-1529 |Support quoted newlines in Google BigQuery load jo|1 >> >> AIRFLOW-1522 |Increase size of val column for variable table in |1 >> >> AIRFLOW-1521 |Template fields definition for bigquery_table_dele|1 >> >> AIRFLOW-1507 |Make src, dst and bucket parameters as templated i|1 >> >> AIRFLOW-1505 |Document when Jinja substitution occurs |1 >> >> AIRFLOW-1504 |Log Cluster Name on Dataproc Operator When Execute|1 >> >> AIRFLOW-1499ss|Eliminate duplicate and unneeded code |0 >> >> AIRFLOW-1493 |Fix race condition with airflow run |1 >> >> AIRFLOW-1492 |Add metric for task success/failure |1 >> >> AIRFLOW-1489 |Docs: Typo in BigQueryCheckOperator |1 >> >> AIRFLOW-1478 |Chart -> Owner column should be sortable |1 >> >> AIRFLOW-1476 |Add INSTALL file for source releases |1 >> >> AIRFLOW-1474 |Add dag_id regex for 'airflow clear' CLI command |1 >> >> AIRFLOW-1459 |integration rst doc is broken in github view |1 >> >> AIRFLOW-1438 |Scheduler batch queries should have a limit |1 >> >> AIRFLOW-1437 |BigQueryTableDeleteOperator should define deletion|1 >> >> AIRFLOW-1402 |Cleanup SafeConfigParser DeprecationWarning |1 >> >> AIRFLOW-1401 |Standardize GCP project, region, and zone argument|1 >> >> AIRFLOW-1394 |Add quote_character parameter to GoogleCloudStorag|1 >> >> AIRFLOW-1389 |BigQueryOperator should support `createDisposition|1 >> >> AIRFLOW-1384 |Add ARGO/CaDC |1 >> >> AIRFLOW-1359 |Provide GoogleCloudML operator for model evaluatio|1 >> >> AIRFLOW-1352 |Revert bad logging Handler |0 >> >> AIRFLOW-1350 |Add "query_uri" parameter for Google DataProc oper|1 >> >> AIRFLOW-1345 |Don't commit on each loop |1 >> >> AIRFLOW-1344 |Builds failing on Python 3.5 with AttributeError |1 >> >> AIRFLOW-1343 |Add airflow default label to the dataproc operator|1 >> >> AIRFLOW-1338 |gcp_dataflow_hook is incompatible with the recent |1 >> >> AIRFLOW-1337 |Customize log format via config file |1 >> >> AIRFLOW-1335 |Use buffered logger |1 >> >> >> >> If you have stuff you want to get in, please set it with a fix version >> of >> >> 1.9.0. >> >> >> >> Please begin testing, stabilizing, and reporting bugs now! :) >> >> >> >> Cheers, >> >> Chris >> >> >> > >> > >> > >
