Thanks for doing all this, Max! When you're ready, I can deploy to some of our environments to verify. Just let me know.
On Thu, Jun 8, 2017 at 8:42 AM, Maxime Beauchemin < [email protected]> wrote: > I branched off `v1-8-test` so all should be good. I just didn't know if I > could move ahead with that branch just yet so I branched off. I just got > back on `v1-8-test` and pushed what I have to Apache [somehow I had to > rebase meaning someone added something over the past 24 hours]. > > I just set `Fix Version` of AIRFLOW-1294 to 1.8.2 and set it to blocker. > > I cherry-picked all the commits targeting 1.8.2 I could without getting > into conflicts. I'd only work at resolving conflicts on blockers. My plan > is to add only the 2 rows in red for RC1. > > Here's the current output of `airflow-jira 1.8.2`: > > ISSUE ID |TYPE ||PRIORITY ||STATUS |DESCRIPTION > |MERGED|PR |COMMITAIRFLOW-1294 > |Bug ||Blocker ||Open |Backfills can loose tasks to > execute due to tasks |0 |- |- > AIRFLOW-1291 |Bug ||Blocker ||Open |Update > NOTICE and LICENSE files to meet ASF specs |0 |- |- > AIRFLOW-1290 |Bug ||Major ||Resolved |Change docs > author from "Maxime Beauchemin" to "Ap|1 |#na > |fc33c040605e7b0121739af841790336bdfa3948 > AIRFLOW-1282 |Bug ||Major ||Resolved |Fix known > event column sorting |1 |#2350 > |2e9eee3697f6261de61c77ba584ace14b22751f0 > AIRFLOW-1281 |Bug ||Minor ||Resolved |Sort variables > by key field by default |1 |#2347 > |57d5bcdad4202fd8a94be07b4aeacee065801481 > AIRFLOW-1275 |Bug ||Major ||In Progress|Fix `airflow > pool` command exception |0 |- |- > AIRFLOW-1274 |Bug ||Major ||Open |HttpSensor > parameter params is overriding BaseOper|0 |- |- > AIRFLOW-1266 |Bug ||Major ||Open |Long task > names are truncated in gannt view |1 |#2345 > |7f33f6e2fe4149e547860fdb237da26a5b7023ec > AIRFLOW-1263 |Bug ||Major ||Resolved |Airflow Charts > Pages Should Have Dynamic Heights |1 |#2344 > |acd0166d8c7f5639898791848d985f6441b3b5b5 > AIRFLOW-1244 |Bug ||Major ||Resolved |Forbid > creation of a pool with empty name |1 |#2324 > |802fc1549d0f173c646794f2476156d34383adc0 > AIRFLOW-1243 |Bug ||Minor ||Resolved |DAGs table has > no default entries to show |1 |#2323 > |1232b6a4fd691e0faf081d4cf792a6d96d33b595 > AIRFLOW-1227 |Bug ||Minor ||Resolved |Remove empty > column on the Logs view |1 |#2310 > |b0ba3c91044b2abd92289d9b3f7d913060ef6c31 > AIRFLOW-1226 |Bug ||Minor ||Resolved |Remove empty > column on the Jobs view |1 |#2309 > |c406652bd4c8f8099841c69f2d1ccc9678d77bfd > AIRFLOW-1221 |Bug ||Major ||Resolved |Fix > DatabricksSubmitRunOperator Templating |0 |- |- > AIRFLOW-1217 |Bug ||Major ||Resolved |Enable logging > in Sqoop hook |0 |- |- > AIRFLOW-1213 |Bug ||Major ||Resolved |Add hcatalog > parameters to the sqoop operator/hook|0 |- |- > AIRFLOW-1203 |Bug ||Major ||Closed |Tests failing > after oauth upgrade |0 |- |- > AIRFLOW-1201 |Bug ||Major ||Resolved |Update > deprecated 'nose-parameterized' library to |0 |- |- > AIRFLOW-1200 |Bug ||Major ||Resolved |Forbid > creation of a variable with an empty key |1 |#2299 > |cac7d4cb571a4c1f7ba53360765ea6c640c719df > AIRFLOW-1199 |Bug ||Major ||Resolved |Create modal > looks like edit modal (edit tab selec|1 |#2293 > |51a83cce96d41eb1b9df9f82188c36c7e12061df > AIRFLOW-1197 |Bug ||Major ||Open |Contrib Spark > Submit Hook on_kill error |0 |- |- > AIRFLOW-1192 |New Feature ||Major ||Resolved |Add support to > run Zeppelin Notebooks with Qubole |1 |#2322 > |bf966e63be2deb19fa2824efdfe854736318f1b5 > AIRFLOW-1191 |Improvement ||Major ||Open |Contrib Spark > Submit hook should permit override o|0 |- |- > AIRFLOW-1186 |Bug ||Major ||Resolved |Task duration > shows weird graph for some dags |1 |#2284 > |5f3ee52da7ca4f1fca8a84b3281b31aa6241514e > AIRFLOW-1180 |Bug ||Major ||Resolved |Fix flask-wtf > version in setup.py for test_csrf_re|1 |#2280 > |93b8e9643cc273ec0ecc063b9f851d9a8012cd03 > AIRFLOW-1179 |Bug ||Major ||Closed |Pandas 0.20 > broke Google BigQuery hook |1 |#2279 > |c7c0b72d934fec10d890f7a3bd8fd06c0e2f44bd > AIRFLOW-1170 |Bug ||Major ||Resolved |DbApiHook > insert rows does not use prepared statem|1 |#2270 > |bb56805e3801c7f8d49acd8095fd3099609d6fe5 > AIRFLOW-1168 |Bug ||Major ||Resolved |DbApiHook does > not close connections when an excep|1 |#2269 > |777f181f81fa6a383a79b71fdcb9c4f4e77c785a > AIRFLOW-1167 |Bug ||Major ||Resolved |FTPHook file > modification date does not support op|1 |#2268 > |272c2f52142c167685646169431eb9dab85b7810 > AIRFLOW-1166 |Bug ||Major ||Resolved > |_change_state_for_tis_without_dagrun can take very|1 |#2267 > |2389a8a7e89ea949e8f9674a3d3b18d2142dfece > AIRFLOW-1160 |Bug ||Major ||Resolved |Upadte > SparkSubmitOperator parameters |1 |#2265 > |69bd26913824c5c796a0a828a5c43ab39d0902f8 > AIRFLOW-1150 |Bug ||Major ||Resolved |Fix scripts > execution in SparkSql hook |1 |#2259 > |093b2f04f7a010107fe6dfbc0658a53a1b8467e6 > AIRFLOW-1149 |Bug ||Major ||Resolved |Allow custom > filters to be added to jinja2 |1 |#2258 > |9692510c4e067d0e81fd8d5893bfb4a0e53941e2 > AIRFLOW-1145 |Bug ||Minor ||Resolved > |Closest_date_partition not working with before = T|1 |#2257 > |f446c0883cab30482436c47b867c4a440412f1ff > AIRFLOW-1136 |Bug ||Minor ||Resolved |Invalid > parameters are not captured for Sqoop oper|0 |- |- > AIRFLOW-1119 |Bug ||Major ||Resolved |Redshift to S3 > operator - headers not on first row|1 |#2245 > |6de5330d5336fb2455eea75c78e95f126320a5b0 > AIRFLOW-1089 |Improvement ||Major ||Resolved |Add Spark > application arguments to SparkSubmitOper|1 |#2229 > |b4e9eb8310957cac47f6964662ce16496061db2c > AIRFLOW-1081 |Bug ||Major ||Resolved |Task duration > page is slow |0 |- |- > AIRFLOW-1078 |Bug ||Major ||Resolved |Latest_runs > endpoint broken in old flask versions |1 |#2224 > |a4083f367b1f0f6525bcd011122d854626ea6768 > AIRFLOW-1074 |Bug ||Major ||Open |Do not count > queued tasks in scheduler concurrency|1 |#2221 > |7a0284149a1dde4dc1d65a1d016eadf619e5320e > AIRFLOW-1064 |Improvement ||Major ||Resolved > |TaskInstanceModelView is slow |1 |#2215 > |a2c18a58d0ba14101ddb70403fb95feda73cf21a > AIRFLOW-1047 |Bug ||Major ||Resolved |Airflow logs > vulnerable to XSS |0 |- |- > AIRFLOW-1041 |Bug ||Major ||Resolved |DockerOperator > replaces its xcom_push method with |0 |- |- > AIRFLOW-1038 |Bug ||Major ||Resolved |Specify celery > serializers explicitly and pin vers|1 |#na > |d1c64ab7b4ee6215900a15849e890ed1b408f5f5 > AIRFLOW-1036 |Improvement ||Major ||Resolved |Exponential > backoff should use randomization |1 |#2262 > |b4ee88aa6d71145daf91fc71763950ede16f8383 > AIRFLOW-1007 |Bug ||Major ||Resolved |Jinja sandbox > is vulnerable to RCE |0 |- |- > AIRFLOW-993 |Bug ||Minor ||Resolved |Dags should > modify the start date and end date of |1 |#2157 > |9fca409cebfd04ce8580c4657099a6db340eb7eb > AIRFLOW-945 |Bug ||Major ||Resolved |Revert > psycopg2 workaround when psycopg2 2.7.1 is |0 |- |- > AIRFLOW-935 |Bug ||Major ||Resolved |Impossible to > use plugin executors |0 |- |- > AIRFLOW-860 |Bug ||Major ||Resolved |Circular > module dependency prevents loading of cus|0 |- |- > > > Max > > On Thu, Jun 8, 2017 at 12:18 AM, Bolke de Bruin <[email protected]> wrote: > > > Hi Max, > > > > Sounds good. Couple of things: > > > > * Can I suggest using the v1-8-test branch as the branch to be used for > > preparing the rc? If we hit RC then move it over to v1-8-stable? > V1-8-test > > already had some fixes in that should land in 1.8.2 and the RC should be > > tagged in the stable branch. That also reduces to amount of merge > conflicts > > probably as many have been merged. Where did you branch off from? > Anyways, > > see also the release management thing on the wiki. > > > > Blocker(!) > > * In the backfills we can loose tasks to execute due to a task setting > its > > own state to NONE if concurrency limits are reached, this makes them fall > > outside of the scope the backfill is managing hence they will not be > > executed (https://issues.apache.org/jira/browse/AIRFLOW-1294 < > > https://issues.apache.org/jira/browse/AIRFLOW-1294>). Setting itself to > > NONE should probably be “CONCURRENCY_REACHED” (new state). I have marked > it > > as a blocker as we had multiple people hitting the issue, but I need 1-2 > > days to get a patch. Feel free to downgrade to critical if you like :). > > > > Cheers > > Bolke > > > > > > > On 8 Jun 2017, at 02:35, Maxime Beauchemin <[email protected] > > > > wrote: > > > > > > What a pleasant, mind numbing afternoon doing some release management > > > > > > Notes: > > > * Added a warning that the package name has changed on Pypi > > > <https://pypi.python.org/pypi/airflow> > > > * Removed references to my name here > > > <https://github.com/apache/incubator-airflow/pull/2352> and merged > > > * Addressed John D. Ament's concerns here > > > <https://github.com/apache/incubator-airflow/pull/2354>, please > review! > > > * "footable" appears to have been removed, not a problem anymore > > > * that `airflow-jira is a god send! thanks Bolke. > > > * reviewed list of Airbnb's production cherries and flagged those as > `Fix > > > Version == 1.8.2` > > > * Started branch v1-8-2.rc1 and started picking cherries using > > > `airflow-jira compare 1.8.2` > > > > > > I'll finish going through picking everything that targeted 1.8.2 that > > does > > > not create merge conflict. > > > > > > If there's anything flagged as "blocker" that generates merge conflict, > > > I'll go case by case about it. > > > > > > Soon after, I should be able to announce 1.8.2 RC1, hopefully sometime > > > tomorrow or Friday. > > > > > > Let me know if there's anything else I'm missing that I should > consider. > > > > > > Cheers! > > > > > > Max > > > > >
