leahecole commented on a change in pull request #274: URL: https://github.com/apache/airflow-site/pull/274#discussion_r462603973
########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. Review comment: Consider adjusting the last part of this sentence to explain why you were able to use it in no time - perhaps something like "...before this realization, but thankfully, Marshmallow 3 isn't too different, so I was able to start using Marshmallow 2 in no time." You can swap out "isn't too different" with whatever reason is true for you - "the docs are detailed" "my mentor helped" or whatever it is 😄 ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. Review comment: add a link to Marshmallow 2 - especially because the word Marshmallow is hard to search (you get lots of food results 😆 - it made me hungry!) ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, Review comment: grammar nit - unless "Extend and Improve" are part of a formal title they should be lowercase - "extend and improve" If it's the formal title of the project though, that's ok! ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating Review comment: possible typo - connexion -> connection ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating Review comment: or this may be a library - in which case, `code` emphasis and a link would help ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with +Apache 2.0 licence. + +I implemented a workaround on this, by defining a function called `conn_parse_datetime` in the +API utils module. This was later refactored and thankfully, [Kamil](https://github.com/mik-laj) + implemented a decorator that allowed us to have cleaner code on the views while using this function. + +We later tried using `rfc3339-validator` whose licence is compatible with Apache 2.0 licence but + later discarded this because with our custom date parser we were able to use duration and + not just date times. + +### Other Endpoints +I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2 +not giving error when extra fields are in the request body. I implemented a `validate_unknown` +method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3, +we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too. + +Here are some PRs I contributed that are related to the REST API: Review comment: This list is cool! Would you consider adding what the impact these PRs had on other developers/users? What did it fix/make better/easier/more transparent? ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with +Apache 2.0 licence. + +I implemented a workaround on this, by defining a function called `conn_parse_datetime` in the +API utils module. This was later refactored and thankfully, [Kamil](https://github.com/mik-laj) + implemented a decorator that allowed us to have cleaner code on the views while using this function. + +We later tried using `rfc3339-validator` whose licence is compatible with Apache 2.0 licence but + later discarded this because with our custom date parser we were able to use duration and + not just date times. + +### Other Endpoints +I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2 Review comment: Yay I love hearing about particular issues and how folks learned from them!! ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. Review comment: are you able to link to the draft PR? I think readers would find it interesting to see the inspiration alongside your end product! (which, thank you for linking to your pull above!) ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that Review comment: Just double checking grammar here - is "parametrized" a particular library? (I genuinely don't know) - if not, did you mean "parameters"? If it's a library, you might want to use the `code` emphasis you used later in the post ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. Review comment: grammar nit - use either "most" or "especially" - together they are redundant ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have Review comment: grammar nits - tense agreement - * means -> meant * have been -> had been ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with +Apache 2.0 licence. + +I implemented a workaround on this, by defining a function called `conn_parse_datetime` in the +API utils module. This was later refactored and thankfully, [Kamil](https://github.com/mik-laj) + implemented a decorator that allowed us to have cleaner code on the views while using this function. + +We later tried using `rfc3339-validator` whose licence is compatible with Apache 2.0 licence but + later discarded this because with our custom date parser we were able to use duration and + not just date times. + +### Other Endpoints +I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2 Review comment: Out of curiosity is this something that was fixed in Marshmallow 3? ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with +Apache 2.0 licence. + +I implemented a workaround on this, by defining a function called `conn_parse_datetime` in the +API utils module. This was later refactored and thankfully, [Kamil](https://github.com/mik-laj) + implemented a decorator that allowed us to have cleaner code on the views while using this function. + +We later tried using `rfc3339-validator` whose licence is compatible with Apache 2.0 licence but + later discarded this because with our custom date parser we were able to use duration and + not just date times. + +### Other Endpoints +I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2 +not giving error when extra fields are in the request body. I implemented a `validate_unknown` +method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3, +we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too. + +Here are some PRs I contributed that are related to the REST API: + + 1. [Add event log endpoints](https://github.com/apache/airflow/pull/9227) + 2. [Add CRUD endpoints for connection](https://github.com/apache/airflow/pull/9266) + 3. [Add log endpoint](https://github.com/apache/airflow/pull/9331) + 4. [Move limit & offset to kwargs in views plus work on a configurable maximum limit](https://github.com/apache/airflow/pull/9431) + 5. [Update FlaskAppBuilder to v3](https://github.com/apache/airflow/pull/9648) + 6. [Add migration guide from the experimental REST API to the stable REST API](https://github.com/apache/airflow/pull/9771) + +### Follow-Ups +There are still lots of works to be done on the REST API including writing helpful documentation. Review comment: Grammar nit - change to "There is still lots of work" ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with Review comment: (happens twice) ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with +Apache 2.0 licence. + +I implemented a workaround on this, by defining a function called `conn_parse_datetime` in the +API utils module. This was later refactored and thankfully, [Kamil](https://github.com/mik-laj) + implemented a decorator that allowed us to have cleaner code on the views while using this function. + +We later tried using `rfc3339-validator` whose licence is compatible with Apache 2.0 licence but Review comment: another licence -> license (license is one of those horrible words for spelling that I always feel never looks right no matter which way you spell it!! 😄 that and vacuum) ########## File path: landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md ########## @@ -0,0 +1,75 @@ +--- +title: "Implementing Stable API for Apache Airflow" +linkTitle: "Implementing Stable API for Apache Airflow" +author: "Ephraim Anierobi" +twitter: "ephraimbuddy" +github: "ephraimbuddy" +description: "An Outreachy intern's progress report on contributing to Apache Airflow REST API." +tags: ["REST API"] +date: "2020-07-19" +--- + +My [Outreachy internship](https://outreachy.org) is coming to its ends which is also the best time to look back and +reflect on the progress so far. + +The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post, +I will be sharing my progress so far. + +We started a bit late implementing the REST API because it took time for the OpenAPI 3.0 +specification we were to use for the project to be merged. Thanks to [Kamil](https://github.com/mik-laj), +who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I +implemented and the challenges I encountered, including how I overcame them. + +### Implementing The Read-Only Connection Endpoints +The [read-only connection endpoints](https://github.com/apache/airflow/pull/9095) were the first endpoint I implemented. Looking back, +I can see how much I have improved. + +I started by implementing the database schema for the Connection table using Marshmallow 2. +We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder +is deeply integrated to Apache Airflow. This means I had to unlearn Marshmallow 3 that I have + been studying before this realization and thankfully, I started using Marshmallow 2 in no time. + +This first PR would have been more difficult than it was had it been there was no reference +endpoint to look at. [Kamil](https://github.com/mik-laj) implemented a draft PR in which I took inspiration from. +Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that + I learned to use parameterized in unit tests :D. + +### Implementing The Read-Only DagRuns Endpoints + +This [endpoint](https://github.com/apache/airflow/pull/9153) came with its many challenges, most especially on filtering with `datetimes`. +This was because the connexion library we were using to build the REST API was not validating +date-time format in OpenAPI 3.0 specification. This I would later find out, was intentional. +Connexion dropped `strict-rfc3339` because of the later licence which is not compatible with Review comment: typo - licence -> license ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
