Re: [openstack-dev] [sahara] PTG Summary

[Jeremy Freudberg]

Thanks Luigi and Telles for striving to work hard, even in my absence. :)
Some notes inline.

On Mon, Mar 12, 2018 at 10:39 AM, Telles Nobrega 
wrote:

> Hello Saharans and interested folks,
>
> This PTG was a very interesting experience, for those not familiar with
> what happened there, here goes a quick summary.
>
> The event became known as SnowOpenStack PTG due to some snow that got in
> the way of event. But that didn't get in the way of determined people that
> wanted to do good work anyways.
> This PTG for Sahara was different, we only had 2 people present so a few
> topics couldn't be fully discussed.
>
> We started our week as Sahara joining the Storyboard room in order to
> understand better their status and what are the needed steps for us to
> migrate from launchpad to storyboard.
> The outcome of that meeting was better than expected. There are only a few
> things needed from our side to fully migrate. Tosky got ahead on this and
> already updated the migration script to support sahara and other projects
> needs and tested the migration with success. Now we only need to wait on
> the storyboard team to test it as well and then we need to document that we
> are migrating to storyboard.
> Action item on this for all involved in the project: Take some time to
> read on Storyboard and get familiar with it.
>
> Than we started Sahara meetings:
>
> We started with the traditional retrospective, we looked over the work
> list from Queens, marking them with Done, Partially Done and Unstarted.
> Please all take a look at that and let me know if anything not prioritized
> from that list is necessary.
>
> Now to specific discussion topics.
>
> Plugins upgrades, deprecation and removals
> 
>
> CDH:
>
>- Upgrade to 5.12, 5.13 and 5.14
>- Deprecate 5.7, 5.9 if we are able to upgrade all three version or
>two of them
>- Remove 5.5.0
>
> MapR:
>
>- Upgrade to 6.0 and update latest packages for 5.2
>
> Looks like MapR 6 offers some new management services which we should
acknowledge and add.

>
>-
>- Remove all references to 5.1
>
> Ambari:
>
>- Upgrade Ambari to 2.6 and HDP to 2.6 as well
>- Use the Ambari 2.6 image with both HDP 2.6 and 2.5 and 2.4
>- Deprecate HDP version 2.3 and work the removal of ambari 2.4 for S
>
> Spark:
>
>- Upgrade to 2.3.0 and check 2.2.1 as subversion as 2.2.0
>- Deprecate 1.6, 2.1 (see move out of trusty discussion)
>- Remove 1.3
>
> Storm:
>
>- Upgrade to 1.2.1 and 1.2.0 (under the same tag 1.2)
>- Deprecate 1.0.1
>- Remove 0.9.2
>
> Vanilla:
>
>- Upgrade to vanilla 3.0.0 and check if we can add 2.7.5
>
> 2.7.5 seems much lower priority to me. Even 2.8.3 or the eventual 2.8.4
seems more important. But I'm not sure.


>
> Sahara CI
> -
> We are still facing issues with our third-party CI. We plan to have
> nightly jobs running each day for a plugin so we won't need too much
> resources, that seems possible with zuulv3.
> Also there is an issue with experimental queue, now we run all
> experimental jobs even when not necessary, we plan to split this in
> separate queues so we can run specific tests.
>

It looks like the wheels of progress are starting to turn on my end...


>
> Python 3 support
> ---
> Python 2 support is closing and we need to fully migrate to Python 3.
> Right now we have unit tests in place, tempest should be easily resolved;
> Scenarios jobs are there but failing with swift issues. Once we have all of
> those we will have a better grasp of what we need to do and how much work
> will be. In any case, we need to get a jump on these soon.
>
> SSL/TLS everywhere
> 
> We need to check our status of secure communication with other projects as
> well as between Sahara and its clusters. Right now Sahara should be able to
> communicate with CDH using SSL but it is hard coded and we need to change
> that and test how it goes. Also we need to check certificate management.
> For Ambari, CA is generated but not reconized on RHEL/CentOS>=7.4, we need
> to make sure this works.
>
> Move out of trusty
> 
> Trusty support will be dropped soon and we need to make sure we don't have
> any dependencies on our side. For that we need to work on removing plugins
> versions that are only supported on trusty.
>
> Plugins outside Sahara
> ---
> After discussion with Doug Hellman, seems like we have a good plan to have
> this finally done. The goal is the have a new project (sahara-plugins) that
> will require stuff from sahara and sahara itself will load plugins from
> sahara-plugins project dinamically with stevedore, this way we don't have a
> circular dependency.
> The bulk of the work is done, spec should be on its way soon.
>

+1


>

> APIv2
> 
> We have it available as experimental, but we miss CLI, tempest 

Re: [openstack-dev] [sahara] PTG Summary

[Jeremy Freudberg]

Thanks Telles - we all appreciate your leadership which was conducive to a
very productive PTG.

Another exciting point to highlight is more community visibility and
involvement regarding Sahara's plugins:
1) Plugin upgrades tied to a release milestone visible on the official
release schedule
2) Collecting community and operator feedback about what new services
Sahara should support

Again, thanks to all involved and hope to see you all at next PTG.

On Wed, Sep 20, 2017 at 10:00 AM, Telles Nobrega 
wrote:

> Hello Saharan's and interested folks, this past week we had our PTG at
> Denver and it was a very productive one, with so many discussions and too
> much to do after that. In the next lines I will describe a little of the
> main decisions made by the Sahara team.
>
> Pike Retrospective
> -
> We started our PTG with a retrospective from the previous cycle. Since
> there were only sahara members on the room and we basically walked through
> the good and bads from the cycle.
>
> On the bad side:
>  - Losing too many community members
>  - First PTL cycle was a little overwhelming
>  - We had too much errors not caught by tests
>  - We lost our third-party CI
> On the good side:
>  - Most of our priorities was finished
>  - Even with minimum hands we managed to get plugins updated and deliver a
> good product
>  - We met the documentation in project goal
>
> Documentation
> 
> Alongside the community with the goal of moving documentation into project
> tree we were able to move a lot of our documentation into tree with just
> details left for Queens cycle.
> Also, we plan to do a Documentation day in order to update and improve our
> documentation.
>
> APIv2
> 
> This feature has been around for quite some time and in Pike we had a good
> advance in its implementation. We were able to do most of the work needed
> for its release but there are a few steps left. Our plan is to finish up
> all these steps in Queens and release APIv2 as experimental (at least).
>
> Sahara-CI
> --
> During the Pike cycle we lost our third-party CI. In Queens we need to
> gather resources to deploy a new CI as well as improve our CI in infra with
> vanilla multinode jobs. We plan to introduce some nightly jobs for more
> exhaustive testing with large files.
> We are also working on automating the CI deployment with ansible.
>
> Sahara-files
> 
> Just as Sahara-CI we might be losing our sahara files hosts soon and we
> need to remove all references of it from code (which is almost done) and
> copy the files for a different source.
>
> Specs cleanup and prioritization
> ---
> We took some time to review old specs and do a little cleanup of our
> queues. On that we moved some specs into backlog and added some new as well
> as fixed some typos on older specs.
>
> New features and bugs
> ---
> We discussed new features and bugs and we have a priority list for the
> work we plan to do in Queens and possible for next two cycles. If you are
> interested in a deeper look into planned features and bugs take a look at
> [0]
>
> Community-goals
> 
> For Queens there are two community goals. The first one is related to
> Split tempest plugin and this was done in Sahara a couple cycles ago. The
> second is regarding policy in code which we already have a patch (WIP) up
> [1] and should get it done soon.
>
> Thanks all!
>
> [0] https://etherpad.openstack.org/p/sahara-queens-ptg
> [1] https://review.openstack.org/#/c/503221/
>
>
>
> --
>
> TELLES NOBREGA
>
> SOFTWARE ENGINEER
>
> Red Hat I 
>
> tenob...@redhat.com
> 
> TRIED. TESTED. TRUSTED. 
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev