Hi, Alex! Good to know someone is doing that. One question about Your initiative: Are You going to reimplement all MOS HA features? Seems first implementation of 'upstream on Fuel' will be HA-less.
On Thu, Nov 12, 2015 at 1:51 AM, Alex Schultz <aschu...@mirantis.com> wrote: > On Tue, Nov 10, 2015 at 11:10 AM, Vladimir Kuklin <vkuk...@mirantis.com> > wrote: > > Alex > > > > Thanks for your very detailed answer - it clarified things a bit. So, if > you > > would allow me to rephrase it - you are actually conducting a research on > > what is the actual gap between our downstream/fork and upstream > > UCA/puppet-openstack. This seems to be a very promising initiative. Are > you > > going to come up with some external-user readable report soon? > > > > Also, regarding multiple distros support. I think, we need to come up > with > > an approach of making 'release manager' piece of Nailgun data driven and > > just allow a user to run any distribution he or she wants. Just register > a > > set of repos with packages and run it. Actually, we already have it - we > > need to make base-image generation for RPM more flexible and any RPM/DEB > > based distro should work ok with it. > > > > The remaining piece is to actually support distro-specific things, e.g. > > CentOS/RHEL networking configuration, e.g. l23 network stored config > puppet > > providers. But this is a distro-supporter/community burden. > > > > > > Yes I hope to have something together by the end of the week. I've > managed to get a controller and compute/cinder nodes up (and passing > basic OSTF tests) with what appears to be some minor adjustments to > the fuel-library code. The one thing that gets dropped because of > lack of upstream support is nova floating network ranges. There's a > pending review that'll get that back in but I also don't know how > important it would be to support for this type of configuration. > Another issue is the upstream murano module is still a work in > progress so that won't work right now either. Hopefully that'll get > sorted out in time for the official release of the liberty puppet > modules. > > As I've been working through this, I've noticed that it would be > possible to use fuel-plugins to only apply UCA packages to specific > nodes via a plugin role. An interesting follow on to this effort would > be to use MOS packages for controllers and UCA for Compute or vice > versa. But that should probably be more an academic exercise rather > than production one. > > -Alex > > > > > On Tue, Nov 10, 2015 at 6:25 PM, Alex Schultz <aschu...@mirantis.com> > wrote: > >> > >> Hey Vladimir, > >> > >> On Tue, Nov 10, 2015 at 5:56 AM, Vladimir Kuklin <vkuk...@mirantis.com> > >> wrote: > >> > Alex > >> > > >> > That's great to hear that. But be aware - making all of the components > >> > work > >> > exactly the way they work within MOS is actually identical to > >> > upstreaming > >> > MOS. We are using some components of different versions to satisfy > many > >> > requirements for our Reference Architecutre implementation. It will > not > >> > be > >> > so easy to base them upon existing 3rd party distributions. For > example, > >> > `read timeout` for SQL requests is crucial for our HA as it handles > >> > cases > >> > when an SQL node dies while serving the request. And you will find > >> > myriads > >> > of them. And as we do not control things in upstream, we will always > >> > have > >> > some downstream divergence. > >> > > >> > >> Yes, I'm aware that it'll be an effort to make it work identically to > >> MOS. Currently that's not my goal. My goal is to get it working at > >> all and be able to document the deficiencies when using upstream > >> packages/modules vs MOS provided ones. Once we have documented these > >> differences we will be able to make decisions as to what efforts > >> should be made if we choose to address the differences. The read > >> timeout thing seems to be an issue with what mysql python driver is > >> used so that could easily be configurable based on a packages or a > >> configuration option. > >> > >> > I guess, the optimal solution is to figure out the actual divergence > >> > between > >> > upstream and downstream and try to push things upstream as hard as we > >> > can, > >> > while retaining overrides for some packages and manifests on top of > >> > upstream > >> > versions. Do not get me wrong, but it seems there is exactly 0 (zero) > >> > ways > >> > you can get Fuel working with upstream packages unless they support > >> > exactly > >> > the same feature set and fix the same bugs in various components that > >> > Fuel > >> > expect them to support or fix. By 'working' I mean passing the same > set > >> > of > >> > at least smoke and destructive tests, let alone passing scale tests. > >> > > >> > >> So I think this is where we are currently backwards in the way we're > >> doings. As we hope to push Fuel as a community project, we need to be > >> more open to supporting the distributions versions of these packages > >> as being supported. If we want to continue with specific versions of > >> things we need to be able to modularize them and apply them to a > >> downstream version of Fuel that we can promote with the MOS package > >> set. I agree that right now it's highly unlikely that one would be > >> able to use Fuel without any MOS packages. Because I don't think it's > >> possible right now, what I'm attempting to do is be able to deploy an > >> alternate OpenStack package set but not all of the other pieces as > >> they relate to our reference architecture. That seems to be a more > >> achievable goal right now and allows us to document the gaps where > >> Fuel/MOS are a head (or behind depending on your views) with upstream. > >> > >> > Simon > >> > > >> > AFAIK, this patch is really simple and can be easily applied to any > >> > version > >> > of HAProxy. So it is not too hard handle it while it provides > sufficient > >> > benefits to our deployment engineers. If you have any better solution, > >> > please feel free to share the code with us. > >> > > >> > >> Unfortunately the haproxy fork that we're currently running with also > >> includes our forked version of the puppet haproxy module. I'm not sure > >> the includes configs patch is really worth carrying the additional > >> work/forks but I think that is a conversation for another day. > >> Personally I think it would better if fuel-library supported the > >> currently available upstream versions of haproxy & haproxy puppet > >> module and if we want to continue using our copies of haproxy & puppet > >> haproxy module that we turn that into a downstream that is applied for > >> the MOS specific version of Fuel. That's the advantage of the > >> Puppetfile we've been working on. We can change out the modules with > >> our own version downstream. > >> > >> Just to add some additional updates issues that I've run into. > >> > >> - nova-vncproxy package names (nova-consoleproxy vs nova-novncproxy) > >> but that's easily solved with the proposed os_package_type fact. > >> - heat-docker package does not exist in UCA. Haven't looked at what > >> exactly is in this package yet. > >> - horizon UCA package runs into > >> https://bugs.launchpad.net/cloud-archive/+bug/1479340 (we solved this > >> for MOS but it's still broken in UCA) > >> - neutron UCA package seems to suffer from > >> https://bugs.launchpad.net/mos/+bug/1510844 > >> - Need to solve for the url issues with the openstack command that are > >> showing up in the CI as I just hit it with my deployment. > >> > >> Thanks, > >> -Alex > >> > >> > On Tue, Nov 10, 2015 at 12:54 PM, Stanislaw Bogatkin > >> > <sbogat...@mirantis.com> wrote: > >> >> > >> >> Hi Simon, > >> >> > >> >> using 'include' in HAProxy is damn conveniently, I don't think it > >> >> should > >> >> die. There is just one way I know to use haproxy with several > different > >> >> conf's - to construct looooooooooong command line with all of them - > >> >> and > >> >> it's really inconvenient. > >> >> > >> >> On Tue, Nov 10, 2015 at 12:20 PM, Simon Pasquier > >> >> <spasqu...@mirantis.com> > >> >> wrote: > >> >>> > >> >>> Hello Alex! > >> >>> > >> >>> On Mon, Nov 9, 2015 at 9:29 PM, Alex Schultz <aschu...@mirantis.com > > > >> >>> wrote: > >> >>>> > >> >>>> Hey folks, > >> >>>> > >> >>>> I'm testing[0] out flipping our current method of consuming > upstream > >> >>>> puppet modules from using pinned versions hosted on fuel-infra to > be > >> >>>> able to use the ones directly from upstream (master). This work is > >> >>>> primarily to be closer aligned with the other OpenStack projects as > >> >>>> well as switching the current way we manage Fuel into a downstream > of > >> >>>> the upstream community version. As part of this work we have also > >> >>>> been > >> >>>> working towards improving the upstream modules support different > >> >>>> package sets. Specifically running Debian packages on > Ubuntu[1][2]. > >> >>>> This work is the start of being able to allow a user of Fuel to be > >> >>>> able to specify a specific package set and having it be able to > work. > >> >>>> If we can properly split out the puppet modules and package > >> >>>> dependencies this will make Fuel a more flexible deployment engine > as > >> >>>> I believe we would be better positioned to support multiple > versions > >> >>>> of OpenStack for a given Fuel release. > >> >>>> > >> >>>> I'm currently working to get a PoC of Fuel consuming upstream > puppet > >> >>>> modules and the UCA packages together and documenting all of the > >> >>>> issues so that we can address them. So far I have been able to > >> >>>> deploy > >> >>>> the upstream modules via a custom ISO using the MOS package set and > >> >>>> it > >> >>>> works locally. Unfortunately the CI seems to be hitting some issues > >> >>>> that I think might be related to recently merged keystone changes > but > >> >>>> I did not run into the same problem when running a manual > deployment. > >> >>>> As I work through this PoC, I'm also attempting to develop a small > >> >>>> plugin that could be used to capture the work arounds to the > >> >>>> deployment process. I've run into a few issues so far as I work to > >> >>>> switch out the package sets. > >> >>>> > >> >>>> For the sake of providing additional visibility into this work, > here > >> >>>> are the issues that I've hit so far. > >> >>>> > >> >>>> The first issue I ran across is that currently the MOS repositories > >> >>>> contain packages for both OpenStack and other system dependencies > for > >> >>>> creating our HA implementation. This is problematic when we want > to > >> >>>> switch out the OpenStack packages but still want the MOS packages > for > >> >>>> our HA items. I'm working around this by adding the UCA repository > >> >>>> as > >> >>>> a higher priorities for deployments. As such I've run into an > issue > >> >>>> with the haproxy package that MOS provides vs the upstream Ubuntu > >> >>>> package. To get around this, I've pinned the MOS version for now > >> >>>> until I can circle back around and figure out if the difference is > a > >> >>>> config or functionality issue. > >> >>> > >> >>> > >> >>> I know at least for one difference between the MOS and Ubuntu > >> >>> versions: > >> >>> HAProxy from MOS has a patch to support the "include" configuration > >> >>> parameter. > >> >>> This is unrelated to your mail but I think this fork should die > since > >> >>> it > >> >>> will never be accepted upstream and there are other ways to address > >> >>> the use > >> >>> case [0]. > >> >>> > >> >>> HTH, > >> >>> Simon > >> >>> > >> >>> [0] http://marc.info/?l=haproxy&m=130817444025140&w=2 > >> >>> > >> >>>> > >> >>>> > >> >>>> The second issue that I have run across is that we are appending > >> >>>> read_timeout=60 to our mysql connection strings for our > >> >>>> configurations. This seems to be unsupported by the libraries and > >> >>>> OpenStack components provided by the UCA package set. I'm not sure > >> >>>> how the priority of python-pymsql vs python-mysqldb is resolved as > I > >> >>>> had both packages installed but it continued to fail on > read_timeout > >> >>>> being in the connection string. For now I've updated the > >> >>>> fuel-library > >> >>>> code to remove this item from the connection strings and will be > >> >>>> circling back around to figure out the correct 'fix' for this > issue. > >> >>>> > >> >>>> I'm hoping to be able to have a working ISO, plugin and a set of > >> >>>> instructions that can be used to deploy a basic cloud using Fuel by > >> >>>> the end of this week. > >> >>>> > >> >>>> Thanks, > >> >>>> -Alex > >> >>>> > >> >>>> [0] https://review.openstack.org/#/c/240325/ > >> >>>> [1] https://review.openstack.org/#/c/241615/ > >> >>>> [2] https://review.openstack.org/#/c/241741/ > >> >>>> > >> >>>> > >> >>>> > >> >>>> > __________________________________________________________________________ > >> >>>> OpenStack Development Mailing List (not for usage questions) > >> >>>> Unsubscribe: > >> >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > __________________________________________________________________________ > >> >>> OpenStack Development Mailing List (not for usage questions) > >> >>> Unsubscribe: > >> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> >>> > >> >> > >> >> > >> >> > >> >> > __________________________________________________________________________ > >> >> OpenStack Development Mailing List (not for usage questions) > >> >> Unsubscribe: > >> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> >> > >> > > >> > > >> > > >> > -- > >> > Yours Faithfully, > >> > Vladimir Kuklin, > >> > Fuel Library Tech Lead, > >> > Mirantis, Inc. > >> > +7 (495) 640-49-04 > >> > +7 (926) 702-39-68 > >> > Skype kuklinvv > >> > 35bk3, Vorontsovskaya Str. > >> > Moscow, Russia, > >> > www.mirantis.com > >> > www.mirantis.ru > >> > vkuk...@mirantis.com > >> > > >> > > >> > > __________________________________________________________________________ > >> > OpenStack Development Mailing List (not for usage questions) > >> > Unsubscribe: > >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > >> > >> > __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > -- > > Yours Faithfully, > > Vladimir Kuklin, > > Fuel Library Tech Lead, > > Mirantis, Inc. > > +7 (495) 640-49-04 > > +7 (926) 702-39-68 > > Skype kuklinvv > > 35bk3, Vorontsovskaya Str. > > Moscow, Russia, > > www.mirantis.com > > www.mirantis.ru > > vkuk...@mirantis.com > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Kind Regards, Alexandr Kostrikov, Mirantis, Inc. 35b/3, Vorontsovskaya St., 109147, Moscow, Russia Tel.: +7 (495) 640-49-04 Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79> Skype: akostrikov_mirantis E-mail: akostri...@mirantis.com <elogut...@mirantis.com> *www.mirantis.com <http://www.mirantis.ru/>* *www.mirantis.ru <http://www.mirantis.ru/>*
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev