Hey Vladimir, On Tue, Nov 10, 2015 at 5:56 AM, Vladimir Kuklin <vkuk...@mirantis.com> wrote: > Alex > > That's great to hear that. But be aware - making all of the components work > exactly the way they work within MOS is actually identical to upstreaming > MOS. We are using some components of different versions to satisfy many > requirements for our Reference Architecutre implementation. It will not be > so easy to base them upon existing 3rd party distributions. For example, > `read timeout` for SQL requests is crucial for our HA as it handles cases > when an SQL node dies while serving the request. And you will find myriads > of them. And as we do not control things in upstream, we will always have > some downstream divergence. >
Yes, I'm aware that it'll be an effort to make it work identically to MOS. Currently that's not my goal. My goal is to get it working at all and be able to document the deficiencies when using upstream packages/modules vs MOS provided ones. Once we have documented these differences we will be able to make decisions as to what efforts should be made if we choose to address the differences. The read timeout thing seems to be an issue with what mysql python driver is used so that could easily be configurable based on a packages or a configuration option. > I guess, the optimal solution is to figure out the actual divergence between > upstream and downstream and try to push things upstream as hard as we can, > while retaining overrides for some packages and manifests on top of upstream > versions. Do not get me wrong, but it seems there is exactly 0 (zero) ways > you can get Fuel working with upstream packages unless they support exactly > the same feature set and fix the same bugs in various components that Fuel > expect them to support or fix. By 'working' I mean passing the same set of > at least smoke and destructive tests, let alone passing scale tests. > So I think this is where we are currently backwards in the way we're doings. As we hope to push Fuel as a community project, we need to be more open to supporting the distributions versions of these packages as being supported. If we want to continue with specific versions of things we need to be able to modularize them and apply them to a downstream version of Fuel that we can promote with the MOS package set. I agree that right now it's highly unlikely that one would be able to use Fuel without any MOS packages. Because I don't think it's possible right now, what I'm attempting to do is be able to deploy an alternate OpenStack package set but not all of the other pieces as they relate to our reference architecture. That seems to be a more achievable goal right now and allows us to document the gaps where Fuel/MOS are a head (or behind depending on your views) with upstream. > Simon > > AFAIK, this patch is really simple and can be easily applied to any version > of HAProxy. So it is not too hard handle it while it provides sufficient > benefits to our deployment engineers. If you have any better solution, > please feel free to share the code with us. > Unfortunately the haproxy fork that we're currently running with also includes our forked version of the puppet haproxy module. I'm not sure the includes configs patch is really worth carrying the additional work/forks but I think that is a conversation for another day. Personally I think it would better if fuel-library supported the currently available upstream versions of haproxy & haproxy puppet module and if we want to continue using our copies of haproxy & puppet haproxy module that we turn that into a downstream that is applied for the MOS specific version of Fuel. That's the advantage of the Puppetfile we've been working on. We can change out the modules with our own version downstream. Just to add some additional updates issues that I've run into. - nova-vncproxy package names (nova-consoleproxy vs nova-novncproxy) but that's easily solved with the proposed os_package_type fact. - heat-docker package does not exist in UCA. Haven't looked at what exactly is in this package yet. - horizon UCA package runs into https://bugs.launchpad.net/cloud-archive/+bug/1479340 (we solved this for MOS but it's still broken in UCA) - neutron UCA package seems to suffer from https://bugs.launchpad.net/mos/+bug/1510844 - Need to solve for the url issues with the openstack command that are showing up in the CI as I just hit it with my deployment. Thanks, -Alex > On Tue, Nov 10, 2015 at 12:54 PM, Stanislaw Bogatkin > <sbogat...@mirantis.com> wrote: >> >> Hi Simon, >> >> using 'include' in HAProxy is damn conveniently, I don't think it should >> die. There is just one way I know to use haproxy with several different >> conf's - to construct looooooooooong command line with all of them - and >> it's really inconvenient. >> >> On Tue, Nov 10, 2015 at 12:20 PM, Simon Pasquier <spasqu...@mirantis.com> >> wrote: >>> >>> Hello Alex! >>> >>> On Mon, Nov 9, 2015 at 9:29 PM, Alex Schultz <aschu...@mirantis.com> >>> wrote: >>>> >>>> Hey folks, >>>> >>>> I'm testing[0] out flipping our current method of consuming upstream >>>> puppet modules from using pinned versions hosted on fuel-infra to be >>>> able to use the ones directly from upstream (master). This work is >>>> primarily to be closer aligned with the other OpenStack projects as >>>> well as switching the current way we manage Fuel into a downstream of >>>> the upstream community version. As part of this work we have also been >>>> working towards improving the upstream modules support different >>>> package sets. Specifically running Debian packages on Ubuntu[1][2]. >>>> This work is the start of being able to allow a user of Fuel to be >>>> able to specify a specific package set and having it be able to work. >>>> If we can properly split out the puppet modules and package >>>> dependencies this will make Fuel a more flexible deployment engine as >>>> I believe we would be better positioned to support multiple versions >>>> of OpenStack for a given Fuel release. >>>> >>>> I'm currently working to get a PoC of Fuel consuming upstream puppet >>>> modules and the UCA packages together and documenting all of the >>>> issues so that we can address them. So far I have been able to deploy >>>> the upstream modules via a custom ISO using the MOS package set and it >>>> works locally. Unfortunately the CI seems to be hitting some issues >>>> that I think might be related to recently merged keystone changes but >>>> I did not run into the same problem when running a manual deployment. >>>> As I work through this PoC, I'm also attempting to develop a small >>>> plugin that could be used to capture the work arounds to the >>>> deployment process. I've run into a few issues so far as I work to >>>> switch out the package sets. >>>> >>>> For the sake of providing additional visibility into this work, here >>>> are the issues that I've hit so far. >>>> >>>> The first issue I ran across is that currently the MOS repositories >>>> contain packages for both OpenStack and other system dependencies for >>>> creating our HA implementation. This is problematic when we want to >>>> switch out the OpenStack packages but still want the MOS packages for >>>> our HA items. I'm working around this by adding the UCA repository as >>>> a higher priorities for deployments. As such I've run into an issue >>>> with the haproxy package that MOS provides vs the upstream Ubuntu >>>> package. To get around this, I've pinned the MOS version for now >>>> until I can circle back around and figure out if the difference is a >>>> config or functionality issue. >>> >>> >>> I know at least for one difference between the MOS and Ubuntu versions: >>> HAProxy from MOS has a patch to support the "include" configuration >>> parameter. >>> This is unrelated to your mail but I think this fork should die since it >>> will never be accepted upstream and there are other ways to address the use >>> case [0]. >>> >>> HTH, >>> Simon >>> >>> [0] http://marc.info/?l=haproxy&m=130817444025140&w=2 >>> >>>> >>>> >>>> The second issue that I have run across is that we are appending >>>> read_timeout=60 to our mysql connection strings for our >>>> configurations. This seems to be unsupported by the libraries and >>>> OpenStack components provided by the UCA package set. I'm not sure >>>> how the priority of python-pymsql vs python-mysqldb is resolved as I >>>> had both packages installed but it continued to fail on read_timeout >>>> being in the connection string. For now I've updated the fuel-library >>>> code to remove this item from the connection strings and will be >>>> circling back around to figure out the correct 'fix' for this issue. >>>> >>>> I'm hoping to be able to have a working ISO, plugin and a set of >>>> instructions that can be used to deploy a basic cloud using Fuel by >>>> the end of this week. >>>> >>>> Thanks, >>>> -Alex >>>> >>>> [0] https://review.openstack.org/#/c/240325/ >>>> [1] https://review.openstack.org/#/c/241615/ >>>> [2] https://review.openstack.org/#/c/241741/ >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Yours Faithfully, > Vladimir Kuklin, > Fuel Library Tech Lead, > Mirantis, Inc. > +7 (495) 640-49-04 > +7 (926) 702-39-68 > Skype kuklinvv > 35bk3, Vorontsovskaya Str. > Moscow, Russia, > www.mirantis.com > www.mirantis.ru > vkuk...@mirantis.com > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
