Hello all,
we have finally finished fixing contrail-packages and successfuly
deployed functional Contrail 2.20 installation from these packages.
All these packages are available on our Launchpad PPA:
https://launchpad.net/~tcpcloud
Extra repository contains mostly dependencies for running and building
Opencontrail and some packages not yet available in Ubuntu. These
packages are not part of any git repository (except for kafka which was
taken here [1] for the moment and I still wonder how this could pass QA)
Packages in PPA are published automatically from our CI so they are
always up to date.
I am soon going to finish blog post with instructions on how to build
Opencontrail packages using clean chroot environments with Pbuilder.
In the meantime I hope our Gerrit fixes will be merged soon so we won't
have to maintain tcpcloud/contrail-packages fork.
https://review.opencontrail.org/#/dashboard/191
Hope our fixes and packages will be useful for others.
[1]
https://github.com/Juniper/contrail-third-party-packages/tree/master/upstream/debian/kafka_2.9.2-0.8.1.1/debian
---
Ad. choice of init systems. We should use what is default in
distribution where out packages are going to be accepted. In case of
Ubuntu it's upstart and systemd for future releases. I don't think
upstream would ever accept packages that are forcing user to use
non-standard system like supervisor.
Unfortunately nodemgr checks supervisord so it would require some patching.
Anyway I think the next step is to get rid of third party software
bundled in contrail packages. I am definately in for further
contribution. We should get list of used third parties and status in
upstream (Debian and Ubuntu) to prepare backported builds for use with
Contrail PPA and contribute missing software.
On CI enhancement, I will publish jenkins job builder setup for our
contrail builds using jenkins debian glue and Pbuilder together with my
blog post so then it can be modified and used for Juniper testing builds.
Filip
On 07/16/2015 09:01 PM, Pedro Marques wrote:
Filip,
On Jul 16, 2015, at 6:47 AM, Filip Pytloun <[email protected]> wrote:
Hello Martin,
so is there some repository that you use to track packaging changes when you
don't contribute back to Juniper/contrail-packages?
I didn't found such repository or any more informations on how syseleven builds
are done, so that was the main reason why we have chosen to use
Juniper/contrail-packages to build packages and contribute back to it.
Are your changes and restructuring so huge or incompatible so they can't be
contributed back to Juniper/contrail-packages?
There are probably the following categories of changes:
1. Migrating third-party downloads to a set of x-dev packages.
2. Choice of init systems (supervisor, systemd, initd).
3. Details that may depend on deployment design.
AFAIK,
In order to solve 1. we just need someone to upload the packages to the
opencontrail PPA so they can be used as build dependencies; it would be good to
have the packaging scripts committed to contrail-packages also so that these
source packages can be recreated. This just means adding packages… Once this is
done, the opencontral CI systems need to be modified so that they can use the
PPA as a repository in order to download these files… then the build
dependencies can be added. It is a sequence of steps requiring coordination but
it not really a design issue.
To address the choice of init systems… I would defer to the debian experts to
suggest an option. Please lets not try to re-hash which init system is best. It
seems to me that there should be a reasonable method to select which one to use.
Options that may be design choices should be done in a way that doesn’t prevent
multiple choice from being available.
OpenContrail is used with and without OpenStack (e.g. cloudstack, custom
orchestration systems, kubernetes). So for instance OpenStack related
dependencies need to be optional (that is the reason there is a
config-openstack package).
Also which build is what you are using in real deployment? For example I didn't
found package contrail-nova-vif in any of Syseleven's builds (including R2.20)
and it's required for Nova API to start. So syseleven's distribution doesn't
seem to be complete enough to be used as is.
BTW I am going to finish and contribute build of contrail-nova-vif to
Juniper/contrail-packages soon.
AFAIK that package is no longer required on OpenStack Kilo.
Anyway it would be great to unify these builds, with support from Juniper where
possible
Sure. It will just require coordination… and a bit of work. My understanding is
that Cloudwatt is also working on 2.2 packaging… the Reliance Jio crowd also
has their own build, i believe. We could probably put together a reasonably
size team assuming we can coordinate. The other aspect is that build and
packaging impacts the opencontrail CI… which is a non-trivial operational
effort for the Juniper team. Contributions to making the CI more robust would
also be greatly appreciated...
(eg. for third party stuff you have mentioned) maybe replacing supervisord by
upstart/systemd integration - but these changes would require fixes in upstream
code (nodemgr for sure, maybe other components).
I would prefer keeping package builds in Juniper/contrail-packages at the
moment (if it's possible) or create Debian packaging team and use Debian's
Alioth.
It is preferable if the packaging scripts are the ones used by the CI system in
order to ensure that they are always functional. The contrail-packages
repository is what the CI uses to build packages. Assuming that what is there
works for everyone, it would be trivial to cut a release and propose that
upstream to debian (along with a specific release of the contrail-source code).
The second option would be better if we are strong enough for it :-)
Filip
On 07/16/2015 07:49 AM, Martin Gerhard Loschwitz wrote:
Good morning,
Am 15.07.2015 um 22:17 schrieb Filip Pytloun <[email protected]>:
Hello Martin,
there are several reasons why we have decided to make our own build:
- at the beginning syseleven packages seemed to be built using
contrail-packages as is (but now it seems there are probably some differences?)
Our packages are still built using the pristine Contrail source as delivered by
GitHub.
Let's take contrail - 2.20~test+1+git+09345e6-1+syseleven2 from
https://launchpad.net/~syseleven-platform/+archive/ubuntu/contrail-2.20/+packages
as an example: The last part of the version number (everything behind the last
+) is
the Git commit ID that was used as source for these packages. There have been a
number of changes to the source:
* Strip out the third_party-stuff after having built proper debian packages of
all the
components that are introduced by third_party stuff -- this alone accomodates
for a
number of patches in debian/patches
* Numerous bug fixes backported from R2.10 and R2.20 to work around issues that
we found in our Contrail installation, mostly suggested by Juniper
All patches can be found in the debian/patches subdirectory, so it's
transparent by
all means to figure out how our contrail sources are different from the
official ones.
Speaking of which: The debian/ subdirectory is also based on what can be found
in
the packaging git repository; there were, however, a number of changes that were
required due to the package restructuring I conducted. Oh, and I did stuff like
adding
back upstart files for all the Contrail daemons because Upstart is what Ubuntu
14.04
uses as its default init system and if you want to support Ubuntu 14.04, you
need to
have upstart scripts for the stuff.
- still there are some missing packages so it doesn't seem to be deployable
easily
For R2.20, we haven't build all the packages yet. For R2.0, we're running a
cluster
automatically deployed with puppet-contrail using exactly these packages. We
are,
in fact, probably not covering the complete Contrail control feature set yet
(we have
left out all the nodemgr stuff, for instance). But the basic functionality is
given by all
means.
- we wanted to have possibility to patch our builds and have some more control
over it
See answer to 1. The Debian Package Source Format 3.0 supports a Quilt mode that
will easily allow to introduce patches and keep track of them.
- we decided not to use official distribution from contrail-packaging, because
unfortunately it doesn't meet expected quality standards on packaging
We're totally on the same side here.
- also making our builds gives us some deeper insight on how the machine works
under the hood
That I can relate to, too.
Anyway I would like to join and maintain contrail-packages so anyone can make
own build and deploy without reinventing the wheel.
Possibly we can try to contribute to Debian so Opencontrail may be part of
Debian and Ubuntu distributions once.
Filip
I'll be more than happy to help. As pointed out in my answer to Robert, there
are some
show stoppers for this though. The third_party stuff is a major problem that we
can
either wait for Juniper to solve or solve on our own. The SysEleven repos have
proper
builds of all packages that would be build by the third_party directory, so
there is a
base to start with. Our contrail-controller packages also contain the patches
required to
make contrail-controller compile without anything from third_party, using the
proper
packages instead. We'd just need a group of people large enough to create the
needed
momentum, and we would need to be willing to create packages diverging from the
Juniper ones if it's necessary to stay compliant with the Debian and Ubuntu
policies.
I'm thinking of stuff such as upstart/systemd integration.
Please let me know what you think.
Best regards
Martin
_______________________________________________
Dev mailing list
[email protected]
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
_______________________________________________
Dev mailing list
[email protected]
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
