Re: [openstack-dev] [sahara] summit wrap-up: subprojects
I agree with this. No real sense in leaving image generation up to novice users at this point. +1 - Original Message - From: "Michael McCune" To: "OpenStack Development Mailing List (not for usage questions)" Sent: Thursday, May 29, 2014 10:39:50 AM Subject: Re: [openstack-dev] [sahara] summit wrap-up: subprojects - Original Message - > the image-elements is too unstable to be used by anyone but an expert at > this point. imho we should make sure the experts produce working images > first, it's what our users will need in the first place, then make the > image generation more stable. +1 mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
- Original Message - > the image-elements is too unstable to be used by anyone but an expert at > this point. imho we should make sure the experts produce working images > first, it's what our users will need in the first place, then make the > image generation more stable. +1 mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
On 05/29/2014 10:15 AM, Michael McCune wrote: - Original Message - Re sahara-image-elements we found a bunch of issues that we should solve and that's why I think that keeping current releasing is still the best option. - we should test it better and depend on stable diskimage-builder version The dib is now published to pypi, so, we could make sahara-image-elements in dib-style and publish it to pypi in the same style. It makes us able to add some sanity tests for images checking and add gate jobs for running them (it could be done anyway, but this approach with separated repo looks more consistent). Developing sahara-image-elements as a pip-installable project we could add diskimage-builder to the requirements.txt of it and manage it's version, it'll provide us good flexibility - for example, we'll be able to specify to use "latest release dib". - all scripts and dib will not be installed with sahara (50/50) I think if we are going to make sahara-image-elements into a full-fledged pypi package we should refactor diskimage-create.sh into a python script. It will give up better options for argument parsing and I feel more control over the flow of operations. mike the image-elements is too unstable to be used by anyone but an expert at this point. imho we should make sure the experts produce working images first, it's what our users will need in the first place, then make the image generation more stable. best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
what were the bunch of issues you ran into? will you capture them in a blueprint or bug so they can be resolved? best, matt On 05/29/2014 10:03 AM, Sergey Lukjanov wrote: Re sahara-image-elements we found a bunch of issues that we should solve and that's why I think that keeping current releasing is still the best option. - we should test it better and depend on stable diskimage-builder version The dib is now published to pypi, so, we could make sahara-image-elements in dib-style and publish it to pypi in the same style. It makes us able to add some sanity tests for images checking and add gate jobs for running them (it could be done anyway, but this approach with separated repo looks more consistent). Developing sahara-image-elements as a pip-installable project we could add diskimage-builder to the requirements.txt of it and manage it's version, it'll provide us good flexibility - for example, we'll be able to specify to use "latest release dib". - all scripts and dib will not be installed with sahara (50/50) On Thu, May 29, 2014 at 3:57 PM, Matthew Farrellee wrote: On 05/29/2014 07:23 AM, Alexander Ignatov wrote: On 28 May 2014, at 20:02, Sergey Lukjanov wrote: sahara-image-elements We're agreed that some common parts should be merged into the diskimage-builder repo (like java support, ssh, etc.). The main issue of keeping -image-elements separated is how to release them and provide mapping sahara version - elements version. You can find different options in etherpad [0], I'll write here about the option that I think will work best for us. So, the idea is that sahara-image-elements is a bunch of scripts and tools for building images for Sahara. It's high coupled with plugins's code in Sahara, so, we need to align them good. Current default decision is to keep aligned versioning like 2014.1 and etc. It'll be discussed on the weekly irc team meeting May 29. I vote to keep sahara-image-elements as separate repo and release it as you Sergey propose. I see problems with sahara-ci when running all bunch of integration tests for checking image-elements and core sahara code on each patch sent to sahara repo in case of collapsed two repos. this problem was raised during the design summit and i thought the resolution was that the sahara-ci could be smart about which set of itests it ran. for instance, a change in the elements would trigger image rebuild, a change outside the elements would trigger service itests. a change that covered both elements and the service could trigger all tests. is that still possible? best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
- Original Message - > Re sahara-image-elements we found a bunch of issues that we should > solve and that's why I think that keeping current releasing is still > the best option. > > - we should test it better and depend on stable diskimage-builder version > The dib is now published to pypi, so, we could make > sahara-image-elements in dib-style and publish it to pypi in the same > style. It makes us able to add some sanity tests for images checking > and add gate jobs for running them (it could be done anyway, but this > approach with separated repo looks more consistent). Developing > sahara-image-elements as a pip-installable project we could add > diskimage-builder to the requirements.txt of it and manage it's > version, it'll provide us good flexibility - for example, we'll be > able to specify to use "latest release dib". > - all scripts and dib will not be installed with sahara (50/50) I think if we are going to make sahara-image-elements into a full-fledged pypi package we should refactor diskimage-create.sh into a python script. It will give up better options for argument parsing and I feel more control over the flow of operations. mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
- Original Message - > > > sahara-extra > > > > Keep it as is, no need to stop releasing, because we're not publishing > > anything to pypi. No real need for tags. > > Even if we keep the repo for now, I think we could simplify a little > bit. The edp-examples could be moved to the Sahara repo. Some of those > examples we use in the integration tests anyway -- why have them > duplicated? +1 i think having the examples in the sahara repo makes it much easier for new users. mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
below, sahara-extra On Wed, 2014-05-28 at 20:02 +0400, Sergey Lukjanov wrote: > Hey folks, > > it's a small wrap-up for the topic "Sahara subprojects releasing and > versioning" that was discussed partially on summit and requires some > more discussions. You can find details in [0]. > > > common > > We'll include only one tarball for sahara to the release launchpad > pages. All other links will be provided in docs. > > > sahara-dashboard > > The merging to Horizon process is now in progress. We've decided that > j1 is the deadline for merging main code parts and during the j2 all > the code should be merged into Horizon, so, if in time of j2 we'll > have some work on merging sahara-dashboard to Horizon not done we'll > need to fallback to the separated sahara-dashboard repo release for > Juno cycle and continue merging the code into the Horizon to be able > to completely kill sahara-dashboard repo in K release. > > Where we should keep our UI integration tests? > > > sahara-image-elements > > We're agreed that some common parts should be merged into the > diskimage-builder repo (like java support, ssh, etc.). The main issue > of keeping -image-elements separated is how to release them and > provide mapping sahara version - elements version. You can find > different options in etherpad [0], I'll write here about the option > that I think will work best for us. > > So, the idea is that sahara-image-elements is a bunch of scripts and > tools for building images for Sahara. It's high coupled with plugins's > code in Sahara, so, we need to align them good. Current default > decision is to keep aligned versioning like 2014.1 and etc. It'll be > discussed on the weekly irc team meeting May 29. > > > sahara-extra > > Keep it as is, no need to stop releasing, because we're not publishing > anything to pypi. No real need for tags. Even if we keep the repo for now, I think we could simplify a little bit. The edp-examples could be moved to the Sahara repo. Some of those examples we use in the integration tests anyway -- why have them duplicated? > > > > open questions > > If you have any objections for this model, please, share your thoughts > before June 3 due to the Juno-1 (June 12) to have enough time to apply > selected approach. > > [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward > > Thanks. > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
> client We have separated launchpad project for client and we're publishing client releases to it. > extra I'm neutral about moving job examples and API samples between sahara and extra > ui tests if we'll be able to remove sahara-dashboard before good integration tests for our pages will be done in horizon than we could move them to the extra repo as a temporary place, but sahara is the wrong place for ui test due to different layers. On Thu, May 29, 2014 at 5:59 PM, Trevor McKay wrote: > below, sahara-extra > > On Wed, 2014-05-28 at 20:02 +0400, Sergey Lukjanov wrote: >> Hey folks, >> >> it's a small wrap-up for the topic "Sahara subprojects releasing and >> versioning" that was discussed partially on summit and requires some >> more discussions. You can find details in [0]. >> >> > common >> >> We'll include only one tarball for sahara to the release launchpad >> pages. All other links will be provided in docs. >> >> > sahara-dashboard >> >> The merging to Horizon process is now in progress. We've decided that >> j1 is the deadline for merging main code parts and during the j2 all >> the code should be merged into Horizon, so, if in time of j2 we'll >> have some work on merging sahara-dashboard to Horizon not done we'll >> need to fallback to the separated sahara-dashboard repo release for >> Juno cycle and continue merging the code into the Horizon to be able >> to completely kill sahara-dashboard repo in K release. >> >> Where we should keep our UI integration tests? >> >> > sahara-image-elements >> >> We're agreed that some common parts should be merged into the >> diskimage-builder repo (like java support, ssh, etc.). The main issue >> of keeping -image-elements separated is how to release them and >> provide mapping sahara version - elements version. You can find >> different options in etherpad [0], I'll write here about the option >> that I think will work best for us. >> >> So, the idea is that sahara-image-elements is a bunch of scripts and >> tools for building images for Sahara. It's high coupled with plugins's >> code in Sahara, so, we need to align them good. Current default >> decision is to keep aligned versioning like 2014.1 and etc. It'll be >> discussed on the weekly irc team meeting May 29. >> >> > sahara-extra >> >> Keep it as is, no need to stop releasing, because we're not publishing >> anything to pypi. No real need for tags. > > Even if we keep the repo for now, I think we could simplify a little > bit. The edp-examples could be moved to the Sahara repo. Some of those > examples we use in the integration tests anyway -- why have them > duplicated? > >> >> >> > open questions >> >> If you have any objections for this model, please, share your thoughts >> before June 3 due to the Juno-1 (June 12) to have enough time to apply >> selected approach. >> >> [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward >> >> Thanks. >> > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
On 05/29/2014 09:59 AM, Trevor McKay wrote: below, sahara-extra sahara-extra Keep it as is, no need to stop releasing, because we're not publishing anything to pypi. No real need for tags. Even if we keep the repo for now, I think we could simplify a little bit. The edp-examples could be moved to the Sahara repo. Some of those examples we use in the integration tests anyway -- why have them duplicated? +1 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
Re sahara-image-elements we found a bunch of issues that we should solve and that's why I think that keeping current releasing is still the best option. - we should test it better and depend on stable diskimage-builder version The dib is now published to pypi, so, we could make sahara-image-elements in dib-style and publish it to pypi in the same style. It makes us able to add some sanity tests for images checking and add gate jobs for running them (it could be done anyway, but this approach with separated repo looks more consistent). Developing sahara-image-elements as a pip-installable project we could add diskimage-builder to the requirements.txt of it and manage it's version, it'll provide us good flexibility - for example, we'll be able to specify to use "latest release dib". - all scripts and dib will not be installed with sahara (50/50) On Thu, May 29, 2014 at 3:57 PM, Matthew Farrellee wrote: > On 05/29/2014 07:23 AM, Alexander Ignatov wrote: >> >> On 28 May 2014, at 20:02, Sergey Lukjanov wrote: > > sahara-image-elements >>> >>> >>> We're agreed that some common parts should be merged into the >>> diskimage-builder repo (like java support, ssh, etc.). The main issue >>> of keeping -image-elements separated is how to release them and >>> provide mapping sahara version - elements version. You can find >>> different options in etherpad [0], I'll write here about the option >>> that I think will work best for us. >>> >>> So, the idea is that sahara-image-elements is a bunch of scripts and >>> tools for building images for Sahara. It's high coupled with plugins's >>> code in Sahara, so, we need to align them good. Current default >>> decision is to keep aligned versioning like 2014.1 and etc. It'll be >>> discussed on the weekly irc team meeting May 29. >> >> >> I vote to keep sahara-image-elements as separate repo and release it >> as you Sergey propose. I see problems with sahara-ci when running all >> bunch >> of integration tests for checking image-elements and core sahara code >> on each patch sent to sahara repo in case of collapsed two repos. > > > this problem was raised during the design summit and i thought the > resolution was that the sahara-ci could be smart about which set of itests > it ran. for instance, a change in the elements would trigger image rebuild, > a change outside the elements would trigger service itests. a change that > covered both elements and the service could trigger all tests. > > is that still possible? > > > best, > > > matt > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
On 05/29/2014 07:23 AM, Alexander Ignatov wrote: On 28 May 2014, at 20:02, Sergey Lukjanov wrote: sahara-image-elements We're agreed that some common parts should be merged into the diskimage-builder repo (like java support, ssh, etc.). The main issue of keeping -image-elements separated is how to release them and provide mapping sahara version - elements version. You can find different options in etherpad [0], I'll write here about the option that I think will work best for us. So, the idea is that sahara-image-elements is a bunch of scripts and tools for building images for Sahara. It's high coupled with plugins's code in Sahara, so, we need to align them good. Current default decision is to keep aligned versioning like 2014.1 and etc. It'll be discussed on the weekly irc team meeting May 29. I vote to keep sahara-image-elements as separate repo and release it as you Sergey propose. I see problems with sahara-ci when running all bunch of integration tests for checking image-elements and core sahara code on each patch sent to sahara repo in case of collapsed two repos. this problem was raised during the design summit and i thought the resolution was that the sahara-ci could be smart about which set of itests it ran. for instance, a change in the elements would trigger image rebuild, a change outside the elements would trigger service itests. a change that covered both elements and the service could trigger all tests. is that still possible? best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
On 28 May 2014, at 20:02, Sergey Lukjanov wrote: > Hey folks, > > it's a small wrap-up for the topic "Sahara subprojects releasing and > versioning" that was discussed partially on summit and requires some > more discussions. You can find details in [0]. > >> common > > We'll include only one tarball for sahara to the release launchpad > pages. All other links will be provided in docs. +1. And keep python-saharaclient on the corresponding launchpad page. > >> sahara-dashboard > > The merging to Horizon process is now in progress. We've decided that > j1 is the deadline for merging main code parts and during the j2 all > the code should be merged into Horizon, so, if in time of j2 we'll > have some work on merging sahara-dashboard to Horizon not done we'll > need to fallback to the separated sahara-dashboard repo release for > Juno cycle and continue merging the code into the Horizon to be able > to completely kill sahara-dashboard repo in K release. > > Where we should keep our UI integration tests? Once sahara-dashboard code is not merged to Horizon we could keep integration tests in the same repo. Once dashboard code is merged we could keep tests in sahara-extra repo. AFAIR we have plans to convert our UI tests to Horizon-capable tests with mocked rest calls. So we could keep non-converted UI tests in sahara-extra until they are done. > >> sahara-image-elements > > We're agreed that some common parts should be merged into the > diskimage-builder repo (like java support, ssh, etc.). The main issue > of keeping -image-elements separated is how to release them and > provide mapping sahara version - elements version. You can find > different options in etherpad [0], I'll write here about the option > that I think will work best for us. > > So, the idea is that sahara-image-elements is a bunch of scripts and > tools for building images for Sahara. It's high coupled with plugins's > code in Sahara, so, we need to align them good. Current default > decision is to keep aligned versioning like 2014.1 and etc. It'll be > discussed on the weekly irc team meeting May 29. I vote to keep sahara-image-elements as separate repo and release it as you Sergey propose. I see problems with sahara-ci when running all bunch of integration tests for checking image-elements and core sahara code on each patch sent to sahara repo in case of collapsed two repos. > >> sahara-extra > > Keep it as is, no need to stop releasing, because we're not publishing > anything to pypi. No real need for tags. +1. Also I think we can move our rest-api-samples from sahara to sahara-extra repo as well. > > >> open questions > > If you have any objections for this model, please, share your thoughts > before June 3 due to the Juno-1 (June 12) to have enough time to apply > selected approach. > > [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward > > Thanks. > > -- > Sincerely yours, > Sergey Lukjanov > Sahara Technical Lead > (OpenStack Data Processing) > Principal Software Engineer > Mirantis Inc. > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: subprojects
On 05/28/2014 12:02 PM, Sergey Lukjanov wrote: Hey folks, it's a small wrap-up for the topic "Sahara subprojects releasing and versioning" that was discussed partially on summit and requires some more discussions. You can find details in [0]. common We'll include only one tarball for sahara to the release launchpad pages. All other links will be provided in docs. safe to assume this is in addition to the client tarball? sahara-dashboard The merging to Horizon process is now in progress. We've decided that j1 is the deadline for merging main code parts and during the j2 all the code should be merged into Horizon, so, if in time of j2 we'll have some work on merging sahara-dashboard to Horizon not done we'll need to fallback to the separated sahara-dashboard repo release for Juno cycle and continue merging the code into the Horizon to be able to completely kill sahara-dashboard repo in K release. we really need to kill sahara-dashboard before the juno release Where we should keep our UI integration tests? ideally w/ the code it tests, so horizon. are there problems w/ that approach? as a fallback they can go into the sahara repo sahara-image-elements We're agreed that some common parts should be merged into the diskimage-builder repo (like java support, ssh, etc.). The main issue of keeping -image-elements separated is how to release them and provide mapping sahara version - elements version. You can find different options in etherpad [0], I'll write here about the option that I think will work best for us. So, the idea is that sahara-image-elements is a bunch of scripts and tools for building images for Sahara. It's high coupled with plugins's code in Sahara, so, we need to align them good. Current default decision is to keep aligned versioning like 2014.1 and etc. It'll be discussed on the weekly irc team meeting May 29. i vote for merging sahara-image-elements into the sahara repo and keeping the strategic direction that common-enough elements get pushed to diskimage-builder sahara-extra Keep it as is, no need to stop releasing, because we're not publishing anything to pypi. No real need for tags. we still need to figure out the examples and swift plugin, but seems reasonable to punt that from the juno cycle if there is no bandwidth open questions If you have any objections for this model, please, share your thoughts before June 3 due to the Juno-1 (June 12) to have enough time to apply selected approach. [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward so ideal situation imho - . sahara (includes image elements and possibly ui tests) . python-saharaclient (as before) . sahara-extra (handle later) . horizon (everything that was in sahara-dashboard) this misses the puppet modules. possibly they should also be merged into the sahara repo. best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sahara] summit wrap-up: subprojects
Hey folks, it's a small wrap-up for the topic "Sahara subprojects releasing and versioning" that was discussed partially on summit and requires some more discussions. You can find details in [0]. > common We'll include only one tarball for sahara to the release launchpad pages. All other links will be provided in docs. > sahara-dashboard The merging to Horizon process is now in progress. We've decided that j1 is the deadline for merging main code parts and during the j2 all the code should be merged into Horizon, so, if in time of j2 we'll have some work on merging sahara-dashboard to Horizon not done we'll need to fallback to the separated sahara-dashboard repo release for Juno cycle and continue merging the code into the Horizon to be able to completely kill sahara-dashboard repo in K release. Where we should keep our UI integration tests? > sahara-image-elements We're agreed that some common parts should be merged into the diskimage-builder repo (like java support, ssh, etc.). The main issue of keeping -image-elements separated is how to release them and provide mapping sahara version - elements version. You can find different options in etherpad [0], I'll write here about the option that I think will work best for us. So, the idea is that sahara-image-elements is a bunch of scripts and tools for building images for Sahara. It's high coupled with plugins's code in Sahara, so, we need to align them good. Current default decision is to keep aligned versioning like 2014.1 and etc. It'll be discussed on the weekly irc team meeting May 29. > sahara-extra Keep it as is, no need to stop releasing, because we're not publishing anything to pypi. No real need for tags. > open questions If you have any objections for this model, please, share your thoughts before June 3 due to the Juno-1 (June 12) to have enough time to apply selected approach. [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward Thanks. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
