Re: [openstack-dev] [puppet] [ceph] Puppet Ceph CI

[David Gurtner]

I finally got around to split the RGW test into 3 steps: 1) mons/osds/keys 2) 
rgw/apache 3) keystone and got the tests for that to pass on Ubuntu. But it 
seems there is new EPEL dependency issue since yesterday:
https://review.openstack.org/#/c/252664/

David, maybe you wan't to rebase your changes on top of this for easier 
debugging?

- Original Message -
> I pushed an overly optimistic review [1] for updating Openstack to Liberty.
> Haven't had the time to look back at it yet.
> 
> The general idea was to defer the repository setup to openstack_extras and
> pull in
> the keystone setup mostly as-is directly from puppet-openstack-integration.
> 
> [1]: https://review.openstack.org/#/c/251531/
> 
> 
> 
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
> 
> dmsimard = [irc, github, twitter]
> 
> On Wed, Dec 2, 2015 at 5:45 AM, David Gurtner <dgurt...@redhat.com> wrote:
> 
> > So from the discussion I gather we should do the following:
> >
> > - Update the jobs to run Infernalis
> > - Split the RGW jobs into smaller chunks where one tests just the RGW and
> > another one tests Keystone integration
> > - Use Liberty (or at least Kilo) for the Keystone integration job
> > - Split the tests more to have a test specifically for cephx functionality
> > - re-enable the tests for CentOS once they work again
> >
> > Open points from my POV are:
> >
> > - should we test older Ceph versions via Jenkins (this would increase the
> > runtime again)
> > - should we still test CentOS 6 and Ubuntu 12.04
> > - if yes, where
> > - should we port more of the deprecated rspec-puppet-system tests? things
> > I can think of are: 1) the profile tests 2) the
> > scenario_node_terminus/hiera tests
> >
> > I'm happy to start working on the split of tests and the
> > Infernalis/Liberty version bump tonight.
> >
> > Cheers,
> > David
> >
> > - Original Message -
> > > Hey Adam,
> > >
> > > A bit late here, sorry.
> > > Ceph works fine with OpenStack Kilo but at the time we developed the
> > > integration tests for puppet-ceph with Kilo, there were some issues
> > > specific to our test implementation and we chose to settle with Juno
> > > at the time.
> > >
> > > On the topic of CI, I can no longer sponsor the third party CI
> > > (through my former employer, iWeb) as I am with Red Hat now.
> > > I see this as an opportunity to drop the custom system tests with
> > > vagrant and instead improve the acceptance tests.
> > >
> > > What do you think ?
> > >
> > >
> > > David Moreau Simard
> > > Senior Software Engineer | Openstack RDO
> > >
> > > dmsimard = [irc, github, twitter]
> > >
> > >
> > > On Mon, Nov 23, 2015 at 6:45 PM, Adam Lawson <alaw...@aqorn.com> wrote:
> > > > I'm confused, what is the context here? We use Ceph with OpenStack Kilo
> > > > without issue.
> > > >
> > > > On Nov 23, 2015 2:28 PM, "David Moreau Simard" <d...@redhat.com> wrote:
> > > >>
> > > >> Last I remember, David Gurtner tried to use Kilo instead of Juno but
> > > >> he bumped into some problems and we settled for Juno at the time [1].
> > > >> At this point we should already be testing against both Liberty and
> > > >> Infernalis, we're overdue for an upgrade in that regard.
> > > >>
> > > >> But, yes, +1 to split acceptance tests:
> > > >> 1) Ceph
> > > >> 2) Ceph + Openstack
> > > >>
> > > >> Actually learning what failed is indeed challenging sometimes, I don't
> > > >> have enough experience with the acceptance testing to suggest anything
> > > >> better.
> > > >> We have the flexibility of creating different logfiles, maybe we can
> > > >> find a way to split out the relevant bits into another file.
> > > >>
> > > >> [1]: https://review.openstack.org/#/c/153783/
> > > >>
> > > >> David Moreau Simard
> > > >> Senior Software Engineer | Openstack RDO
> > > >>
> > > >> dmsimard = [irc, github, twitter]
> > > >>
> > > >>
> > > >> On Mon, Nov 23, 2015 at 2:45 PM, Andrew Woodward <xar...@gmail.com>
> > wrote:
> > > >> > I think I have a good lead on the recent failures in openstack /
> > swift /
> > > >> > 

Re: [openstack-dev] [puppet] [ceph] Puppet Ceph CI

[David Gurtner]

So from the discussion I gather we should do the following:

- Update the jobs to run Infernalis
- Split the RGW jobs into smaller chunks where one tests just the RGW and 
another one tests Keystone integration
- Use Liberty (or at least Kilo) for the Keystone integration job
- Split the tests more to have a test specifically for cephx functionality
- re-enable the tests for CentOS once they work again

Open points from my POV are:

- should we test older Ceph versions via Jenkins (this would increase the 
runtime again)
- should we still test CentOS 6 and Ubuntu 12.04
- if yes, where
- should we port more of the deprecated rspec-puppet-system tests? things I can 
think of are: 1) the profile tests 2) the scenario_node_terminus/hiera tests

I'm happy to start working on the split of tests and the Infernalis/Liberty 
version bump tonight.

Cheers,
David

- Original Message -
> Hey Adam,
> 
> A bit late here, sorry.
> Ceph works fine with OpenStack Kilo but at the time we developed the
> integration tests for puppet-ceph with Kilo, there were some issues
> specific to our test implementation and we chose to settle with Juno
> at the time.
> 
> On the topic of CI, I can no longer sponsor the third party CI
> (through my former employer, iWeb) as I am with Red Hat now.
> I see this as an opportunity to drop the custom system tests with
> vagrant and instead improve the acceptance tests.
> 
> What do you think ?
> 
> 
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
> 
> dmsimard = [irc, github, twitter]
> 
> 
> On Mon, Nov 23, 2015 at 6:45 PM, Adam Lawson <alaw...@aqorn.com> wrote:
> > I'm confused, what is the context here? We use Ceph with OpenStack Kilo
> > without issue.
> >
> > On Nov 23, 2015 2:28 PM, "David Moreau Simard" <d...@redhat.com> wrote:
> >>
> >> Last I remember, David Gurtner tried to use Kilo instead of Juno but
> >> he bumped into some problems and we settled for Juno at the time [1].
> >> At this point we should already be testing against both Liberty and
> >> Infernalis, we're overdue for an upgrade in that regard.
> >>
> >> But, yes, +1 to split acceptance tests:
> >> 1) Ceph
> >> 2) Ceph + Openstack
> >>
> >> Actually learning what failed is indeed challenging sometimes, I don't
> >> have enough experience with the acceptance testing to suggest anything
> >> better.
> >> We have the flexibility of creating different logfiles, maybe we can
> >> find a way to split out the relevant bits into another file.
> >>
> >> [1]: https://review.openstack.org/#/c/153783/
> >>
> >> David Moreau Simard
> >> Senior Software Engineer | Openstack RDO
> >>
> >> dmsimard = [irc, github, twitter]
> >>
> >>
> >> On Mon, Nov 23, 2015 at 2:45 PM, Andrew Woodward <xar...@gmail.com> wrote:
> >> > I think I have a good lead on the recent failures in openstack / swift /
> >> > radosgw integration component that we have since disabled. It looks like
> >> > there is a oslo.config version upgrade conflict in the Juno repo we
> >> > where
> >> > using for CentOS. I think moving to Kilo will help sort this out, but at
> >> > the
> >> > same time I think it would be prudent to separate the Ceph v.s.
> >> > OpenStack
> >> > integration into separate jobs so that we have a better idea of which is
> >> > a
> >> > problem. If there is census for this, I'd need some direction / help, as
> >> > well as set them up as non-voting for now.
> >> >
> >> > Looking into this I also found that the only place that we do
> >> > integration
> >> > any of the cephx logic was in the same test so we will need to create a
> >> > component for it in the ceph integration as well as use it in the
> >> > OpenStack
> >> > side.
> >> >
> >> > Lastly un-winding the integration failure seemed overly complex. Is
> >> > there a
> >> > way that we can correlate the test status inside the job at a high level
> >> > besides the entire job passed / failed without breaking them into
> >> > separate
> >> > jobs?
> >> > --
> >> >
> >> > --
> >> >
> >> > Andrew Woodward
> >> >
> >> > Mirantis
> >> >
> >> > Fuel Community Ambassador
> >> >
> >> > Ceph Community
> >> >
> >> >
> >> >
> >> > ___

Re: [openstack-dev] [puppet] [ceph] puppet-ceph working session

[David Gurtner]

Hi Andrew

Sadly I'm not present at the summit - but I'm looking forward to the
outcome of the meeting.

Please let me know if there is anything specific I can contribute
towards getting rid of CI issues?

Cheers,
David


On Wed, Oct 28, 2015 at 4:52 AM, Andrew Woodward  wrote:
> Thanks,
> I've added it to the puppet-code session etherpad.
>
> https://etherpad.openstack.org/p/HND-puppet-code
>
>
> On Wed, Oct 28, 2015 at 12:00 PM Emilien Macchi 
> wrote:
>>
>>
>>
>> On 10/28/2015 11:09 AM, Andrew Woodward wrote:
>> > For those of you interested at the summit, I'd like to get together at
>> > some point and discuss / resolve issues on CI, and then talk about
>> > release and possible roadmap.
>> >
>> > Let's pick a time so that we can meet together on this.
>>
>> Good idea, I suggest we meet in the Puppet work sessions, so we get
>> attention from the team.
>>
>> Thanks,
>>
>> Emilien
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> --
>
> --
>
> Andrew Woodward
>
> Mirantis
>
> Fuel Community Ambassador
>
> Ceph Community
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev