Re: [openstack-dev] [nova] FreeBSD host support, round 2

2015-06-12 Thread Roman Bogorodskiy
  Daniel P. Berrange wrote:

 On Tue, Jun 09, 2015 at 07:52:39PM +0300, Roman Bogorodskiy wrote:
  Hi,
  
  Few months ago I've started a discussion on FreeBSD host support for
  OpenStack. [1] At a result of discussion it was figured out that there
  are a number of limitations, mainly on the BHyVe (the FreeBSD hypervisor)
  side, that make the effort not feasible.
 
 Do you still intend to add the missing features to BHyve to let it be
 supported in Nova eventually too ?

As I don't take part in bhyve development, it's hard for me to give a
detailed answer on that. I know that bhyve developers are
planning/working on some important improvements like UEFI support,
moving to single step guest boot and some others. I'm just trying to
keep libvirt/bhyve driver up to date.

 
  However, some things changed since then. Specifically, FreeBSD's got Xen
  dom0 support. [2] In context of OpenStack deployment that has a number of
  benefits over bhyve. Specifically:
  
   * The stack is more mature and feature-rich
   * The toolstack is here already: libxl is available through the FreeBSD
 ports tree, libvirt/libxl works there with minimal modifications
 (already available in the git master)
   * OpenStack libvirt/libxl driver is already here
  
  I was able to setup a proof-of-concept environment on FreeBSD that
  required quite a small amount of modifications required in OpenStack:
  
   * Glance and Keystone didn't require any changes
   * Nove required some minor modifications mainly in the linux_net.py
 area
  
  The summary of Nova modifications:
  
   * I had to implement FreeBSD version of linux_net.LinuxNetInterfaceDriver.
 It currently doesn't support vlans though. 
  
 
  https://github.com/novel/bsdstack/blob/master/bsdstack/nova/network/freebsd_net.py
  
 I keep it as an external package and configure Nova to use it through
 linuxnet_interface_driver config option in nova.conf
   * I had to create a stub for the IptablesManager class. I also had to
 add a config option to be able to override class for that in a way
 similar to interface driver.
   * I had to fix a minor interface incapability for NullL3 stub, that's
 already in the Nova tree: https://review.openstack.org/#/c/189001/
   * I added a hack to use 'phy' driver in domain's xml for disks because
 for some reason driver='qemu' results in guests not able to access
 disk devices (tried both FreeBSD and Linux guests). Need to
 investigate
   * Dropped some LinuxBridgeInterfaceDriver hardcodes in
 virt.libvirt.vif.
  
  Here's a quick overview of my changes:
  
  https://github.com/novel/nova/compare/stable/kilo...novel:stable/kilo_freebsd?expand=1
  
  With this changes I was able to get things working, i.e. VM starting,
  obtaining IP addresses (with nova-network configured with FlatDHCP) etc.
  
  Having that said I'm wondering if community is interested in integrating
  FreeBSD support through libvirt/libxl into mainline? Obviously, the
  changes I provided are shortcuts and the appropriate specs should be
  create with proposals of proper designs, not quick hacks like that.
 
 I'd be happy to see FreeBSD support for any of the libvirt hypervisors
 added to Nova. As you point out, the changes to the libvirt code are
 going to be pretty minimal for libxl, so there's not going to be any
 appreciable ongoing maint burden for this. So I see no serious reason
 to reject the changes to Nova libvirt driver code for libxl+FreeBSD.
 The fun bit will be the changes to non-libvirt code in nova...
 
  The biggest part of the unportable code, just like it was in bhyve case,
  is still linux_net.py, so probably it makes sense to revive the old
  spec:
  
  https://review.openstack.org/#/c/127827/
  
  TBH, IMHO linux_net.py could have a refactor regardless of FreeBSD
  support. It's approx. 2000 lines long, contains a lot of stuff like
  dnsmasq handling code, interface handing code and firewall management
  that could have their own place.
 
 Yes, that network setup code is really awful and I'd love to see
 it refactored regardless of FreeBSD support.
 
 With my crystal ball, probably the main question wrt any kind of
 FreeBSD support will be that of 3rd party CI testing. I don't know
 whether there is any company backing your work who has resources
 to provide a CI system, or whether FreeBSD project can manage it
 independently ?

I do all my FreeBSD related activities in my free time, in other words
it's not backed by any company. Anyway, I think it would not be a
problem to arrange virtual machines for running unit tests. As for
integration testing it's going to be harder because it'd need a real
hardware and I need to figure out how I could obtain one.

 The refactoring of the linux_net.py code could be done even without
 CI support of course - that would only block the second part where
 you actually add a FreeBSD implementation

That sounds good. I'll create a new spec for linux_net.py

[openstack-dev] [nova] FreeBSD host support, round 2

2015-06-09 Thread Roman Bogorodskiy
Hi,

Few months ago I've started a discussion on FreeBSD host support for
OpenStack. [1] At a result of discussion it was figured out that there
are a number of limitations, mainly on the BHyVe (the FreeBSD hypervisor)
side, that make the effort not feasible.

However, some things changed since then. Specifically, FreeBSD's got Xen
dom0 support. [2] In context of OpenStack deployment that has a number of
benefits over bhyve. Specifically:

 * The stack is more mature and feature-rich
 * The toolstack is here already: libxl is available through the FreeBSD
   ports tree, libvirt/libxl works there with minimal modifications
   (already available in the git master)
 * OpenStack libvirt/libxl driver is already here

I was able to setup a proof-of-concept environment on FreeBSD that
required quite a small amount of modifications required in OpenStack:

 * Glance and Keystone didn't require any changes
 * Nove required some minor modifications mainly in the linux_net.py
   area

The summary of Nova modifications:

 * I had to implement FreeBSD version of linux_net.LinuxNetInterfaceDriver.
   It currently doesn't support vlans though. 

   
https://github.com/novel/bsdstack/blob/master/bsdstack/nova/network/freebsd_net.py

   I keep it as an external package and configure Nova to use it through
   linuxnet_interface_driver config option in nova.conf
 * I had to create a stub for the IptablesManager class. I also had to
   add a config option to be able to override class for that in a way
   similar to interface driver.
 * I had to fix a minor interface incapability for NullL3 stub, that's
   already in the Nova tree: https://review.openstack.org/#/c/189001/
 * I added a hack to use 'phy' driver in domain's xml for disks because
   for some reason driver='qemu' results in guests not able to access
   disk devices (tried both FreeBSD and Linux guests). Need to
   investigate
 * Dropped some LinuxBridgeInterfaceDriver hardcodes in
   virt.libvirt.vif.

Here's a quick overview of my changes:

https://github.com/novel/nova/compare/stable/kilo...novel:stable/kilo_freebsd?expand=1

With this changes I was able to get things working, i.e. VM starting,
obtaining IP addresses (with nova-network configured with FlatDHCP) etc.

Having that said I'm wondering if community is interested in integrating
FreeBSD support through libvirt/libxl into mainline? Obviously, the
changes I provided are shortcuts and the appropriate specs should be
create with proposals of proper designs, not quick hacks like that.

The biggest part of the unportable code, just like it was in bhyve case,
is still linux_net.py, so probably it makes sense to revive the old
spec:

https://review.openstack.org/#/c/127827/

TBH, IMHO linux_net.py could have a refactor regardless of FreeBSD
support. It's approx. 2000 lines long, contains a lot of stuff like
dnsmasq handling code, interface handing code and firewall management
that could have their own place.

Feedback is appreciated,
Thanks

1: http://lists.openstack.org/pipermail/openstack-dev/2014-October/048721.html
2: http://wiki.xen.org/wiki/FreeBSD_Dom0

Roman Bogorodskiy


pgpu0F_6bRBdH.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD host support

2014-10-30 Thread Roman Bogorodskiy
  Monty Taylor wrote:

 On 10/27/2014 06:39 AM, Michael Still wrote:
  On Tuesday, October 21, 2014, Roman Bogorodskiy rbogorods...@mirantis.com
  wrote:
  
  On Mon, Oct 20, 2014 at 10:19 PM, Joe Gordon joe.gord...@gmail.com
  javascript:; wrote:
  On Sat, Oct 18, 2014 at 10:04 AM, Roman Bogorodskiy 
  rbogorods...@mirantis.com javascript:; wrote:
 
  
  [snip]
  
  
  High level overview of what needs to be done:
 
   - Nova
* linux_net needs to be re-factored to allow to plug in FreeBSD
  support (that's what the spec linked above is about)
* nova.virt.disk.mount needs to be extended to support FreeBSD's
  mdconfig(8) in a similar way to Linux's losetup
 
  
  [snip]
  
  
  What about neutron? We are in the process of trying to deprecate
  nova-network, so any new thing needs to support neutron.
 
 
  AFAIK, there's no defined migration plan yet, unless I missed that.
  Anyway, I don't see any blockers regarding an implementation of a driver
  similar to linuxbridge that'd work on FreeBSD.
 
  Also, Semihalf guys are working on OpenContail/FreeBSD and
  Neutron/OpenContrial support, so that's an option as well.
  
  
  I have no problem with supporting FreeBSD as a hypervisor operating system,
  especially if there is a solid team on the FreeBSD side that will commit to
  maintaining the changes required and adding the necessary CI (especially
  ensuring that when it breaks it gets fixed).
 
 I believe that the CI related things that would be needed would be:
 
 - solid devstack support
 - someone willing to step up and make sure that nodepool can provide
 freebsd images like ianw recently did with centos

Semihalf guys implemented FreeBSD support devstack as well (Michał
CCed):

 https://github.com/Semihalf/openstack-devstack

I don't know if they did an attempt to push these changes back.

Creating FreeBSD images is not hard and I could do that.

Anyway, there are some points regarding the CI that are not quite clear
to me. 

 - Should it be a 3rd party CI or integrated to the main CI?
 - At what point we want to start tempest/devstack testing over FreeBSD?
   I think it'll take quite some time to make these pass (maybe several
   release cycles).

  However, I see Neutron support as a firm requirement. We've spent a large
  amount of time getting closer and closer to deprecating nova-network.
  Despite opening it up for limited development again, I don't think we
  should be making the transition plan harder by introducing new features
  that don't work with Neutron.
 
 I agree with Mikal on this.

Good. It doesn't look like a problem to me to bring the support into
Neutron over nova-network. After a brief view the level of effort for
the Neutron implementation is not much higher comparing to nova-network.

Roman Bogorodskiy

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD host support

2014-10-24 Thread Roman Bogorodskiy
  Roman Bogorodskiy wrote:

 On Mon, Oct 20, 2014 at 10:19 PM, Joe Gordon joe.gord...@gmail.com wrote:
 
 
 
  On Sat, Oct 18, 2014 at 10:04 AM, Roman Bogorodskiy 
  rbogorods...@mirantis.com wrote:
 

ping?

Roman Bogorodskiy


pgpz_SKT6P9Gy.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD host support

2014-10-21 Thread Roman Bogorodskiy
On Mon, Oct 20, 2014 at 10:19 PM, Joe Gordon joe.gord...@gmail.com wrote:



 On Sat, Oct 18, 2014 at 10:04 AM, Roman Bogorodskiy 
 rbogorods...@mirantis.com wrote:

 Hi,

 In discussion of this spec proposal:
 https://review.openstack.org/#/c/127827/ it was suggested by Joe Gordon
 to start a discussion on the mailing list.

 So I'll share my thoughts and a long term plan on adding FreeBSD host
 support for OpenStack.

 An ultimate goal is to allow using libvirt/bhyve as a compute driver.
 However, I think it would be reasonable to start with libvirt/qemu
 support first as it will allow to prepare the ground.


 Before diving into the technical details below, I have one question. Why, 
 What is the benefit of this, besides the obvious 'we not support FreeBSD'?  
 Adding support for a new kernel introduces yet another column in our support 
 matrix, and will require a long term commitment to testing and maintaining 
 OpenStack on FreeBSD.

There a number of FreeBSD users that are interested in virtualization
and an effective
management of the virtualized resources. Using OpenStack would be much
more convenient
than using some custom scripts / home grown solutions people usually use now.



 High level overview of what needs to be done:

  - Nova
   * linux_net needs to be re-factored to allow to plug in FreeBSD
 support (that's what the spec linked above is about)
   * nova.virt.disk.mount needs to be extended to support FreeBSD's
 mdconfig(8) in a similar way to Linux's losetup
  - Glance and Keystone
 These components are fairly free of system specifics. Most likely
 they will require some small fixes like e.g. I made for Glance
 https://review.openstack.org/#/c/94100/
  - Cinder
 I didn't look close at Cinder from a porting perspective, tbh.
 Obviously, it'll need some backend driver that would work on
 FreeBSD, e.g. ZFS. I've seen some patches floating around for ZFS
 though. Also, I think it'll need an implementation of iSCSI stack
 on FreeBSD, because it has its own stack, not stgt. On the other
 hand, Cinder is not required for a minimal installation and that
 could be done after adding support of the other components.


 What about neutron? We are in the process of trying to deprecate 
 nova-network, so any new thing needs to support neutron.


AFAIK, there's no defined migration plan yet, unless I missed that.
Anyway, I don't see any blockers regarding an implementation of a driver
similar to linuxbridge that'd work on FreeBSD.

Also, Semihalf guys are working on OpenContail/FreeBSD and
Neutron/OpenContrial support, so that's an option as well.




 Also, it's worth to mention that a discussion on this topic already
 happened on this maillist:

 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031431.html

 Some of the limitations were resolved since then, specifically,
 libvirt/bhyve has no limitation on count of disk and ethernet devices
 anymore.

 Roman Bogorodskiy



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] FreeBSD host support

2014-10-18 Thread Roman Bogorodskiy
Hi,

In discussion of this spec proposal:
https://review.openstack.org/#/c/127827/ it was suggested by Joe Gordon
to start a discussion on the mailing list.

So I'll share my thoughts and a long term plan on adding FreeBSD host
support for OpenStack. 

An ultimate goal is to allow using libvirt/bhyve as a compute driver.
However, I think it would be reasonable to start with libvirt/qemu
support first as it will allow to prepare the ground.

High level overview of what needs to be done:

 - Nova
  * linux_net needs to be re-factored to allow to plug in FreeBSD
support (that's what the spec linked above is about)
  * nova.virt.disk.mount needs to be extended to support FreeBSD's
mdconfig(8) in a similar way to Linux's losetup
 - Glance and Keystone
These components are fairly free of system specifics. Most likely
they will require some small fixes like e.g. I made for Glance
https://review.openstack.org/#/c/94100/
 - Cinder
I didn't look close at Cinder from a porting perspective, tbh.
Obviously, it'll need some backend driver that would work on
FreeBSD, e.g. ZFS. I've seen some patches floating around for ZFS
though. Also, I think it'll need an implementation of iSCSI stack
on FreeBSD, because it has its own stack, not stgt. On the other
hand, Cinder is not required for a minimal installation and that
could be done after adding support of the other components.

Also, it's worth to mention that a discussion on this topic already
happened on this maillist: 

http://lists.openstack.org/pipermail/openstack-dev/2014-March/031431.html

Some of the limitations were resolved since then, specifically,
libvirt/bhyve has no limitation on count of disk and ethernet devices
anymore.

Roman Bogorodskiy


pgpYJ0yz8_DVc.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Deprecating localfs?

2014-09-23 Thread Roman Bogorodskiy
  Michael Still wrote:

 Hi.
 
 I know we've been talking about deprecating nova.virt.disk.vfs.localfs
 for a long time, in favour of wanting people to use libguestfs
 instead. However, I can't immediately find any written documentation
 for when we said we'd do that thing.
 
 Additionally, this came to my attention because Ubuntu 14.04 is
 apparently shipping a libguestfs old enough to cause us to emit the
 falling back to localfs warning, so I think we need Ubuntu to catch
 up before we can do this thing.
 
 So -- how about we remove localfs early in Kilo to give Canonical a
 release to update libguestfs?
 
 Thoughts appreciated,
 Michael

If at some point we'd start moving into getting FreeBSD supported as a
host OS for OpenStack, then it would make sense to keep localfs for that
configuration.

libguestfs doesn't work on FreeBSD yet. On the other hand, localfs
code in Nova doesn't look like it'd be hard to port.

Roman Bogorodskiy

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Infra][Cinder] Coraid CI system

2014-09-12 Thread Roman Bogorodskiy
Hi,

Mykola has some problems sending emails to the list, so he asked me to post
a response, here it goes:

---
Remy, I have improved Coraid CI system and added logs of all components of
devstack. Please have a look:

http://38.111.159.9:8080/job/Coraid_CI/164/

According to the requirements from
http://ci.openstack.org/third_party.html#requesting-a-service-account ,
Gerrit plugin from Jenkins should be given the following options:

Successful: gerrit approve CHANGE,PATCHSET --message 'Build
Successful BUILDS_STATS' --verified VERIFIED --code-review
CODE_REVIEW
Failed: gerrit approve CHANGE,PATCHSET --message 'Build Failed
BUILDS_STATS' --verified VERIFIED --code-review CODE_REVIEW
Unstable: gerrit approve CHANGE,PATCHSET --message 'Build Unstable
BUILDS_STATS' --verified VERIFIED --code-review CODE_REVIEW

I configured gerrit plugin this way, so it sends the following comment
after checking patchset or comment with recheck. For example,
https://review.openstack.org/#/c/120907/

Patch Set 1:

Build Successful

http://38.111.159.9:8080/job/Coraid_CI/164/ : SUCCESS


All logs are on this page. They are there as artifacts.

 I took a quick look and I don’t see which test cases are being run?
We test Coraid Cinder driver with standard tempest tests using
./driver_certs/cinder_driver_cert.sh script. Test cases are in the log of
job.

Please look at Coraid third-party system one more time and, please, show us
what we have to add or improve in order to get voting rights for gerrit
user coraid-ci.

Also I have set gerrit plugin on our Jenkins to the silent mode as you
suggested.

Thank you in advance.


On Fri, Sep 5, 2014 at 7:34 PM, Asselin, Ramy ramy.asse...@hp.com wrote:

 -1 from me (non-cinder core)

 It very nice to see you're making progress. I, personally, was very
 confused about voting.
 Here's my understanding: Voting: it is the ability to provide an
 official +1 -1 vote in the gerrit system.

 I don't see a stable history [1]. Before requesting voting, you should
 enable your system on the cinder project itself.
 Initially, you should disable ALL gerrit comments, i.e. run in silent
 mode, per request from cinder PTL [2]. Once stable there, you can enable
 gerrit comments. At this point, everyone can see pass/fail comments with a
 vote=0.
 Once stable there on real patches, you can request voting again, where the
 pass/fail would vote +1/-1.

 Ramy
 [1] http://38.111.159.9:8080/job/Coraid_CI/35/console
 [2]
 http://lists.openstack.org/pipermail/openstack-dev/2014-August/043876.html


 -Original Message-
 From: Duncan Thomas [mailto:duncan.tho...@gmail.com]
 Sent: Friday, September 05, 2014 7:55 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Infra][Cinder] Coraid CI system

 +1 from me (Cinder core)

 On 5 September 2014 15:09, Mykola Grygoriev mgrygor...@mirantis.com
 wrote:
  Hi,
 
  My name is Mykola Grygoriev and I'm engineer who currently working on
  deploying 3d party CI for Сoraid Сinder driver.
 
  Following instructions on
 
  http://ci.openstack.org/third_party.html#requesting-a-service-account
 
  asking for adding gerrit CI account (coraid-ci) to the Voting
  Third-Party CI Gerrit group.
 
 
 
  We have already added description of Coraid CI system to wiki page -
  https://wiki.openstack.org/wiki/ThirdPartySystems/Coraid_CI
 
  We used openstack-dev/sandbox project to test current CI
  infrastructure with OpenStack Gerrit system. Please find our history
 there.
 
  Please have a look to results of Coraid CI system. it currently takes
  updates from openstack/cinder project:
  http://38.111.159.9:8080/job/Coraid_CI/32/
  http://38.111.159.9:8080/job/Coraid_CI/33/
 
  Thank you in advance.
 
  --
  Best regards,
  Mykola Grygoriev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 --
 Duncan Thomas

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Infra][Cinder] Coraid CI system

2014-09-07 Thread Roman Bogorodskiy
Hi Ramy,

Thanks for the links; looks like these address a problem of creating a
proper volume type that happens on deployment time and that would be
used as a default volume type.

And I'm talking about a volume type that Tempest creates in runtime:

https://github.com/openstack/tempest/blob/master/tempest/api/volume/admin/test_volume_types.py#L41

Currently it uses a fixed set of keys, 'snapshot_protocol' and
'vendor_name', however it needs to be extended to allow to pass
additional keys that the Coraid driver needs to operate.

  Asselin, Ramy wrote:

 Not sure if https://review.openstack.org/#/c/118619/ is necessary.
 See if this solves your issue [1], simple update to your devstack 
 localrc/local.conf to create the type keys. It is merged to master.
 Otherwise, consider creating a cinder backend file[1].
 
 Ramy
 
 [1] http://lists.openstack.org/pipermail/openstack-dev/2014-August/043274.html
 [2] https://github.com/openstack-dev/devstack/tree/master/lib/cinder_backends
 
 From: Roman Bogorodskiy [mailto:rbogorods...@mirantis.com]
 Sent: Saturday, September 06, 2014 8:34 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Infra][Cinder] Coraid CI system
 
 Mykola, thanks for update!
 
 Few additional notes on this. For now, we have to inject two patches to make 
 it work.
 
 The first patch is a simple bugfix for the Coraid driver itself:
 
 https://review.openstack.org/#/c/119022/
 
 The second one is for Tempest to allow specification of custom volume type 
 keys that are required for the Coraid
 driver to operate properly:
 
 https://review.openstack.org/#/c/118619/
 
 On Fri, Sep 5, 2014 at 6:09 PM, Mykola Grygoriev 
 mgrygor...@mirantis.commailto:mgrygor...@mirantis.com wrote:
 Hi,
 
 My name is Mykola Grygoriev and I'm engineer who currently working on 
 deploying 3d party CI for Сoraid Сinder driver.
 
 Following instructions on
 
 http://ci.openstack.org/third_party.html#requesting-a-service-account
 
 asking for adding gerrit CI account (coraid-ci) to the Voting Third-Party CI 
 Gerrit grouphttps://review.openstack.org/#/admin/groups/91,members.
 
 
 We have already added description of Coraid CI system to wiki page - 
 https://wiki.openstack.org/wiki/ThirdPartySystems/Coraid_CI
 We used openstack-dev/sandbox project to test current CI infrastructure with 
 OpenStack Gerrit system. Please find our history there.
 Please have a look to results of Coraid CI system. it currently takes updates 
 from openstack/cinder project:
 http://38.111.159.9:8080/job/Coraid_CI/32/
 http://38.111.159.9:8080/job/Coraid_CI/33/
 Thank you in advance.
 
 --
 Best regards,
 Mykola Grygoriev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Roman Bogorodskiy


pgpmrSINTYAFw.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Infra][Cinder] Coraid CI system

2014-09-06 Thread Roman Bogorodskiy
Mykola, thanks for update!

Few additional notes on this. For now, we have to inject two patches to
make it work.

The first patch is a simple bugfix for the Coraid driver itself:

https://review.openstack.org/#/c/119022/

The second one is for Tempest to allow specification of custom volume type
keys that are required for the Coraid
driver to operate properly:

https://review.openstack.org/#/c/118619/



On Fri, Sep 5, 2014 at 6:09 PM, Mykola Grygoriev mgrygor...@mirantis.com
wrote:

 Hi,

 My name is Mykola Grygoriev and I'm engineer who currently working on
 deploying 3d party CI for Сoraid Сinder driver.

 Following instructions on

 http://ci.openstack.org/third_party.html#requesting-a-service-account

 asking for adding gerrit CI account (coraid-ci) to the Voting Third-Party
 CI Gerrit group https://review.openstack.org/#/admin/groups/91,members.


 We have already added description of Coraid CI system to wiki page -
 https://wiki.openstack.org/wiki/ThirdPartySystems/Coraid_CI

 We used openstack-dev/sandbox project to test current CI infrastructure
 with OpenStack Gerrit system. Please find our history there.

 Please have a look to results of Coraid CI system. it currently takes
 updates from openstack/cinder project:
 http://38.111.159.9:8080/job/Coraid_CI/32/
 http://38.111.159.9:8080/job/Coraid_CI/33/

 Thank you in advance.

 --
 Best regards,
 Mykola Grygoriev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Nova] [Spec freeze exception] libvirt/linux_net refactor

2014-07-18 Thread Roman Bogorodskiy
Hi,

I would like to propose this spec for an exception:

https://review.openstack.org/#/c/95328/

The original idea was to add FreeBSD support for the network driver
(linux_net) so it was possible to run Nova on FreeBSD with libvirt/qemu
driver.

The preparation for that would require linux_net refactoring. So while
it's probably too late to aim for FreeBSD network support in Juno, I
think it would be good to refactor linux_net in Juno to make it more
readable, modular and maintainable. And the FreeBSD support could be
added later in Kilo. More details on the need of linux_net refactoring
are available in the spec itself.

Roman Bogorodskiy


pgpZ5IhFQZXcU.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] fair standards for all hypervisor drivers

2014-07-16 Thread Roman Bogorodskiy
  Eric Windisch wrote:

 This thread highlights more deeply the problems for the FreeBSD folks.
 First, I still disagree with the recommendation that they contribute to
 libvirt. It's a classic example of creating two or more problems from one.
 Once they have support in libvirt, how long before their code is in a
 version of libvirt acceptable to Nova? When they hit edge-cases or bugs,
 requiring changes in libvirt, how long before those fixes are accepted by
 Nova?

Could you please elaborate why you disagree on the contributing patches
to libvirt approach and what the alternative approach do you propose?

Also, could you please elaborate on what is 'version of libvirt
acceptable to Nova'? Cannot we just say that e.g. Nova requires libvirt
X.Y to be deployed on FreeBSD?

Anyway, speaking about FreeBSD support I assume we actually talking
about Bhyve support. I think it'd be good to break the task and
implement FreeBSD support for libvirt/Qemu first.

Qemu driver of libvirt works fine with FreeBSD for quite some time
already and adding support for that in Nova will allow to do all the
ground work before we could move to the libvirt/bhyve support.

I'm planning to start with adding networking support. Unfortunately, it
seems I got late with the spec for Juno though:

https://review.openstack.org/#/c/95328/

Roman Bogorodskiy


pgpkeNEjFWmYC.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD/bhyve support for nova with libvirt

2014-03-31 Thread Roman Bogorodskiy
  Michał Dubiel wrote:

 Hi All,
 
 I have prepared commits I would like to have it reviewed and eventually
 merged that add initial, limited support for FreeBSD as a host to nova. It
 includes basic networking via freebsd_net driver (similar to the linux_net)
 and few addons to libvirt compute driver in order to support the bhyve
 hypervisor. Intent for those commits is let other play with openstack on
 FreeBSD and to provide a code base for further development, as the current
 version comes with many limitations like:
 
 - Only FreeBSD guest OSes can be used
 - No support for the config drive
 - Only one disk and one Ethernet interface
 - No pause/resume functionality
 - No VM migration support
 - No files injection to VMs filesystem
 - Only works with bridged networking using nova-network with Flat/FlatDHCP
 multi-host mode
 
 Unit test are included, however, for all that to work on a real system you
 have to use a slightly patched version of libvirt as not all features has
 been merged to the official repository yet. My question is if that is
 applicable to be merged at all, or should I wait for all necessary stuff to
 be in libvirt official repository at first? I want also mention that there
 is an active work underway in libvirt community to have all them
 implemented and included in the libvirt code.

Hi Michal,

I'd say it would be good to revive the old blueprint about bhyve
support and push the patches. Also, probably freebsd_net itself
deserve a blueprint on its own. And I guess it could be tested
independently of bhyve driver because as you might now qemu driver is
also supported on FreeBSD.

As for the bhyve, I think it's ok to push it now and mark it WIP for
example. I'd think it's not a small piece of code and will take some
time to review anyway.

But I think it cannot be merged until libvirt supports all the required
features (otherwise it'd be troublesome to setup gate jobs for bhyve for
I think).

As for the modifications to libvirt, unfortunately, they'll not get it
into the upcoming 1.2.3 as the freeze started already.

Roman Bogorodskiy


pgptY0LlVscTq.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Bogus -1 scores from turbo hipster

2014-01-09 Thread Roman Bogorodskiy
Hi,

Is it the same reason my change fails with turbo-hipster over and over:
https://review.openstack.org/#/c/54492/ ?


On Mon, Dec 30, 2013 at 1:21 PM, Michael Still mi...@stillhq.com wrote:

 Hi.

 The purpose of this email to is apologise for some incorrect -1 review
 scores which turbo hipster sent out today. I think its important when
 a third party testing tool is new to not have flakey results as people
 learn to trust the tool, so I want to explain what happened here.

 Turbo hipster is a system which takes nova code reviews, and runs
 database upgrades against them to ensure that we can still upgrade for
 users in the wild. It uses real user datasets, and also times
 migrations and warns when they are too slow for large deployments. It
 started voting on gerrit in the last week.

 Turbo hipster uses zuul to learn about reviews in gerrit that it
 should test. We run our own zuul instance, which talks to the
 openstack.org zuul instance. This then hands out work to our pool of
 testing workers. Another thing zuul does is it handles maintaining a
 git repository for the workers to clone from.

 This is where things went wrong today. For reasons I can't currently
 explain, the git repo on our zuul instance ended up in a bad state (it
 had a patch merged to master which wasn't in fact merged upstream
 yet). As this code is stock zuul from openstack-infra, I have a
 concern this might be a bug that other zuul users will see as well.

 I've corrected the problem for now, and kicked off a recheck of any
 patch with a -1 review score from turbo hipster in the last 24 hours.
 I'll talk to the zuul maintainers tomorrow about the git problem and
 see what we can learn.

 Thanks heaps for your patience.

 Michael

 --
 Rackspace Australia

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD hypervisor (bhyve) driver

2013-11-28 Thread Roman Bogorodskiy
Hello,

Yes, libvirt's qemu driver works almost fine currently, except the fact
that it
needs a 'real' bridge driver, so all the networking configuration like
filtering rules, NAT, etc
could be done automatically, like for Linux now, instead of making user to
perform
all the configuration manually.

I've been planning to get to bhyve driver as well, but probably after
finishing with the bridge driver
(but unfortunately, I don't have a full picture what would be the best way
to implement that).


On Mon, Nov 25, 2013 at 3:50 PM, Daniel P. Berrange berra...@redhat.comwrote:

 On Fri, Nov 22, 2013 at 10:46:19AM -0500, Russell Bryant wrote:
  On 11/22/2013 10:43 AM, Rafał Jaworowski wrote:
   Russell,
   First, thank you for the whiteboard input regarding the blueprint for
   FreeBSD hypervisor nova driver:
   https://blueprints.launchpad.net/nova/+spec/freebsd-compute-node
  
   We were considering libvirt support for bhyve hypervisor as well, only
   wouldn't want to do this as the first approach for FreeBSD+OpenStack
   integration. We'd rather bring bhyve bindings for libvirt later as
   another integration option.
  
   For FreeBSD host support a native hypervisor driver is important and
   desired long-term and we would like to have it anyways. Among things
   to consider are the following:
   - libvirt package is additional (non-OpenStack), external dependency
   (maintained in the 'ports' collection, not included in base system),
   while native API (via libvmmapi.so library) is integral part of the
   base system.
   - libvirt license is LGPL, which might be an important aspect for some
 users.
 
  That's perfectly fine if you want to go that route as a first step.
  However, that doesn't mean it's appropriate for merging into Nova.
  Unless there are strong technical justifications for why this approach
  should be taken, I would probably turn down this driver until you were
  able to go the libvirt route.

 The idea of a FreeBSD bhyve driver for libvirt has been mentioned
 a few times. We've already got a FreeBSD port of libvirt being
 actively maintained to support QEMU (and possibly Xen, not 100% sure
 on that one), and we'd be more than happy to see further contributions
 such as a bhyve driver.

 I am of course biased, as libvirt project maintainer, but I do agree
 that supporting bhyve via libvirt would make sense, since it opens up
 opportunities beyond just OpenStack. There are a bunch of applications
 built on libvirt that could be used to manage bhyve, and a fair few
 applications which have plugins using libvirt

 Taking on maint work for a new OpenStack driver is a non-trivial amount
 of work in itself. If the burden for OpenStack maintainers can be reduced
 by, pushing work out to / relying on support from, libvirt, that makes
 sense from OpenStack/Nova's POV.

 Regards,
 Daniel
 --
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/:|
 |: http://libvirt.org  -o- http://virt-manager.org:|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/:|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc:|

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FreeBSD hypervisor (bhyve) driver

2013-11-28 Thread Roman Bogorodskiy
On Wed, Nov 27, 2013 at 7:32 PM, Rafał Jaworowski r...@semihalf.com wrote:

 The maintenance aspect and testing coverage are valid points, on the
 other hand future changes would have to go a longer way for us: first
 upstream to libvirt, then downstream to the FreeBSD ports collection
 (+ perhaps some OpenStack code bits as well), which makes the process
 more complicated.


I don't think that there would be a huge problem with that, because libvirt
releases quite often, and Jason Helfman, who maintains libvirt package in
FreeBSD ports tree, always updates to the new versions on time.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Cinder][Glance] OSLO update

2013-11-20 Thread Roman Bogorodskiy
I know it was brought up on the list a number of times, but...

If we're talking about storing commit ids for each module and writing
some shell scripts for that, isn't it a chance to reconsider using git
submodules?




On Wed, Nov 20, 2013 at 12:37 PM, Elena Ezhova eezh...@mirantis.com wrote:


 20.11.2013, 06:18, John Griffith john.griff...@solidfire.com:


 On Mon, Nov 18, 2013 at 3:53 PM, Mark McLoughlin mar...@redhat.com
 wrote:

   On Mon, 2013-11-18 at 17:24 +, Duncan Thomas wrote:
   Random OSLO updates with no list of what changed, what got fixed etc
   are unlikely to get review attention - doing such a review is
   extremely difficult. I was -2ing them and asking for more info, but
   they keep popping up. I'm really not sure what the best way of
   updating from OSLO is, but this isn't it.
   Best practice is to include a list of changes being synced, for example:
 
 https://review.openstack.org/54660
 
   Every so often, we throw around ideas for automating the generation of
   this changes list - e.g. cinder would have the oslo-incubator commit ID
   for each module stored in a file in git to tell us when it was last
   synced.
 
   Mark.
 
   _

 __
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.org
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 Been away on vacation so I'm afraid I'm a bit late on this... but;

 I think the point Duncan is bringing up here is that there are some
 VERY large and significant patches coming from OSLO pulls.  The DB
 patch in particular being over 1K lines of code to a critical portion
 of the code is a bit unnerving to try and do a review on.  I realize
 that there's a level of trust that goes with the work that's done in
 OSLO and synchronizing those changes across the projects, but I think
 a few key concerns here are:

 1. Doing huge pulls from OSLO like the DB patch here are nearly
 impossible to thoroughly review and test.  Over time we learn a lot
 about real usage scenarios and the database and tweak things as we go,
 so seeing a patch set like this show up is always a bit unnerving and
 frankly nobody is overly excited to review it.

 2. Given a certain level of *trust* for the work that folks do on the
 OSLO side in submitting these patches and new additions, I think some
 of the responsibility on the review of the code falls on the OSLO
 team.  That being said there is still the issue of how these changes
 will impact projects *other* than Nova which I think is sometimes
 neglected.  There have been a number of OSLO synchs pushed to Cinder
 that fail gating jobs, some get fixed, some get abandoned, but in
 either case it shows that there wasn't any testing done with projects
 other than Nova (PLEASE note, I'm not referring to this particular
 round of patches or calling any patch set out, just stating a
 historical fact).

 3. We need better documentation in commit messages explaining why the
 changes are necessary and what they do for us.  I'm sorry but in my
 opinion the answer it's the latest in OSLO and Nova already has it
 is not enough of an answer in my opinion.  The patches mentioned in
 this thread in my opinion met the minimum requirements because they at
 least reference the OSLO commit which is great.  In addition I'd like
 to see something to address any discovered issues or testing done with
 the specific projects these changes are being synced to.

 I'm in no way saying I don't want Cinder to play nice with the common
 code or to get in line with the way other projects do things but I am
 saying that I think we have a ways to go in terms of better
 communication here and in terms of OSLO code actually keeping in mind
 the entire OpenStack eco-system as opposed to just changes that were
 needed/updated in Nova.  Cinder in particular went through some pretty
 massive DB re-factoring and changes during Havana and there was a lot
 of really good work there but it didn't come without a cost and the
 benefits were examined and weighed pretty heavily.  I also think that
 some times the indirection introduced by adding some of the
 openstack.common code is unnecessary and in some cases makes things
 more difficult than they should be.

 I'm just not sure that we always do a very good ROI investigation or
 risk assessment on changes, and that opinion applies to ALL changes in
 OpenStack projects, not OSLO specific or anything else.

 All of that being said, a couple of those syncs on the list are
 outdated.  We should start by doing a fresh pull for these and if
 possible add some better documentation in the commit messages as to
 the justification for the patches that would be great.  We can take a
 closer look at the changes and the history behind them and try to get
 some review progress made here.  Mark mentioned some good ideas
 regarding capturing commit ID's from synchronization pulls and I'd
 like to look into that a bit as well.

 Thanks,
 

Re: [openstack-dev] check-grenade-devstack-vm job problem?

2013-10-28 Thread Roman Bogorodskiy
Joe, thanks for clarification.


On Mon, Oct 28, 2013 at 5:34 PM, Joe Gordon joe.gord...@gmail.com wrote:




 On Mon, Oct 28, 2013 at 11:10 AM, Roman Bogorodskiy 
 rbogorods...@mirantis.com wrote:

 Hi,

 I've been noticing since this morning that the check-grenade-devstack-vm
 job constantly fails.

 https://jenkins01.openstack.org/job/check-grenade-devstack-vm/

 You can see that there were no single successful completion for 5 hours
 in a raw.

 Would could be wrong with that?


 Grenade has been fixed. There are several issues.  But in short the fix
 consisted of https://review.openstack.org/#/c/54108/.

 Grenade has some asynchronous gating issues at the moment, along with
 trying to upgrade from Grizzly and not Havana.  Now that gate isn't blocked
 these issues can be resolved off to the side.

 Thank you Sean Dague and Jeremy Stanley for helping fix this.




 Roman

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][glance] Future of nova's image API

2013-08-06 Thread Roman Bogorodskiy
  Joe Gordon wrote:

 On Mon, Aug 5, 2013 at 9:11 AM, Roman Bogorodskiy rbogorods...@mirantis.com
  wrote:
 
Joe Gordon wrote:
 
   Hi All,
  
   even though Glance, has been pulled out of Nova years ago, Nova still
  has a
   images API that proxies back to Glance.  Since Nova is in the process of
   creating a new, V3, API, we know have a chance to re-evaluate this API.
  
   * Do we still need this in Nova, is there any reason to not just use
  Glance
   directly?  I have vague concerns about making Glance API publicly
   accessible, but I am not sure what the underlying reason is
 
  From the end user point of view, images are strongly tied to
  logical models nova operates with, such as servers, flavors etc. So for
  an API user, it would be move convenient to manage all these in a
  single API, IMHO.
 
 
 
 -1, I think Monty stated this well
 
 Honestly, I think we should ditch it. Glance is our image service, not
 nova, we should use it. For user-experience stuff,
 python-openstackclient should be an excellent way to expose both through
 a single tool without needing to proxy one service through another.

Well, end user doesn't have to know such internals of the cloud such as
if it's using separate image service or not. But...

 We don't want nova to be the proxy for all other services, that
 partially defeats the purpose of splitting them off.   We have better ways
 of making everything look like a single API, such as:
 
 * better python-openstackclient
 * Make sure all services can run as one endpoint, on the same port.  So a
 REST call to $IP/images/ would go to glance and a call to $IP/instances
 would go to nova

... this thing sounds reasonable.

Also, there could be many reasons not to use python-openstackclient, so
making API more complex and simplifying things for users through
openstackclient will not work for everybody.

 
   * If it is still needed in Nova today, can we remove it in the future and
   if so what is the timeline?
  
   best,
   Joe Gordon
 
   ___
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.org
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  Roman Bogorodskiy
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Roman Bogorodskiy


pgp794awItr0S.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Usage of mox through out the Openstack project.

2013-07-26 Thread Roman Bogorodskiy
  Alex Meade wrote:

 +1 to everything Russell just said and of course Blueprints for this. One for 
 #3 (changing from mox - Mock) would be good so that anyone who is bored or 
 finds this urgent can collaborate. Also, we need to make sure reviewers are 
 aware (Hopefully they are reading this).
 
 -Alex

I have created a blueprint for Nova:

https://blueprints.launchpad.net/nova/+spec/mox-to-mock-conversion

Our team has some spare cycles, so we can work on that.


 -Original Message-
 From: Russell Bryant rbry...@redhat.com
 Sent: Wednesday, July 24, 2013 2:45pm
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] Usage of mox through out the Openstack project.
 
 On 07/24/2013 02:32 PM, Kevin L. Mitchell wrote:
  On Wed, 2013-07-24 at 14:12 -0400, Chuck Short wrote:
  1. Change mox usage to more python3 friendly such as mock.
  (https://pypi.python.org/pypi/mock/1.0.1). However this will cause
  alot of code churn in the projects as we move away from mox to mock.
 
 
  2. Use the python3 fork called pymox
  (https://github.com/emonty/pymox). This project has reasonable
  compatibility with mox and is python3 compatible. Using this option
  causes less code churn. IMHO this would be the better option.
  
  My personal preference is that we move to mock; I think it is a better
  methodology, and I like its features.
  
 
 That's fine with me if everyone feels that way.  I'm afraid it's not a
 quick move because of how much we're using mox.  A practical approach
 would probably be:
 
 1) Prefer mock for new tests.
 
 2) Use suggestion #2 above to mitigate the Python 3 concern.
 
 3) Convert tests to mock over time, opportunistically, as tests are
 being updated anyway.  (Or if someone *really* wants to take this on as
 a project ...)
 
 -- 
 Russell Bryant

Roman Bogorodskiy


pgpWfqjbowTNu.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev