Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
Excerpts from Herndon, John Luke's message of 2014-01-03 08:17:15 -0800: On 1/2/14, 7:06 PM, Sean Dague s...@dague.net wrote: On 01/02/2014 08:36 PM, Robert Collins wrote: On 3 January 2014 14:34, Robert Collins robe...@robertcollins.net wrote: On 3 January 2014 12:40, Herndon, John Luke john.hern...@hp.com wrote: On 1/2/14, 4:27 PM, Clint Byrum cl...@fewbar.com wrote: I don¹t think it would be that hard to get the review or gate jobs to use a real vertica instance, actually. Who do I talk to about that? http://ci.openstack.org/third_party.html Oh, if you meant setting up a gate variant to use vertica community edition - I'd run it past the ceilometer folk and then just submit patches to devstack, devstack-gate and infra/config to do it. devstack - code for setting up a real vertica devstack-gate - handles passing the right flags to devstack for the configuration scenarios we test against infra/config - has the jenkins job builder definitions to define the jobs I think general policy (thus far) has been that we're not going to put non Open Source software into upstream gate jobs. So you really should approach this via 3rd party testing instead. The DB2 folks are approaching it that way, for that reason. Ok, that makes sense, but tbh, setting up 3rd party testing is going to be as much or more work than writing the driver. Given schedule constraints, it probably isn¹t feasible right now. I think for starters, I will write some unit tests that ensure that changes to the storage interface don¹t break the driver, and will work on a 3rd party testing strategy over time. From my perspective, I would -2 any change to projects I'm a core on if they just add a 3rd party driver without a commitment to testing. After seeing what Nova went through with this, it doesn't seem like a should but a must. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
On 1/2/14, 7:06 PM, Sean Dague s...@dague.net wrote: On 01/02/2014 08:36 PM, Robert Collins wrote: On 3 January 2014 14:34, Robert Collins robe...@robertcollins.net wrote: On 3 January 2014 12:40, Herndon, John Luke john.hern...@hp.com wrote: On 1/2/14, 4:27 PM, Clint Byrum cl...@fewbar.com wrote: I don¹t think it would be that hard to get the review or gate jobs to use a real vertica instance, actually. Who do I talk to about that? http://ci.openstack.org/third_party.html Oh, if you meant setting up a gate variant to use vertica community edition - I'd run it past the ceilometer folk and then just submit patches to devstack, devstack-gate and infra/config to do it. devstack - code for setting up a real vertica devstack-gate - handles passing the right flags to devstack for the configuration scenarios we test against infra/config - has the jenkins job builder definitions to define the jobs I think general policy (thus far) has been that we're not going to put non Open Source software into upstream gate jobs. So you really should approach this via 3rd party testing instead. The DB2 folks are approaching it that way, for that reason. Ok, that makes sense, but tbh, setting up 3rd party testing is going to be as much or more work than writing the driver. Given schedule constraints, it probably isn¹t feasible right now. I think for starters, I will write some unit tests that ensure that changes to the storage interface don¹t break the driver, and will work on a 3rd party testing strategy over time. Thanks! -john -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer] Vertica Storage Driver Testing
Hi, I¹m working on adding a vertica (www.vertica.com) storage driver to ceilometer. I would love to get this driver into upstream. However, I¹ve run into a bit of a snag with the tests. It looks like all of the existing storage drivers have ³in-memory² versions that are used for unit tests. Vertica does not have an in-memory implementation, and is not trivial to set-up. Given this constraint, I don¹t think it will be possible to run unit tests ³out-of-the-box² against a real vertica database. Vertica is mostly sql compliant, so I could use a sqlite or h2 backend to test the query parts of the driver. Data loading can¹t be done with sqlite, and will probably need to be tested with mocks. Is this an acceptable approach for unit tests, or do the tests absolutely need to run against the database under test? Thanks! -john smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
Excerpts from Herndon, John Luke's message of 2014-01-02 15:16:26 -0800: Hi, I¹m working on adding a vertica (www.vertica.com) storage driver to ceilometer. I would love to get this driver into upstream. However, I¹ve run into a bit of a snag with the tests. It looks like all of the existing storage drivers have ³in-memory² versions that are used for unit tests. Vertica does not have an in-memory implementation, and is not trivial to set-up. Given this constraint, I don¹t think it will be possible to run unit tests ³out-of-the-box² against a real vertica database. Well arguably those other implementations aren't really running against a real database either so I don't see a problem with this. Vertica is mostly sql compliant, so I could use a sqlite or h2 backend to test the query parts of the driver. Data loading can¹t be done with sqlite, and will probably need to be tested with mocks. Is this an acceptable approach for unit tests, or do the tests absolutely need to run against the database under test? A fake Vertica or mocking it out seems like a good idea. I'm not deeply involved with Ceilometer, but in general I think it is preferable to test only the _code_ in unit tests. However, it may be a good idea to adopt an approach similar to Nova's approach and require that a 3rd party run Vertica integration tests in the gate. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
On 1/2/14, 4:27 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Herndon, John Luke's message of 2014-01-02 15:16:26 -0800: Hi, I¹m working on adding a vertica (www.vertica.com) storage driver to ceilometer. I would love to get this driver into upstream. However, I¹ve run into a bit of a snag with the tests. It looks like all of the existing storage drivers have ³in-memory² versions that are used for unit tests. Vertica does not have an in-memory implementation, and is not trivial to set-up. Given this constraint, I don¹t think it will be possible to run unit tests ³out-of-the-box² against a real vertica database. Well arguably those other implementations aren't really running against a real database either so I don't see a problem with this. Vertica is mostly sql compliant, so I could use a sqlite or h2 backend to test the query parts of the driver. Data loading can¹t be done with sqlite, and will probably need to be tested with mocks. Is this an acceptable approach for unit tests, or do the tests absolutely need to run against the database under test? A fake Vertica or mocking it out seems like a good idea. I'm not deeply involved with Ceilometer, but in general I think it is preferable to test only the _code_ in unit tests. However, it may be a good idea to adopt an approach similar to Nova's approach and require that a 3rd party run Vertica integration tests in the gate. I don’t think it would be that hard to get the review or gate jobs to use a real vertica instance, actually. Who do I talk to about that? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
On Thu, Jan 2, 2014 at 5:36 PM, Robert Collins robe...@robertcollins.net wrote: On 3 January 2014 14:34, Robert Collins robe...@robertcollins.net wrote: On 3 January 2014 12:40, Herndon, John Luke john.hern...@hp.com wrote: On 1/2/14, 4:27 PM, Clint Byrum cl...@fewbar.com wrote: I don’t think it would be that hard to get the review or gate jobs to use a real vertica instance, actually. Who do I talk to about that? http://ci.openstack.org/third_party.html Oh, if you meant setting up a gate variant to use vertica community edition - I'd run it past the ceilometer folk and then just submit patches to devstack, devstack-gate and infra/config to do it. devstack - code for setting up a real vertica devstack-gate - handles passing the right flags to devstack for the configuration scenarios we test against infra/config - has the jenkins job builder definitions to define the jobs -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I don't think Veritca is open source. There is a community edition that requires registration to use. This is problematic for upstream gating, but seems like a good candidate for third party testing. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Vertica Storage Driver Testing
On 01/02/2014 08:36 PM, Robert Collins wrote: On 3 January 2014 14:34, Robert Collins robe...@robertcollins.net wrote: On 3 January 2014 12:40, Herndon, John Luke john.hern...@hp.com wrote: On 1/2/14, 4:27 PM, Clint Byrum cl...@fewbar.com wrote: I don’t think it would be that hard to get the review or gate jobs to use a real vertica instance, actually. Who do I talk to about that? http://ci.openstack.org/third_party.html Oh, if you meant setting up a gate variant to use vertica community edition - I'd run it past the ceilometer folk and then just submit patches to devstack, devstack-gate and infra/config to do it. devstack - code for setting up a real vertica devstack-gate - handles passing the right flags to devstack for the configuration scenarios we test against infra/config - has the jenkins job builder definitions to define the jobs I think general policy (thus far) has been that we're not going to put non Open Source software into upstream gate jobs. So you really should approach this via 3rd party testing instead. The DB2 folks are approaching it that way, for that reason. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
