Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-26 Thread Karthik Natarajan
Hi Edgar,

We are also facing CI issues when the neutron patch set is not rebased with 
latest changes.
For e.g. CI audit that you posted today 
(https://review.openstack.org/#/c/114393/) is not rebased with neutron test_lib 
related changes.
We had refactored the Brocade Vyatta plugin unit tests to accommodate the 
test_lib related changes.
But our plugin is not compatible with the patch you have posted. So CI is 
failing.

I had a discussion with Dane Leblanc on this. We also need to post the SKIPPED 
status for such patch sets.
We will also experiment with Kevin's suggestion.

Thanks,
Karthik

-Original Message-
From: Dane Leblanc (leblancd) [mailto:lebla...@cisco.com]
Sent: Monday, August 25, 2014 10:02 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Edgar, Kyle:

Kevin's suggestion should work for me (still hashing out the implementation).  
I've added an item to the 3rd Party IRC agenda anyway to discuss this corner 
case.

Thanks!
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Monday, August 25, 2014 12:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Dane,

I will second Kyle's idea. Let's discuss this during today IRC meeting if 
Kevin's suggestion does not work for you.

Thanks,

Edgar

On 8/25/14, 10:08 AM, Kyle Mestery mest...@mestery.com wrote:

Dane, thanks for all the great work you're doing in the third-party CI
area. It's great to see you working to share this knowledge with others
as well!

Did Kevin's idea work for you to move past this issue? If not, I
suggest you put an item on the neutron meeting agenda today and we
cover this there. You could put the item on the third-party meeting
agenda as well.

Thanks!
Kyle

On Sun, Aug 24, 2014 at 9:43 AM, Dane Leblanc (leblancd)
lebla...@cisco.com wrote:
 Hi Kevin:



 Thanks, this is a great idea! I may try just a slight variation of
this  concept. Maybe your idea could be the recommended way to create
a 3rd party  CI for plugins that are just being introduced and need to
limit the scope of  testing to a small set of plugin-related commits
(or plugins blocked on a  certain fix).



 Thanks,
 Dane



 From: Kevin Benton [mailto:blak...@gmail.com]
 Sent: Saturday, August 23, 2014 5:47 AM


 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required  to be run



 Can you disable posting of results directly from your Jenkins/Zuul
setup and  have a script that just checks the log file for special
markers to determine  if the vote should be FAILED/PASSED/SKIPPED?
Another advantage of this  approach is that it gives you an
opportunity to detect when a job just  failed to setup due to
infrastructure reasons and trigger a recheck without  ever first
posting a failure to gerrit.



 On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd)
 lebla...@cisco.com wrote:

 Thanks Edgar for updating the APIC status!!!

 Edgar and Kyle: *PLEASE NOTE**  I need your understanding
and  advice on the following:

 We are still stuck with a problem stemming from a design limitation
 of Jenkins that prevents us from being compliant with Neutron 3rd
 Party CI requirements for our DFA CI.

 The issue is that Jenkins only allows our scripts to
(programmatically)  return either Success or Fail. There is no option
to return Aborted, Not  Tested, or Skipped.

 Why does this matter? The DFA plugin is just being introduced, and
initial  DFA-enabling change sets have not yet been merged. Therefore,
all other  change sets will fail our Tempest tests, since they are not
DFA-enabled.

 Similarly, we were recently blocked in our APIC CI with a critical
 bug, causing all change sets without this fix to fail on our APIC testbed.

 In these cases, we would like to enter a throttled or partially
blocked
 mode, where we would skip testing on change sets we know will fail,
and (in  an ideal world) signal this shortcoming to Gerrit e.g. by
returning a  Skipped status. Unfortunately, this option is not
available in Jenkins  scripts, as Jenkins is currently designed. The
only options we have  available is Success or all Fail, which are
both misleading. We would  also incorrectly report success or fail on
one of the following test
 commits:
 https://review.openstack.org/#/c/114393/
 https://review.openstack.org/#/c/40296/

 I've brought this issue up on the openstack-infra IRC, and jeblair
confirmed  the Jenkins limitation, but asked me to get consensus from
the Neutron  community as to this being a problem/requirement. I've
also sent out an  e-mail on the Neutron ML trying to start a
discussion on this problem (no  traction). I plan on bringing this up
in the 3rd Party CI IRC on Monday,  assuming there is time permitted

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-26 Thread Salvatore Orlando
Hi Karthik,

what do you mean that the plugin is incompatible with
https://review.openstack.org/#/c/114393/?
you're mentioning a rebase issue - but the patch in question appears to
cleanly apply to master.

Is your probably because patch #114393 does not have in its log some
changes you need to accommodate test_lib changes?
Are these changes you need already merged? In this case you might try to
rebase the patch you're going to test on master before running devstack,
which I think it's also what happens in the upstream gate.

Salvatore



On 26 August 2014 21:57, Karthik Natarajan natar...@brocade.com wrote:

 Hi Edgar,

 We are also facing CI issues when the neutron patch set is not rebased
 with latest changes.
 For e.g. CI audit that you posted today (
 https://review.openstack.org/#/c/114393/) is not rebased with neutron
 test_lib related changes.
 We had refactored the Brocade Vyatta plugin unit tests to accommodate the
 test_lib related changes.
 But our plugin is not compatible with the patch you have posted. So CI is
 failing.

 I had a discussion with Dane Leblanc on this. We also need to post the
 SKIPPED status for such patch sets.
 We will also experiment with Kevin's suggestion.

 Thanks,
 Karthik

 -Original Message-
 From: Dane Leblanc (leblancd) [mailto:lebla...@cisco.com]
 Sent: Monday, August 25, 2014 10:02 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

 Edgar, Kyle:

 Kevin's suggestion should work for me (still hashing out the
 implementation).  I've added an item to the 3rd Party IRC agenda anyway to
 discuss this corner case.

 Thanks!
 Dane

 -Original Message-
 From: Edgar Magana [mailto:edgar.mag...@workday.com]
 Sent: Monday, August 25, 2014 12:44 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

 Dane,

 I will second Kyle's idea. Let's discuss this during today IRC meeting if
 Kevin's suggestion does not work for you.

 Thanks,

 Edgar

 On 8/25/14, 10:08 AM, Kyle Mestery mest...@mestery.com wrote:

 Dane, thanks for all the great work you're doing in the third-party CI
 area. It's great to see you working to share this knowledge with others
 as well!
 
 Did Kevin's idea work for you to move past this issue? If not, I
 suggest you put an item on the neutron meeting agenda today and we
 cover this there. You could put the item on the third-party meeting
 agenda as well.
 
 Thanks!
 Kyle
 
 On Sun, Aug 24, 2014 at 9:43 AM, Dane Leblanc (leblancd)
 lebla...@cisco.com wrote:
  Hi Kevin:
 
 
 
  Thanks, this is a great idea! I may try just a slight variation of
 this  concept. Maybe your idea could be the recommended way to create
 a 3rd party  CI for plugins that are just being introduced and need to
 limit the scope of  testing to a small set of plugin-related commits
 (or plugins blocked on a  certain fix).
 
 
 
  Thanks,
  Dane
 
 
 
  From: Kevin Benton [mailto:blak...@gmail.com]
  Sent: Saturday, August 23, 2014 5:47 AM
 
 
  To: OpenStack Development Mailing List (not for usage questions)
  Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required  to be run
 
 
 
  Can you disable posting of results directly from your Jenkins/Zuul
 setup and  have a script that just checks the log file for special
 markers to determine  if the vote should be FAILED/PASSED/SKIPPED?
 Another advantage of this  approach is that it gives you an
 opportunity to detect when a job just  failed to setup due to
 infrastructure reasons and trigger a recheck without  ever first
 posting a failure to gerrit.
 
 
 
  On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd)
  lebla...@cisco.com wrote:
 
  Thanks Edgar for updating the APIC status!!!
 
  Edgar and Kyle: *PLEASE NOTE**  I need your understanding
 and  advice on the following:
 
  We are still stuck with a problem stemming from a design limitation
  of Jenkins that prevents us from being compliant with Neutron 3rd
  Party CI requirements for our DFA CI.
 
  The issue is that Jenkins only allows our scripts to
 (programmatically)  return either Success or Fail. There is no option
 to return Aborted, Not  Tested, or Skipped.
 
  Why does this matter? The DFA plugin is just being introduced, and
 initial  DFA-enabling change sets have not yet been merged. Therefore,
 all other  change sets will fail our Tempest tests, since they are not
 DFA-enabled.
 
  Similarly, we were recently blocked in our APIC CI with a critical
  bug, causing all change sets without this fix to fail on our APIC
 testbed.
 
  In these cases, we would like to enter a throttled or partially
 blocked
  mode, where we would skip testing on change sets we know will fail,
 and (in  an ideal world) signal this shortcoming to Gerrit e.g. by
 returning a  Skipped status. Unfortunately, this option

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-26 Thread Jeremy Stanley
On 2014-08-27 00:52:35 +0200 (+0200), Salvatore Orlando wrote:
[...]
 In this case you might try to rebase the patch you're going to
 test on master before running devstack, which I think it's also
 what happens in the upstream gate.

More or less. Technically we try to merge the proposed change (and
its open parent changes in Gerrit if any, in series) onto the
current target branch tip. Rebasing/cherry-picking are a little
muddier since they alter the commits being tested rather than merely
adding merge commits to the mix.

Also, upstream's Zuul (the Jenkins account in Gerrit) now
automatically runs a built-in job against all open changes for a
project+branch every time a change is merged for that combination,
testing to see if the remaining open changes can still merge without
conflict and leaves a -1 verify on them with a relevant message if
not. So if a third-party CI is encountering changes which really
can't be merged (rebased, cherry-picked, whatever) to the project
then they should refrain from commenting at all... at this point it
would be at best redundant, and at worst entirely incorrect.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-25 Thread Kyle Mestery
Dane, thanks for all the great work you're doing in the third-party CI
area. It's great to see you working to share this knowledge with
others as well!

Did Kevin's idea work for you to move past this issue? If not, I
suggest you put an item on the neutron meeting agenda today and we
cover this there. You could put the item on the third-party meeting
agenda as well.

Thanks!
Kyle

On Sun, Aug 24, 2014 at 9:43 AM, Dane Leblanc (leblancd)
lebla...@cisco.com wrote:
 Hi Kevin:



 Thanks, this is a great idea! I may try just a slight variation of this
 concept. Maybe your idea could be the recommended way to create a 3rd party
 CI for plugins that are just being introduced and need to limit the scope of
 testing to a small set of plugin-related commits (or plugins blocked on a
 certain fix).



 Thanks,
 Dane



 From: Kevin Benton [mailto:blak...@gmail.com]
 Sent: Saturday, August 23, 2014 5:47 AM


 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required
 to be run



 Can you disable posting of results directly from your Jenkins/Zuul setup and
 have a script that just checks the log file for special markers to determine
 if the vote should be FAILED/PASSED/SKIPPED? Another advantage of this
 approach is that it gives you an opportunity to detect when a job just
 failed to setup due to infrastructure reasons and trigger a recheck without
 ever first posting a failure to gerrit.



 On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd)
 lebla...@cisco.com wrote:

 Thanks Edgar for updating the APIC status!!!

 Edgar and Kyle: *PLEASE NOTE**  I need your understanding and
 advice on the following:

 We are still stuck with a problem stemming from a design limitation of
 Jenkins that prevents us from being compliant with Neutron 3rd Party CI
 requirements for our DFA CI.

 The issue is that Jenkins only allows our scripts to (programmatically)
 return either Success or Fail. There is no option to return Aborted, Not
 Tested, or Skipped.

 Why does this matter? The DFA plugin is just being introduced, and initial
 DFA-enabling change sets have not yet been merged. Therefore, all other
 change sets will fail our Tempest tests, since they are not DFA-enabled.

 Similarly, we were recently blocked in our APIC CI with a critical bug,
 causing all change sets without this fix to fail on our APIC testbed.

 In these cases, we would like to enter a throttled or partially blocked
 mode, where we would skip testing on change sets we know will fail, and (in
 an ideal world) signal this shortcoming to Gerrit e.g. by returning a
 Skipped status. Unfortunately, this option is not available in Jenkins
 scripts, as Jenkins is currently designed. The only options we have
 available is Success or all Fail, which are both misleading. We would
 also incorrectly report success or fail on one of the following test
 commits:
 https://review.openstack.org/#/c/114393/
 https://review.openstack.org/#/c/40296/

 I've brought this issue up on the openstack-infra IRC, and jeblair confirmed
 the Jenkins limitation, but asked me to get consensus from the Neutron
 community as to this being a problem/requirement. I've also sent out an
 e-mail on the Neutron ML trying to start a discussion on this problem (no
 traction). I plan on bringing this up in the 3rd Party CI IRC on Monday,
 assuming there is time permitted in the open discussion.

 I'm also investigating

 For the short term, I would like to propose the following:
 * We bring this up on the 3rd Party CI IRC on Monday to get a solution or
 workaround, if available. If a solution is available, let's consider
 including that as a hint when we come up with CI requirements for handling
 CIs bocked by some critical fix.
 * I'm also looking into using a REST API to cancel a Jenkins job
 programmatically.
 * If no solution or workaround is available, we work with infra team or with
 Jenkins team to create a solution.
 * Until a solution is available, for plugins which are blocked by a critical
 bug, we post a status/notes indicating the plugin's situation on our 3rd
 party CI status wiki, e.g.:

 Vendor  Plugin/Driver Name  Contact Name
 Status  Notes
 My Vendor Name  My Plugin CIMy Contact Person   T
 Throttled / Partially blocked / Awaiting Intial Commits

 The status/notes should be clear and understood by the Neutron team.  The
 console logs for change sets where the tests were skipped should also
 contain a message that all testing is being skipped for that commit.

 Note that when the DFA initial commits are merged, then this issue would go
 away for the DFA CI. However, this problem will reappear every time a
 blocking critical bug shows up for a 3rd party CI setup, or a new plugin is
 introduced and the hardware-enabling commits are not yet merged.  (That is,
 until we have a solution for the Jenkins limitation).

 Let me know what you think

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-25 Thread Edgar Magana
Dane,

I will second Kyle's idea. Let's discuss this during today IRC meeting if
Kevin's suggestion does not work for you.

Thanks,

Edgar

On 8/25/14, 10:08 AM, Kyle Mestery mest...@mestery.com wrote:

Dane, thanks for all the great work you're doing in the third-party CI
area. It's great to see you working to share this knowledge with
others as well!

Did Kevin's idea work for you to move past this issue? If not, I
suggest you put an item on the neutron meeting agenda today and we
cover this there. You could put the item on the third-party meeting
agenda as well.

Thanks!
Kyle

On Sun, Aug 24, 2014 at 9:43 AM, Dane Leblanc (leblancd)
lebla...@cisco.com wrote:
 Hi Kevin:



 Thanks, this is a great idea! I may try just a slight variation of this
 concept. Maybe your idea could be the recommended way to create a 3rd
party
 CI for plugins that are just being introduced and need to limit the
scope of
 testing to a small set of plugin-related commits (or plugins blocked on
a
 certain fix).



 Thanks,
 Dane



 From: Kevin Benton [mailto:blak...@gmail.com]
 Sent: Saturday, August 23, 2014 5:47 AM


 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required
 to be run



 Can you disable posting of results directly from your Jenkins/Zuul
setup and
 have a script that just checks the log file for special markers to
determine
 if the vote should be FAILED/PASSED/SKIPPED? Another advantage of this
 approach is that it gives you an opportunity to detect when a job just
 failed to setup due to infrastructure reasons and trigger a recheck
without
 ever first posting a failure to gerrit.



 On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd)
 lebla...@cisco.com wrote:

 Thanks Edgar for updating the APIC status!!!

 Edgar and Kyle: *PLEASE NOTE**  I need your understanding
and
 advice on the following:

 We are still stuck with a problem stemming from a design limitation of
 Jenkins that prevents us from being compliant with Neutron 3rd Party CI
 requirements for our DFA CI.

 The issue is that Jenkins only allows our scripts to (programmatically)
 return either Success or Fail. There is no option to return Aborted,
Not
 Tested, or Skipped.

 Why does this matter? The DFA plugin is just being introduced, and
initial
 DFA-enabling change sets have not yet been merged. Therefore, all other
 change sets will fail our Tempest tests, since they are not DFA-enabled.

 Similarly, we were recently blocked in our APIC CI with a critical bug,
 causing all change sets without this fix to fail on our APIC testbed.

 In these cases, we would like to enter a throttled or partially
blocked
 mode, where we would skip testing on change sets we know will fail, and
(in
 an ideal world) signal this shortcoming to Gerrit e.g. by returning a
 Skipped status. Unfortunately, this option is not available in Jenkins
 scripts, as Jenkins is currently designed. The only options we have
 available is Success or all Fail, which are both misleading. We
would
 also incorrectly report success or fail on one of the following test
 commits:
 https://review.openstack.org/#/c/114393/
 https://review.openstack.org/#/c/40296/

 I've brought this issue up on the openstack-infra IRC, and jeblair
confirmed
 the Jenkins limitation, but asked me to get consensus from the Neutron
 community as to this being a problem/requirement. I've also sent out an
 e-mail on the Neutron ML trying to start a discussion on this problem
(no
 traction). I plan on bringing this up in the 3rd Party CI IRC on Monday,
 assuming there is time permitted in the open discussion.

 I'm also investigating

 For the short term, I would like to propose the following:
 * We bring this up on the 3rd Party CI IRC on Monday to get a solution
or
 workaround, if available. If a solution is available, let's consider
 including that as a hint when we come up with CI requirements for
handling
 CIs bocked by some critical fix.
 * I'm also looking into using a REST API to cancel a Jenkins job
 programmatically.
 * If no solution or workaround is available, we work with infra team or
with
 Jenkins team to create a solution.
 * Until a solution is available, for plugins which are blocked by a
critical
 bug, we post a status/notes indicating the plugin's situation on our 3rd
 party CI status wiki, e.g.:

 Vendor  Plugin/Driver Name  Contact Name
 Status  Notes
 My Vendor Name  My Plugin CIMy Contact Person   T
 Throttled / Partially blocked / Awaiting Intial Commits

 The status/notes should be clear and understood by the Neutron team.
The
 console logs for change sets where the tests were skipped should also
 contain a message that all testing is being skipped for that commit.

 Note that when the DFA initial commits are merged, then this issue
would go
 away for the DFA CI. However, this problem will reappear every time a
 blocking critical bug shows up for a 3rd

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-25 Thread Dane Leblanc (leblancd)
Edgar, Kyle:

Kevin's suggestion should work for me (still hashing out the implementation).  
I've added an item to the 3rd Party IRC agenda anyway to discuss this corner 
case.

Thanks!
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Monday, August 25, 2014 12:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Dane,

I will second Kyle's idea. Let's discuss this during today IRC meeting if 
Kevin's suggestion does not work for you.

Thanks,

Edgar

On 8/25/14, 10:08 AM, Kyle Mestery mest...@mestery.com wrote:

Dane, thanks for all the great work you're doing in the third-party CI 
area. It's great to see you working to share this knowledge with others 
as well!

Did Kevin's idea work for you to move past this issue? If not, I 
suggest you put an item on the neutron meeting agenda today and we 
cover this there. You could put the item on the third-party meeting 
agenda as well.

Thanks!
Kyle

On Sun, Aug 24, 2014 at 9:43 AM, Dane Leblanc (leblancd) 
lebla...@cisco.com wrote:
 Hi Kevin:



 Thanks, this is a great idea! I may try just a slight variation of 
this  concept. Maybe your idea could be the recommended way to create 
a 3rd party  CI for plugins that are just being introduced and need to 
limit the scope of  testing to a small set of plugin-related commits 
(or plugins blocked on a  certain fix).



 Thanks,
 Dane



 From: Kevin Benton [mailto:blak...@gmail.com]
 Sent: Saturday, August 23, 2014 5:47 AM


 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required  to be run



 Can you disable posting of results directly from your Jenkins/Zuul 
setup and  have a script that just checks the log file for special 
markers to determine  if the vote should be FAILED/PASSED/SKIPPED? 
Another advantage of this  approach is that it gives you an 
opportunity to detect when a job just  failed to setup due to 
infrastructure reasons and trigger a recheck without  ever first 
posting a failure to gerrit.



 On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd) 
 lebla...@cisco.com wrote:

 Thanks Edgar for updating the APIC status!!!

 Edgar and Kyle: *PLEASE NOTE**  I need your understanding 
and  advice on the following:

 We are still stuck with a problem stemming from a design limitation 
 of Jenkins that prevents us from being compliant with Neutron 3rd 
 Party CI requirements for our DFA CI.

 The issue is that Jenkins only allows our scripts to 
(programmatically)  return either Success or Fail. There is no option 
to return Aborted, Not  Tested, or Skipped.

 Why does this matter? The DFA plugin is just being introduced, and 
initial  DFA-enabling change sets have not yet been merged. Therefore, 
all other  change sets will fail our Tempest tests, since they are not 
DFA-enabled.

 Similarly, we were recently blocked in our APIC CI with a critical 
 bug, causing all change sets without this fix to fail on our APIC testbed.

 In these cases, we would like to enter a throttled or partially 
blocked
 mode, where we would skip testing on change sets we know will fail, 
and (in  an ideal world) signal this shortcoming to Gerrit e.g. by 
returning a  Skipped status. Unfortunately, this option is not 
available in Jenkins  scripts, as Jenkins is currently designed. The 
only options we have  available is Success or all Fail, which are 
both misleading. We would  also incorrectly report success or fail on 
one of the following test
 commits:
 https://review.openstack.org/#/c/114393/
 https://review.openstack.org/#/c/40296/

 I've brought this issue up on the openstack-infra IRC, and jeblair 
confirmed  the Jenkins limitation, but asked me to get consensus from 
the Neutron  community as to this being a problem/requirement. I've 
also sent out an  e-mail on the Neutron ML trying to start a 
discussion on this problem (no  traction). I plan on bringing this up 
in the 3rd Party CI IRC on Monday,  assuming there is time permitted 
in the open discussion.

 I'm also investigating

 For the short term, I would like to propose the following:
 * We bring this up on the 3rd Party CI IRC on Monday to get a 
solution or  workaround, if available. If a solution is available, 
let's consider  including that as a hint when we come up with CI 
requirements for handling  CIs bocked by some critical fix.
 * I'm also looking into using a REST API to cancel a Jenkins job  
programmatically.
 * If no solution or workaround is available, we work with infra team 
or with  Jenkins team to create a solution.
 * Until a solution is available, for plugins which are blocked by a 
critical  bug, we post a status/notes indicating the plugin's 
situation on our 3rd  party CI status wiki, e.g.:

 Vendor  Plugin/Driver Name  Contact Name
 Status  Notes
 My Vendor Name

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-24 Thread Dane Leblanc (leblancd)
Hi Kevin:

Thanks, this is a great idea! I may try just a slight variation of this 
concept. Maybe your idea could be the recommended way to create a 3rd party CI 
for plugins that are just being introduced and need to limit the scope of 
testing to a small set of plugin-related commits (or plugins blocked on a 
certain fix).

Thanks,
Dane

From: Kevin Benton [mailto:blak...@gmail.com]
Sent: Saturday, August 23, 2014 5:47 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Can you disable posting of results directly from your Jenkins/Zuul setup and 
have a script that just checks the log file for special markers to determine if 
the vote should be FAILED/PASSED/SKIPPED? Another advantage of this approach is 
that it gives you an opportunity to detect when a job just failed to setup due 
to infrastructure reasons and trigger a recheck without ever first posting a 
failure to gerrit.

On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd) 
lebla...@cisco.commailto:lebla...@cisco.com wrote:
Thanks Edgar for updating the APIC status!!!

Edgar and Kyle: *PLEASE NOTE**  I need your understanding and 
advice on the following:

We are still stuck with a problem stemming from a design limitation of Jenkins 
that prevents us from being compliant with Neutron 3rd Party CI requirements 
for our DFA CI.

The issue is that Jenkins only allows our scripts to (programmatically) return 
either Success or Fail. There is no option to return Aborted, Not Tested, 
or Skipped.

Why does this matter? The DFA plugin is just being introduced, and initial 
DFA-enabling change sets have not yet been merged. Therefore, all other change 
sets will fail our Tempest tests, since they are not DFA-enabled.

Similarly, we were recently blocked in our APIC CI with a critical bug, causing 
all change sets without this fix to fail on our APIC testbed.

In these cases, we would like to enter a throttled or partially blocked 
mode, where we would skip testing on change sets we know will fail, and (in an 
ideal world) signal this shortcoming to Gerrit e.g. by returning a Skipped 
status. Unfortunately, this option is not available in Jenkins scripts, as 
Jenkins is currently designed. The only options we have available is Success 
or all Fail, which are both misleading. We would also incorrectly report 
success or fail on one of the following test commits:
https://review.openstack.org/#/c/114393/
https://review.openstack.org/#/c/40296/

I've brought this issue up on the openstack-infra IRC, and jeblair confirmed 
the Jenkins limitation, but asked me to get consensus from the Neutron 
community as to this being a problem/requirement. I've also sent out an e-mail 
on the Neutron ML trying to start a discussion on this problem (no traction). I 
plan on bringing this up in the 3rd Party CI IRC on Monday, assuming there is 
time permitted in the open discussion.

I'm also investigating

For the short term, I would like to propose the following:
* We bring this up on the 3rd Party CI IRC on Monday to get a solution or 
workaround, if available. If a solution is available, let's consider including 
that as a hint when we come up with CI requirements for handling CIs bocked by 
some critical fix.
* I'm also looking into using a REST API to cancel a Jenkins job 
programmatically.
* If no solution or workaround is available, we work with infra team or with 
Jenkins team to create a solution.
* Until a solution is available, for plugins which are blocked by a critical 
bug, we post a status/notes indicating the plugin's situation on our 3rd party 
CI status wiki, e.g.:

Vendor  Plugin/Driver Name  Contact NameStatus  
Notes
My Vendor Name  My Plugin CIMy Contact Person   T   
Throttled / Partially blocked / Awaiting Intial Commits

The status/notes should be clear and understood by the Neutron team.  The 
console logs for change sets where the tests were skipped should also contain a 
message that all testing is being skipped for that commit.

Note that when the DFA initial commits are merged, then this issue would go 
away for the DFA CI. However, this problem will reappear every time a blocking 
critical bug shows up for a 3rd party CI setup, or a new plugin is introduced 
and the hardware-enabling commits are not yet merged.  (That is, until we have 
a solution for the Jenkins limitation).

Let me know what you think.

Thanks,
Dane

-Original Message-
From: Edgar Magana 
[mailto:edgar.mag...@workday.commailto:edgar.mag...@workday.com]
Sent: Friday, August 22, 2014 1:57 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Sorry my bad but I just changed.

Edgar

On 8/21/14, 2:13 PM, Dane Leblanc (leblancd) 
lebla...@cisco.commailto:lebla...@cisco.com

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-23 Thread Kevin Benton
Can you disable posting of results directly from your Jenkins/Zuul setup
and have a script that just checks the log file for special markers to
determine if the vote should be FAILED/PASSED/SKIPPED? Another advantage of
this approach is that it gives you an opportunity to detect when a job just
failed to setup due to infrastructure reasons and trigger a recheck without
ever first posting a failure to gerrit.


On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd) lebla...@cisco.com
 wrote:

 Thanks Edgar for updating the APIC status!!!

 Edgar and Kyle: *PLEASE NOTE**  I need your understanding and
 advice on the following:

 We are still stuck with a problem stemming from a design limitation of
 Jenkins that prevents us from being compliant with Neutron 3rd Party CI
 requirements for our DFA CI.

 The issue is that Jenkins only allows our scripts to (programmatically)
 return either Success or Fail. There is no option to return Aborted, Not
 Tested, or Skipped.

 Why does this matter? The DFA plugin is just being introduced, and initial
 DFA-enabling change sets have not yet been merged. Therefore, all other
 change sets will fail our Tempest tests, since they are not DFA-enabled.

 Similarly, we were recently blocked in our APIC CI with a critical bug,
 causing all change sets without this fix to fail on our APIC testbed.

 In these cases, we would like to enter a throttled or partially
 blocked mode, where we would skip testing on change sets we know will
 fail, and (in an ideal world) signal this shortcoming to Gerrit e.g. by
 returning a Skipped status. Unfortunately, this option is not available
 in Jenkins scripts, as Jenkins is currently designed. The only options we
 have available is Success or all Fail, which are both misleading. We
 would also incorrectly report success or fail on one of the following test
 commits:
 https://review.openstack.org/#/c/114393/
 https://review.openstack.org/#/c/40296/

 I've brought this issue up on the openstack-infra IRC, and jeblair
 confirmed the Jenkins limitation, but asked me to get consensus from the
 Neutron community as to this being a problem/requirement. I've also sent
 out an e-mail on the Neutron ML trying to start a discussion on this
 problem (no traction). I plan on bringing this up in the 3rd Party CI IRC
 on Monday, assuming there is time permitted in the open discussion.

 I'm also investigating

 For the short term, I would like to propose the following:
 * We bring this up on the 3rd Party CI IRC on Monday to get a solution or
 workaround, if available. If a solution is available, let's consider
 including that as a hint when we come up with CI requirements for handling
 CIs bocked by some critical fix.
 * I'm also looking into using a REST API to cancel a Jenkins job
 programmatically.
 * If no solution or workaround is available, we work with infra team or
 with Jenkins team to create a solution.
 * Until a solution is available, for plugins which are blocked by a
 critical bug, we post a status/notes indicating the plugin's situation on
 our 3rd party CI status wiki, e.g.:

 Vendor  Plugin/Driver Name  Contact Name
 Status  Notes
 My Vendor Name  My Plugin CIMy Contact Person   T
  Throttled / Partially blocked / Awaiting Intial Commits

 The status/notes should be clear and understood by the Neutron team.  The
 console logs for change sets where the tests were skipped should also
 contain a message that all testing is being skipped for that commit.

 Note that when the DFA initial commits are merged, then this issue would
 go away for the DFA CI. However, this problem will reappear every time a
 blocking critical bug shows up for a 3rd party CI setup, or a new plugin is
 introduced and the hardware-enabling commits are not yet merged.  (That is,
 until we have a solution for the Jenkins limitation).

 Let me know what you think.

 Thanks,
 Dane

 -Original Message-
 From: Edgar Magana [mailto:edgar.mag...@workday.com]
 Sent: Friday, August 22, 2014 1:57 PM
 To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for
 usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

 Sorry my bad but I just changed.

 Edgar

 On 8/21/14, 2:13 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

 Edgar:
 
 I'm still seeing the comment Results are not accurate. Needs
 clarification...
 
 Dane
 
 -Original Message-
 From: Edgar Magana [mailto:edgar.mag...@workday.com]
 Sent: Thursday, August 21, 2014 2:58 PM
 To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not
 for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run
 
 Dane,
 
 Wiki has been updated.
 
 Thanks,
 
 Edgar
 
 On 8/21/14, 7:57 AM, Dane Leblanc (leblancd) lebla...@cisco.com
 wrote:
 
 Edgar:
 
 The status on the wiki page says Results are not accurate. Needs
 clarification from Cisco.
 Can

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-22 Thread Edgar Magana
Excellent Job!

Thanks a lot, I have updated the wiki.

Edgar

From: Hemanth Ravi hemanthrav...@gmail.commailto:hemanthrav...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Thursday, August 21, 2014 at 1:37 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Edgar,

CI name: One Convergence CI

The logs issue has been fixed and earlier tests were failing due to  
https://review.openstack.org/#/c/114146

For reference:

1. Please take a look at the vote on patchset 7 in 
https://review.openstack.org/#/c/114968/
2. Logs at https://www.dropbox.com/sh/czydzz5bn2rc2lp/AABZByV8UQUIqWaZSSrZvzvDa

Thanks,
-hemanth



On Thu, Aug 21, 2014 at 12:41 PM, Kevin Benton 
blak...@gmail.commailto:blak...@gmail.com wrote:
Our system would get backed up with patches and sometimes take up to 10 hours 
to respond with results for a change.
We should establish some maximum acceptable time to get the results for a patch.


On Thu, Aug 21, 2014 at 11:59 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Maximum time to vote, can you clarify?

Edgar

From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Tuesday, August 19, 2014 at 1:11 PM

To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run


I also added a two more nodes to our cluster to reduce the delay. Can we 
establish a maximum time to vote on the wiki?

On Aug 19, 2014 9:39 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Kevin,

I just verified, Thanks a lot.

Edgar

From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 4:56 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

You didn't wait long enough for the Big Switch CI to reach your negative test. 
:-)
 Currently the CI system is backed by about 100 patches to push responses out 
~22 hours.

This does bring up an old discussion that was never resolved.
What is the minimum expected response time for CI systems?


On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-22 Thread Edgar Magana
Sorry my bad but I just changed.

Edgar

On 8/21/14, 2:13 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

I'm still seeing the comment Results are not accurate. Needs
clarification...

Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Thursday, August 21, 2014 2:58 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for
usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Dane,

Wiki has been updated.

Thanks,

Edgar

On 8/21/14, 7:57 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The status on the wiki page says Results are not accurate. Needs
clarification from Cisco.
Can you please tell me what we are missing?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage
questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit. This seems to be an
intermittent quirk of Jenkins/Gerrit: We have 3 CIs triggered from this
Jenkins/Gerrit server. Whenever we disable another one of our other
Jenkins jobs (in this case, we disabled DFA for some rework), the
review comments sometimes stop showing up on Gerrit.

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 1:33 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, Dane Leblanc (leblancd) lebla...@cisco.com
wrote:

From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com
wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results
are fake to something like results are only valid for APIC-related
commits? I think this more accurately represents our current results
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few
plugins and drivers are running properly and testing all Neutron
commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_
P
l
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting.
So please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the
dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-22 Thread Dane Leblanc (leblancd)
Thanks Edgar for updating the APIC status!!! 

Edgar and Kyle: *PLEASE NOTE**  I need your understanding and 
advice on the following:

We are still stuck with a problem stemming from a design limitation of Jenkins 
that prevents us from being compliant with Neutron 3rd Party CI requirements 
for our DFA CI.

The issue is that Jenkins only allows our scripts to (programmatically) return 
either Success or Fail. There is no option to return Aborted, Not Tested, 
or Skipped.

Why does this matter? The DFA plugin is just being introduced, and initial 
DFA-enabling change sets have not yet been merged. Therefore, all other change 
sets will fail our Tempest tests, since they are not DFA-enabled.

Similarly, we were recently blocked in our APIC CI with a critical bug, causing 
all change sets without this fix to fail on our APIC testbed.

In these cases, we would like to enter a throttled or partially blocked 
mode, where we would skip testing on change sets we know will fail, and (in an 
ideal world) signal this shortcoming to Gerrit e.g. by returning a Skipped 
status. Unfortunately, this option is not available in Jenkins scripts, as 
Jenkins is currently designed. The only options we have available is Success 
or all Fail, which are both misleading. We would also incorrectly report 
success or fail on one of the following test commits:
https://review.openstack.org/#/c/114393/
https://review.openstack.org/#/c/40296/

I've brought this issue up on the openstack-infra IRC, and jeblair confirmed 
the Jenkins limitation, but asked me to get consensus from the Neutron 
community as to this being a problem/requirement. I've also sent out an e-mail 
on the Neutron ML trying to start a discussion on this problem (no traction). I 
plan on bringing this up in the 3rd Party CI IRC on Monday, assuming there is 
time permitted in the open discussion.

I'm also investigating 

For the short term, I would like to propose the following:
* We bring this up on the 3rd Party CI IRC on Monday to get a solution or 
workaround, if available. If a solution is available, let's consider including 
that as a hint when we come up with CI requirements for handling CIs bocked by 
some critical fix.
* I'm also looking into using a REST API to cancel a Jenkins job 
programmatically.
* If no solution or workaround is available, we work with infra team or with 
Jenkins team to create a solution.
* Until a solution is available, for plugins which are blocked by a critical 
bug, we post a status/notes indicating the plugin's situation on our 3rd party 
CI status wiki, e.g.:

Vendor  Plugin/Driver Name  Contact NameStatus  
Notes
My Vendor Name  My Plugin CIMy Contact Person   T   
Throttled / Partially blocked / Awaiting Intial Commits

The status/notes should be clear and understood by the Neutron team.  The 
console logs for change sets where the tests were skipped should also contain a 
message that all testing is being skipped for that commit.

Note that when the DFA initial commits are merged, then this issue would go 
away for the DFA CI. However, this problem will reappear every time a blocking 
critical bug shows up for a 3rd party CI setup, or a new plugin is introduced 
and the hardware-enabling commits are not yet merged.  (That is, until we have 
a solution for the Jenkins limitation).

Let me know what you think.

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Friday, August 22, 2014 1:57 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Sorry my bad but I just changed.

Edgar

On 8/21/14, 2:13 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

I'm still seeing the comment Results are not accurate. Needs 
clarification...

Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Thursday, August 21, 2014 2:58 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not 
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Dane,

Wiki has been updated.

Thanks,

Edgar

On 8/21/14, 7:57 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The status on the wiki page says Results are not accurate. Needs 
clarification from Cisco.
Can you please tell me what we are missing?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage
questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Dane Leblanc (leblancd)
Edgar:

The status on the wiki page says Results are not accurate. Needs clarification 
from Cisco.
Can you please tell me what we are missing?

-Dane

-Original Message-
From: Dane Leblanc (leblancd) 
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit. This seems to be an 
intermittent quirk of Jenkins/Gerrit: We have 3 CIs triggered from this 
Jenkins/Gerrit server. Whenever we disable another one of our other Jenkins 
jobs (in this case, we disabled DFA for some rework), the review comments 
sometimes stop showing up on Gerrit.

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 1:33 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not 
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and 
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results 
are fake to something like results are only valid for APIC-related 
commits? I think this more accurately represents our current results 
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few 
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_P
l
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. 
So please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and 
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore 
this report that plugin/driver will be candidate to be remove from 
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty 
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below 
and make sure you are running appropriate tests. If you have 
questions, the weekly third-party meeting [2] is a great place to ask 
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Edgar Magana
Dane,

Wiki has been updated.

Thanks,

Edgar

On 8/21/14, 7:57 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The status on the wiki page says Results are not accurate. Needs
clarification from Cisco.
Can you please tell me what we are missing?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage
questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit. This seems to be an
intermittent quirk of Jenkins/Gerrit: We have 3 CIs triggered from this
Jenkins/Gerrit server. Whenever we disable another one of our other
Jenkins jobs (in this case, we disabled DFA for some rework), the review
comments sometimes stop showing up on Gerrit.

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 1:33 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for
usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com
wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results
are fake to something like results are only valid for APIC-related
commits? I think this more accurately represents our current results
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few
plugins and drivers are running properly and testing all Neutron
commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_P
l
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting.
So please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Edgar Magana
Maximum time to vote, can you clarify?

Edgar

From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Tuesday, August 19, 2014 at 1:11 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run


I also added a two more nodes to our cluster to reduce the delay. Can we 
establish a maximum time to vote on the wiki?

On Aug 19, 2014 9:39 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Kevin,

I just verified, Thanks a lot.

Edgar

From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 4:56 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

You didn't wait long enough for the Big Switch CI to reach your negative test. 
:-)
 Currently the CI system is backed by about 100 patches to push responses out 
~22 hours.

This does bring up an old discussion that was never resolved.
What is the minimum expected response time for CI systems?


On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Kevin Benton

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Kevin Benton
Our system would get backed up with patches and sometimes take up to 10
hours to respond with results for a change.
We should establish some maximum acceptable time to get the results for a
patch.


On Thu, Aug 21, 2014 at 11:59 AM, Edgar Magana edgar.mag...@workday.com
wrote:

  Maximum time to vote, can you clarify?

  Edgar

   From: Kevin Benton blak...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Tuesday, August 19, 2014 at 1:11 PM

 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   I also added a two more nodes to our cluster to reduce the delay. Can
 we establish a maximum time to vote on the wiki?
 On Aug 19, 2014 9:39 AM, Edgar Magana edgar.mag...@workday.com wrote:

  Kevin,

  I just verified, Thanks a lot.

  Edgar

   From: Kevin Benton blak...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: Friday, August 15, 2014 at 4:56 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   You didn't wait long enough for the Big Switch CI to reach your
 negative test. :-)
  Currently the CI system is backed by about 100 patches to push responses
 out ~22 hours.

  This does bring up an old discussion that was never resolved.
 What is the minimum expected response time for CI systems?


 On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana edgar.mag...@workday.com
 wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:

 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore
 this
 report that plugin/driver will be candidate to be remove from Neutron
 tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




  --
 Kevin Benton

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Hemanth Ravi
Edgar,

CI name: One Convergence CI

The logs issue has been fixed and earlier tests were failing due to
https://review.openstack.org/#/c/114146

For reference:

1. Please take a look at the vote on patchset 7 in
https://review.openstack.org/#/c/114968/
2. Logs at
https://www.dropbox.com/sh/czydzz5bn2rc2lp/AABZByV8UQUIqWaZSSrZvzvDa

Thanks,
-hemanth



On Thu, Aug 21, 2014 at 12:41 PM, Kevin Benton blak...@gmail.com wrote:

 Our system would get backed up with patches and sometimes take up to 10
 hours to respond with results for a change.
 We should establish some maximum acceptable time to get the results for a
 patch.


 On Thu, Aug 21, 2014 at 11:59 AM, Edgar Magana edgar.mag...@workday.com
 wrote:

  Maximum time to vote, can you clarify?

  Edgar

   From: Kevin Benton blak...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: Tuesday, August 19, 2014 at 1:11 PM

 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   I also added a two more nodes to our cluster to reduce the delay. Can
 we establish a maximum time to vote on the wiki?
 On Aug 19, 2014 9:39 AM, Edgar Magana edgar.mag...@workday.com wrote:

  Kevin,

  I just verified, Thanks a lot.

  Edgar

   From: Kevin Benton blak...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage
 questions) openstack-dev@lists.openstack.org
 Date: Friday, August 15, 2014 at 4:56 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   You didn't wait long enough for the Big Switch CI to reach your
 negative test. :-)
  Currently the CI system is backed by about 100 patches to push
 responses out ~22 hours.

  This does bring up an old discussion that was never resolved.
 What is the minimum expected response time for CI systems?


 On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana edgar.mag...@workday.com
 wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few
 plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:

 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore
 this
 report that plugin/driver will be candidate to be remove from Neutron
 tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




  --
 Kevin Benton

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Kevin Benton

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-21 Thread Dane Leblanc (leblancd)
Edgar:

I'm still seeing the comment Results are not accurate. Needs clarification...

Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Thursday, August 21, 2014 2:58 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Dane,

Wiki has been updated.

Thanks,

Edgar

On 8/21/14, 7:57 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The status on the wiki page says Results are not accurate. Needs 
clarification from Cisco.
Can you please tell me what we are missing?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage
questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit. This seems to be an 
intermittent quirk of Jenkins/Gerrit: We have 3 CIs triggered from this 
Jenkins/Gerrit server. Whenever we disable another one of our other 
Jenkins jobs (in this case, we disabled DFA for some rework), the 
review comments sometimes stop showing up on Gerrit.

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 1:33 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not 
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not 
for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com
wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and 
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results 
are fake to something like results are only valid for APIC-related 
commits? I think this more accurately represents our current results 
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few 
plugins and drivers are running properly and testing all Neutron 
commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_
P
l
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting.
So please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and 
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore 
this report that plugin/driver will be candidate to be remove from 
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the 
dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Parikshit Manur
Hi Edgar,
NetScaler CI is not listed in the report. NetScaler CI up and running 
and it is voting for patchsets which contains LBaaS changes, that's why you did 
not see it voting for the test change ref.
Also, you can add me as a contact person and alias 
networking-cloudinteg...@citrix.com as email contact point for the NetScaler 
CI. 

 Please let me know if you have any questions.

Thanks,
Parikshit Manur

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Saturday, 16 August 2014 4:06 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and 
drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So please, 
reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and failing for 
the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this 
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below and 
make sure you are running appropriate tests. If you have questions, the 
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Parikshit,

I will add your information right now.

Thanks,

Edgar

On 8/19/14, 4:11 AM, Parikshit Manur parikshit.ma...@citrix.com wrote:

Hi Edgar,
   NetScaler CI is not listed in the report. NetScaler CI up and running
and it is voting for patchsets which contains LBaaS changes, that's why
you did not see it voting for the test change ref.
   Also, you can add me as a contact person and alias
networking-cloudinteg...@citrix.com as email contact point for the
NetScaler CI.  

 Please let me know if you have any questions.

Thanks,
Parikshit Manur

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Saturday, 16 August 2014 4:06 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below and
make sure you are running appropriate tests. If you have questions, the
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Hemanth,

What is your system reference name?
Please, provide me a link to the latest tempest logs.

Thanks,

Edgar

From: Hemanth Ravi hemanthrav...@gmail.commailto:hemanthrav...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Monday, August 18, 2014 at 12:24 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org, 
Kedar K 
kedar.kulka...@oneconvergence.commailto:kedar.kulka...@oneconvergence.com, 
Deepak Gupta 
deepak.gu...@oneconvergence.commailto:deepak.gu...@oneconvergence.com
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Edgar,

Our CI is running the tests (non voting), but don't see it listed on the review 
for any patch. Is this due to missing logs? I would like to confirm this is the 
issue, will resolve this.

Thanks,
-hemanth


On Mon, Aug 18, 2014 at 8:35 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Thank you Akihiro.

I will propose a better organization for this section. Stay tune!

Edgar

On 8/17/14, 10:53 PM, Akihiro Motoki 
mot...@da.jp.nec.commailto:mot...@da.jp.nec.com wrote:


On 2014/08/18 0:12, Kyle Mestery wrote:
 On Fri, Aug 15, 2014 at 5:35 PM, Edgar Magana
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few
plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:

https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plu
gin
 _and_Drivers

 Can you link this and/or move it to this page:

 https://wiki.openstack.org/wiki/NeutronPlugins

 This is under the NeutronPolicies wiki page which I did at the start
 of Juno. This tracks all policies and procedures for Neutron, and
 there's a Plugins page (which I linked to above) where this should
 land.

I just added the link Neutron_Plugins_and_Drivers#Existing_Plugin
to NeutronPlugins wiki.

The wiki pages NeutronPlugins and Neutron_Plugins_and_Drivers
cover the similar contents. According to the history of the page,
the latter one was created by Mark at Nov 2013 (beginning of Icehouse
cycle).
It seems better to merge these two pages to avoid the confusion.

Akihiro



 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore
this
 report that plugin/driver will be candidate to be remove from Neutron
tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!

 Thanks for sending this out Edgar and doing this analysis! Can you
 please put an agenda item on Monday's meeting to discuss this? I won't
 be at the meeting as I'm on PTO (Mark is running the meeting in my
 absence), but I'd like the team to discuss this and allow all
 third-party people a chance to be there and share their feelings here.

 Thanks,
 Kyle


 On 8/14/14, 8:30 AM, Kyle Mestery 
 mest...@mestery.commailto:mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.

 Thanks,
 Kyle

 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
I just added it to the wiki page.

Thanks,

Edgar

On 8/17/14, 8:49 PM, balaj...@freescale.com balaj...@freescale.com
wrote:

Hi Edgar,

Freescale CI is not listed in the below report:

https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl

We are following all the requirements of CI setup and as well
participating in the IRC Meeting. Can you please let us know if we are
missing any other requirements of CI Setup.

Regards,
Balaji.P



 -Original Message-
 From: trinath.soman...@freescale.com
 [mailto:trinath.soman...@freescale.com]
 Sent: Sunday, August 17, 2014 12:04 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run
 
 Hi Edgar,
 
 Freescale CI is reporting the results for ML2 Mechanism driver (J-1) and
 FWaaS Plugin (to be approved for J-3).
 I'm the owner for this CI.
 
 The Wiki page for this CI is
 https://wiki.openstack.org/wiki/ThirdPartySystems/Freescale_CI.
 
 --
 Trinath Somanchi - B39208
 trinath.soman...@freescale.com | extn: 4048
 
 -Original Message-
 From: Edgar Magana [mailto:edgar.mag...@workday.com]
 Sent: Saturday, August 16, 2014 4:06 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run
 Importance: High
 
 Team,
 
 I did a quick audit on the Neutron CI. Very sad results. Only few
plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plug
 in
 _and_Drivers
 
 
 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:
 
 https://review.openstack.org/#/c/114393/
 
 https://review.openstack.org/#/c/40296/
 
 
 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.
 
 Neutron code quality and reliability is a top priority, if you ignore
 this report that plugin/driver will be candidate to be remove from
 Neutron tree.
 
 Cheers,
 
 Edgar
 
 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!
 
 
 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:
 
 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below and
 make sure you are running appropriate tests. If you have questions, the
 weekly third-party meeting [2] is a great place to ask questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results are
fake to something like results are only valid for APIC-related
commits? I think this more accurately represents our current results
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below and
make sure you are running appropriate tests. If you have questions, the
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
I just updated the wiki, I will run a verification by end of the day.

Thanks,

Edgar

On 8/16/14, 11:34 PM, trinath.soman...@freescale.com
trinath.soman...@freescale.com wrote:

Hi Edgar,

Freescale CI is reporting the results for ML2 Mechanism driver (J-1) and
FWaaS Plugin (to be approved for J-3).
I'm the owner for this CI.

The Wiki page for this CI is
https://wiki.openstack.org/wiki/ThirdPartySystems/Freescale_CI.

--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Saturday, August 16, 2014 4:06 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below and
make sure you are running appropriate tests. If you have questions, the
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Thanks a lot, Brocade CI is looking good.

Cheers,

Edgar

On 8/15/14, 6:22 PM, Karthik Natarajan natar...@brocade.com wrote:

Hi Edgar,

Brocade Vyatta CI is reporting the results and providing log links.
For Brocade Vyatta Plugin, I have updated my name as the owner.
For Brocade VDX Plugin, Shiv Haris is the owner.
Please let me know if you have any questions.

Thanks,
Karthik

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 3:59 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar

On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Great!

I will run another test later today.

Thanks,

Edgar

From: Ichihara Hirofumi 
ichihara.hirof...@gmail.commailto:ichihara.hirof...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 5:33 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Hi, Edgar

MetapluginCI didn't run for document changes.
I changed filter so that CI run for ALL changes include documents.

Thanks,
Hirofumi


2014-08-16 7:35 GMT+09:00 Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
Kevin,

I just verified, Thanks a lot.

Edgar

From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 4:56 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

You didn't wait long enough for the Big Switch CI to reach your negative test. 
:-)
 Currently the CI system is backed by about 100 patches to push responses out 
~22 hours.

This does bring up an old discussion that was never resolved.
What is the minimum expected response time for CI systems?


On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Dane Leblanc (leblancd)
From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and 
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results 
are fake to something like results are only valid for APIC-related 
commits? I think this more accurately represents our current results 
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few 
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So 
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and 
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore 
this report that plugin/driver will be candidate to be remove from 
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty 
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below 
and make sure you are running appropriate tests. If you have 
questions, the weekly third-party meeting [2] is a great place to ask 
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-19 Thread Edgar Magana
I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

From which commit is it missing?
https://review.openstack.org/#/c/114629/
https://review.openstack.org/#/c/114393/

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Tuesday, August 19, 2014 12:28 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for
usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Dane,

Are you sure about it?
I just went to this commit and I could not find the APIC tests.

Thanks,

Edgar

On 8/17/14, 8:47 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote:

Edgar:

The Cisco APIC should be reporting results for both APIC-related and
non-APIC related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results
are fake to something like results are only valid for APIC-related
commits? I think this more accurately represents our current results
(for reasons we chatted about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore
this report that plugin/driver will be candidate to be remove from
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-18 Thread Edgar Magana
Neutron Ci Folks,

I have received answers from almost all the CI contacts and I want to
thank you all.
Every case is different and I will review each one of your answer and
questions.

I do understand that every CI is different and this is why I would suggest
two things:

1) Today's Neutron IRC meeting we can discuss the current status and what
we want to achieve by the end of Juno release.
2) Have a short session during the Kilo summit to have an agreement on the
requirements for the plugins and drivers under Neutron tree.

I will answer each one of you and again I thank you all for your responses.

Thanks,

Edgar


On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-18 Thread Edgar Magana
Thank you Akihiro.

I will propose a better organization for this section. Stay tune!

Edgar

On 8/17/14, 10:53 PM, Akihiro Motoki mot...@da.jp.nec.com wrote:


On 2014/08/18 0:12, Kyle Mestery wrote:
 On Fri, Aug 15, 2014 at 5:35 PM, Edgar Magana
edgar.mag...@workday.com wrote:
 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few
plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plu
gin
 _and_Drivers

 Can you link this and/or move it to this page:

 https://wiki.openstack.org/wiki/NeutronPlugins

 This is under the NeutronPolicies wiki page which I did at the start
 of Juno. This tracks all policies and procedures for Neutron, and
 there's a Plugins page (which I linked to above) where this should
 land.

I just added the link Neutron_Plugins_and_Drivers#Existing_Plugin
to NeutronPlugins wiki.

The wiki pages NeutronPlugins and Neutron_Plugins_and_Drivers
cover the similar contents. According to the history of the page,
the latter one was created by Mark at Nov 2013 (beginning of Icehouse
cycle).
It seems better to merge these two pages to avoid the confusion.

Akihiro



 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore
this
 report that plugin/driver will be candidate to be remove from Neutron
tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!

 Thanks for sending this out Edgar and doing this analysis! Can you
 please put an agenda item on Monday's meeting to discuss this? I won't
 be at the meeting as I'm on PTO (Mark is running the meeting in my
 absence), but I'd like the team to discuss this and allow all
 third-party people a chance to be there and share their feelings here.

 Thanks,
 Kyle


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.

 Thanks,
 Kyle

 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-18 Thread Hemanth Ravi
Edgar,

Our CI is running the tests (non voting), but don't see it listed on the
review for any patch. Is this due to missing logs? I would like to confirm
this is the issue, will resolve this.

Thanks,
-hemanth


On Mon, Aug 18, 2014 at 8:35 AM, Edgar Magana edgar.mag...@workday.com
wrote:

 Thank you Akihiro.

 I will propose a better organization for this section. Stay tune!

 Edgar

 On 8/17/14, 10:53 PM, Akihiro Motoki mot...@da.jp.nec.com wrote:

 
 On 2014/08/18 0:12, Kyle Mestery wrote:
  On Fri, Aug 15, 2014 at 5:35 PM, Edgar Magana
 edgar.mag...@workday.com wrote:
  Team,
 
  I did a quick audit on the Neutron CI. Very sad results. Only few
 plugins
  and drivers are running properly and testing all Neutron commits.
  I created a report here:
 
 
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plu
 gin
  _and_Drivers
 
  Can you link this and/or move it to this page:
 
  https://wiki.openstack.org/wiki/NeutronPlugins
 
  This is under the NeutronPolicies wiki page which I did at the start
  of Juno. This tracks all policies and procedures for Neutron, and
  there's a Plugins page (which I linked to above) where this should
  land.
 
 I just added the link Neutron_Plugins_and_Drivers#Existing_Plugin
 to NeutronPlugins wiki.
 
 The wiki pages NeutronPlugins and Neutron_Plugins_and_Drivers
 cover the similar contents. According to the history of the page,
 the latter one was created by Mark at Nov 2013 (beginning of Icehouse
 cycle).
 It seems better to merge these two pages to avoid the confusion.
 
 Akihiro
 
 
 
  We will discuss the actions to take on the next Neutron IRC meeting. So
  please, reach me out to clarify what is the status of your CI.
  I had two commits to quickly verify the CI reliability:
 
  https://review.openstack.org/#/c/114393/
 
  https://review.openstack.org/#/c/40296/
 
 
  I would expect all plugins and drivers passing on the first one and
  failing for the second but I got so many surprises.
 
  Neutron code quality and reliability is a top priority, if you ignore
 this
  report that plugin/driver will be candidate to be remove from Neutron
 tree.
 
  Cheers,
 
  Edgar
 
  P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!
 
  Thanks for sending this out Edgar and doing this analysis! Can you
  please put an agenda item on Monday's meeting to discuss this? I won't
  be at the meeting as I'm on PTO (Mark is running the meeting in my
  absence), but I'd like the team to discuss this and allow all
  third-party people a chance to be there and share their feelings here.
 
  Thanks,
  Kyle
 
 
  On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:
 
  Folks, I'm not sure if all CI accounts are running sufficient tests.
  Per the requirements wiki page here [1], everyone needs to be running
  more than just Tempest API tests, which I still see most neutron
  third-party CI setups doing. I'd like to ask everyone who operates a
  third-party CI account for Neutron to please look at the link below
  and make sure you are running appropriate tests. If you have
  questions, the weekly third-party meeting [2] is a great place to ask
  questions.
 
  Thanks,
  Kyle
 
  [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
  [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-18 Thread Edgar Magana
Salvatore,

Thank you for your input. I actually took your suggestion about the area to 
cover for the CI systems and I updated the wiki page for the third party 
testing in Neutron.
https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

Cheers,

Edgar

From: Salvatore Orlando sorla...@nicira.commailto:sorla...@nicira.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 4:11 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

VMware minesweeper has filters which have been designed to cover the largest 
possible subset of submissions without add unnecessary load to our scarce 
resources for CI validation. This is probably why the analysis reveals not all 
patches are covered.

Therefore our filters exclude neutral changes such as those in README.rst 
(they're unlikely to crash neutron are they?), or changes in wsgi.py. The 
latter is because the wsgi framework is shared with all plugins, and when it 
breaks the NSX plugin it's very likely upstream tests will break too.

I think the areas which are important to cover are:
- neutron/agents/.* (or at least the agents you use)
- neutron/openstack/common/.*
- neutron/notifiers/.* (if your drivers report vif plugging events to nova)
- neutron/db/.*
- neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours}
- your plugin patch

Salvatore

PS: Openstack proposal bot submission's checks have been suspended for vmware 
minesweeper about 2 days ago. They will resume soon.


On 16 August 2014 01:02, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That should 
be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro ivarlazz...@gmail.commailto:ivarlazz...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 3:47 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to 
be merged, but I was wondering if it is a requirement that CIs vote *ALL* the 
Neutron patches. Some may have missed your call just because of their filters 
(see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-17 Thread trinath.soman...@freescale.com
Hi Edgar,

Freescale CI is reporting the results for ML2 Mechanism driver (J-1) and FWaaS 
Plugin (to be approved for J-3).
I'm the owner for this CI. 

The Wiki page for this CI is 
https://wiki.openstack.org/wiki/ThirdPartySystems/Freescale_CI.

--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Saturday, August 16, 2014 4:06 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and 
drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So please, 
reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and failing for 
the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this 
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below and 
make sure you are running appropriate tests. If you have questions, the 
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-17 Thread Kyle Mestery
On Fri, Aug 15, 2014 at 5:35 PM, Edgar Magana edgar.mag...@workday.com wrote:
 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers

Can you link this and/or move it to this page:

https://wiki.openstack.org/wiki/NeutronPlugins

This is under the NeutronPolicies wiki page which I did at the start
of Juno. This tracks all policies and procedures for Neutron, and
there's a Plugins page (which I linked to above) where this should
land.


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!

Thanks for sending this out Edgar and doing this analysis! Can you
please put an agenda item on Monday's meeting to discuss this? I won't
be at the meeting as I'm on PTO (Mark is running the meeting in my
absence), but I'd like the team to discuss this and allow all
third-party people a chance to be there and share their feelings here.

Thanks,
Kyle


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-17 Thread Dane Leblanc (leblancd)
Edgar:

The Cisco APIC should be reporting results for both APIC-related and non-APIC 
related changes now.
(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).

Will you be updating the wiki page?

-Dane

-Original Message-
From: Dane Leblanc (leblancd) 
Sent: Friday, August 15, 2014 8:18 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd)
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results are fake 
to something like results are only valid for APIC-related commits? I think 
this more accurately represents our current results (for reasons we chatted 
about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and 
drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So please, 
reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and failing for 
the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this 
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below and 
make sure you are running appropriate tests. If you have questions, the 
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-17 Thread balaj...@freescale.com
Hi Edgar,

Freescale CI is not listed in the below report:

https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl

We are following all the requirements of CI setup and as well participating in 
the IRC Meeting. Can you please let us know if we are missing any other 
requirements of CI Setup.

Regards,
Balaji.P



 -Original Message-
 From: trinath.soman...@freescale.com
 [mailto:trinath.soman...@freescale.com]
 Sent: Sunday, August 17, 2014 12:04 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run
 
 Hi Edgar,
 
 Freescale CI is reporting the results for ML2 Mechanism driver (J-1) and
 FWaaS Plugin (to be approved for J-3).
 I'm the owner for this CI.
 
 The Wiki page for this CI is
 https://wiki.openstack.org/wiki/ThirdPartySystems/Freescale_CI.
 
 --
 Trinath Somanchi - B39208
 trinath.soman...@freescale.com | extn: 4048
 
 -Original Message-
 From: Edgar Magana [mailto:edgar.mag...@workday.com]
 Sent: Saturday, August 16, 2014 4:06 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run
 Importance: High
 
 Team,
 
 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plug
 in
 _and_Drivers
 
 
 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:
 
 https://review.openstack.org/#/c/114393/
 
 https://review.openstack.org/#/c/40296/
 
 
 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.
 
 Neutron code quality and reliability is a top priority, if you ignore
 this report that plugin/driver will be candidate to be remove from
 Neutron tree.
 
 Cheers,
 
 Edgar
 
 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!
 
 
 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:
 
 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below and
 make sure you are running appropriate tests. If you have questions, the
 weekly third-party meeting [2] is a great place to ask questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-17 Thread Akihiro Motoki

On 2014/08/18 0:12, Kyle Mestery wrote:
 On Fri, Aug 15, 2014 at 5:35 PM, Edgar Magana edgar.mag...@workday.com 
 wrote:
 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers

 Can you link this and/or move it to this page:

 https://wiki.openstack.org/wiki/NeutronPlugins

 This is under the NeutronPolicies wiki page which I did at the start
 of Juno. This tracks all policies and procedures for Neutron, and
 there's a Plugins page (which I linked to above) where this should
 land.

I just added the link Neutron_Plugins_and_Drivers#Existing_Plugin
to NeutronPlugins wiki.

The wiki pages NeutronPlugins and Neutron_Plugins_and_Drivers
cover the similar contents. According to the history of the page,
the latter one was created by Mark at Nov 2013 (beginning of Icehouse cycle).
It seems better to merge these two pages to avoid the confusion.

Akihiro



 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!

 Thanks for sending this out Edgar and doing this analysis! Can you
 please put an agenda item on Monday's meeting to discuss this? I won't
 be at the meeting as I'm on PTO (Mark is running the meeting in my
 absence), but I'd like the team to discuss this and allow all
 third-party people a chance to be there and share their feelings here.

 Thanks,
 Kyle


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.

 Thanks,
 Kyle

 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Edgar Magana
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Ivar Lazzaro
Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches
to be merged, but I was wondering if it is a requirement that CIs vote
*ALL* the Neutron patches. Some may have missed your call just because of
their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana edgar.mag...@workday.com
wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Edgar Magana
Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar

On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Edgar Magana
Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That should 
be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro ivarlazz...@gmail.commailto:ivarlazz...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Friday, August 15, 2014 at 3:47 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to 
be merged, but I was wondering if it is a requirement that CIs vote *ALL* the 
Neutron patches. Some may have missed your call just because of their filters 
(see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana 
edgar.mag...@workday.commailto:edgar.mag...@workday.com wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery 
mest...@mestery.commailto:mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Salvatore Orlando
VMware minesweeper has filters which have been designed to cover the
largest possible subset of submissions without add unnecessary load to our
scarce resources for CI validation. This is probably why the analysis
reveals not all patches are covered.

Therefore our filters exclude neutral changes such as those in README.rst
(they're unlikely to crash neutron are they?), or changes in wsgi.py. The
latter is because the wsgi framework is shared with all plugins, and when
it breaks the NSX plugin it's very likely upstream tests will break too.

I think the areas which are important to cover are:
- neutron/agents/.* (or at least the agents you use)
- neutron/openstack/common/.*
- neutron/notifiers/.* (if your drivers report vif plugging events to nova)
- neutron/db/.*
- neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours}
- your plugin patch

Salvatore

PS: Openstack proposal bot submission's checks have been suspended for
vmware minesweeper about 2 days ago. They will resume soon.


On 16 August 2014 01:02, Edgar Magana edgar.mag...@workday.com wrote:

  Ivar,

  Very valid point. This is why I need clarification from those CI owner.
 I will run a new test with a basic change in the Neutron DB code. That
 should be covered by almost all CI systems.

  Edgar

   From: Ivar Lazzaro ivarlazz...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Friday, August 15, 2014 at 3:47 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   Hi Edgar,

  Nice shot, to be the inquisitor is not necessarily a bad thing :)

  I know some CIs are 'stuck' waiting for bugs to be fixed or certain
 patches to be merged, but I was wondering if it is a requirement that CIs
 vote *ALL* the Neutron patches. Some may have missed your call just because
 of their filters (see [0] section 'what changes to run against').

  Cheers,
 Ivar.

  [0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




 On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana edgar.mag...@workday.com
 wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:

 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron
 tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Ivar Lazzaro
That's great!

Also, I would suggest we decide on a 'standard' way for CI owners to
comunicate their CIs status (eg. MyCompany CI is not voting waiting for bug
#9001 to be fixed).
Using the ML for that may be confusing... What about an etherpad (linked
from [0]) or a dedicated section of [0] itself?

Ivar.

[0]
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin_and_Drivers


On Sat, Aug 16, 2014 at 1:02 AM, Edgar Magana edgar.mag...@workday.com
wrote:

  Ivar,

  Very valid point. This is why I need clarification from those CI owner.
 I will run a new test with a basic change in the Neutron DB code. That
 should be covered by almost all CI systems.

  Edgar

   From: Ivar Lazzaro ivarlazz...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Friday, August 15, 2014 at 3:47 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
 required to be run

   Hi Edgar,

  Nice shot, to be the inquisitor is not necessarily a bad thing :)

  I know some CIs are 'stuck' waiting for bugs to be fixed or certain
 patches to be merged, but I was wondering if it is a requirement that CIs
 vote *ALL* the Neutron patches. Some may have missed your call just because
 of their filters (see [0] section 'what changes to run against').

  Cheers,
 Ivar.

  [0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




 On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana edgar.mag...@workday.com
 wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:

 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron
 tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
 job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Kevin Benton
You didn't wait long enough for the Big Switch CI to reach your negative
test. :-)
 Currently the CI system is backed by about 100 patches to push responses
out ~22 hours.

This does bring up an old discussion that was never resolved.
What is the minimum expected response time for CI systems?


On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana edgar.mag...@workday.com
wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Franck Yelles
Hi Edgar,

I am running the Nuage CI.
The Nuage CI has run and posted successfully the result for the first
review.

Because of infra issue(internet can be down, etc) we are running manually
the -1
 -1 take some time away for developper and before doing a -1, I want to be
sure that it's a valid issue. (we had a talk about it during the
third-party meeting on mondays)

I have posted the failure report and you can see that it was run, uploaded
the result to our server but we didn't vote (reason mention above)
http://208.113.169.228/nuage-ci/29_114629_1/ (seetimestamp of the files)

Let me know if you have any questions.

Franck (lyxus)


Franck


On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana edgar.mag...@workday.com
wrote:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Dane Leblanc (leblancd)
Edgar:

For the Notes for the Cisco APIC, can you change the comment results are fake 
to something like results are only valid for APIC-related commits? I think 
this more accurately represents our current results (for reasons we chatted 
about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and 
drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So please, 
reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and failing for 
the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this 
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below and 
make sure you are running appropriate tests. If you have questions, the 
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Dane Leblanc (leblancd)
Also, you can add me as a contact person for the Cisco VPNaaS driver.

-Original Message-
From: Dane Leblanc (leblancd) 
Sent: Friday, August 15, 2014 8:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are required to 
be run

Edgar:

For the Notes for the Cisco APIC, can you change the comment results are fake 
to something like results are only valid for APIC-related commits? I think 
this more accurately represents our current results (for reasons we chatted 
about on another thread).

Thanks,
Dane

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and 
drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So please, 
reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and failing for 
the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this 
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below and 
make sure you are running appropriate tests. If you have questions, the 
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Ichihara Hirofumi
Hi, Edgar

MetapluginCI didn't run for document changes.
I changed filter so that CI run for ALL changes include documents.

Thanks,
Hirofumi


2014-08-16 7:35 GMT+09:00 Edgar Magana edgar.mag...@workday.com:

 Team,

 I did a quick audit on the Neutron CI. Very sad results. Only few plugins
 and drivers are running properly and testing all Neutron commits.
 I created a report here:
 https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
 _and_Drivers


 We will discuss the actions to take on the next Neutron IRC meeting. So
 please, reach me out to clarify what is the status of your CI.
 I had two commits to quickly verify the CI reliability:

 https://review.openstack.org/#/c/114393/

 https://review.openstack.org/#/c/40296/


 I would expect all plugins and drivers passing on the first one and
 failing for the second but I got so many surprises.

 Neutron code quality and reliability is a top priority, if you ignore this
 report that plugin/driver will be candidate to be remove from Neutron tree.

 Cheers,

 Edgar

 P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


 On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

 Folks, I'm not sure if all CI accounts are running sufficient tests.
 Per the requirements wiki page here [1], everyone needs to be running
 more than just Tempest API tests, which I still see most neutron
 third-party CI setups doing. I'd like to ask everyone who operates a
 third-party CI account for Neutron to please look at the link below
 and make sure you are running appropriate tests. If you have
 questions, the weekly third-party meeting [2] is a great place to ask
 questions.
 
 Thanks,
 Kyle
 
 [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
 [2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Ignacio Scopetta
Hi Edgar,
I am the reference for the Embrane CI (added my name to the page you linked)

Our Ci had some issues that were pointed out during  a third party meeting some 
weeks ago,
we solved the issues that were reported and the CI was running fine.

There was a list of issues tracked on this etherpad for various CIs
https://etherpad.openstack.org/p/neutron-thirdparty-juno

Right now it's only running against changes to the plugin until this review 
gets finalized and merged:
https://review.openstack.org/#/c/108226/

Ignacio

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Friday, August 15, 2014 3:59 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar

On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few 
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So 
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and 
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore 
this report that plugin/driver will be candidate to be remove from 
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty 
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below 
and make sure you are running appropriate tests. If you have 
questions, the weekly third-party meeting [2] is a great place to ask 
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Karthik Natarajan
Hi Edgar,

Brocade Vyatta CI is reporting the results and providing log links.
For Brocade Vyatta Plugin, I have updated my name as the owner.
For Brocade VDX Plugin, Shiv Haris is the owner.
Please let me know if you have any questions.

Thanks,
Karthik

-Original Message-
From: Edgar Magana [mailto:edgar.mag...@workday.com] 
Sent: Friday, August 15, 2014 3:59 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
Importance: High

Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar

On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few 
plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Pl
ugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So 
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and 
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore 
this report that plugin/driver will be candidate to be remove from 
Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty 
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running 
more than just Tempest API tests, which I still see most neutron 
third-party CI setups doing. I'd like to ask everyone who operates a 
third-party CI account for Neutron to please look at the link below 
and make sure you are running appropriate tests. If you have 
questions, the weekly third-party meeting [2] is a great place to ask 
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread daya kamath
hi edgar,
the IBM SDNVE system runs a subset of tempest tests currently, but is triggered 
for all neutron changes. we are working on adding more tests to our system. 
There was an issue with uploading all the logs properly, which i have fixed. 
one pending item is to upload the console log as well. also, am experiencing 
some tempest failures since this morning, and i'm investigating those presently.
please take a look at this run from today, for any additional feedback. thanks 
for your input.
Gerrit Code Review

  
          
Gerrit Code Review
Loading Gerrit Code Review ... Gerrit requires a JavaScript enabled browser.   
View on review.openstack.org Preview by Yahoo  
  
 

 

thanks!
daya kamath



 From: Edgar Magana edgar.mag...@workday.com
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org 
Sent: Saturday, August 16, 2014 4:29 AM
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
 

Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar


On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron
 commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI
 accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread daya kamath
wrong link-
please take a look at this one after log fixes have been put in -
Gerrit Code Review

  
          
Gerrit Code Review
Loading Gerrit Code Review ... Gerrit requires a JavaScript enabled browser.   
View on review.openstack.org Preview by Yahoo  
  



 From: daya kamath day...@yahoo.com
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org 
Sent: Saturday, August 16, 2014 9:46 AM
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
 


hi edgar,
the IBM SDNVE system runs a subset of tempest tests currently, but is triggered 
for all neutron changes. we are working on adding more tests to our system. 
There was an issue with uploading all the logs properly, which i have fixed. 
one pending item is to upload the console log as well. also, am experiencing 
some tempest failures since this morning, and i'm investigating those presently.
please take a look at this run from today, for any additional feedback. thanks 
for your input.
Gerrit Code Review

  
          
Gerrit Code Review
Loading Gerrit Code Review ... Gerrit requires a JavaScript enabled browser.   
View on review.openstack.org Preview by Yahoo  
  
 

 

thanks!
daya kamath



 From: Edgar Magana edgar.mag...@workday.com
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org 
Sent: Saturday, August 16, 2014 4:29 AM
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to 
be run
 

Correction

The right links are:

https://review.openstack.org/#/c/114629/

https://review.openstack.org/#/c/114393/


Not this one:
https://review.openstack.org/#/c/40296/


Edgar


On 8/15/14, 3:35 PM, Edgar Magana edgar.mag...@workday.com wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron
 commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty
job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI
 accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-14 Thread trinath.soman...@freescale.com
Hi-

I'm hitting bugs for (basic/advanced)_server_ops testing. 

https://bugs.launchpad.net/nova/+bug/1232303

Kindly help me with a fix to this.

Thanking you.




--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048

-Original Message-
From: Kyle Mestery [mailto:mest...@mestery.com] 
Sent: Thursday, August 14, 2014 9:00 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron] [third-party] What tests are required to be 
run

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running more than 
just Tempest API tests, which I still see most neutron third-party CI setups 
doing. I'd like to ask everyone who operates a third-party CI account for 
Neutron to please look at the link below and make sure you are running 
appropriate tests. If you have questions, the weekly third-party meeting [2] is 
a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev