from:"Tang Chen"

Re: [openstack-dev] [osc] Meeting time preferences for OSC team

2016-04-14 Thread Tang Chen


Hi all,

In yesterday's meeting, Dean, Richard and I have discussed the meeting 
time issue.

The following two options work for us.

E.1 Every two weeks (on even weeks) on Thursday at 1300 UTC in

O.3 Every two weeks (on odd/even weeks) on Thursday at 1900 UTC in


Sheel, are you OK with these two options ?

Thanks. :)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [osc] Meeting time preferences for OSC team

2016-04-06 Thread Tang Chen


Hi guys,

Here are my preferences:

1. even week: E.4
2. odd week: O.1

Thanks a lot for the time change proposal. :)



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStackClient] New core team members

2016-02-17 Thread Tang Chen


Hi guys,

Thank you for your help and the precious opportunity for me.

And congratulations Richard. :)

Thanks.


On 02/18/2016 12:31 AM, Dean Troyer wrote:
I would like to announce the addition of Richard Theis and Tang Chen 
to the OpenStackClient core team.  They both have been contributing 
quality reviews and code for some time now, particularly in the areas 
of SDK integration and new Network commands.


Thank you Richard and Tang for your work and welcome to the core team.

dt

--

Dean Troyer
dtro...@gmail.com <mailto:dtro...@gmail.com>


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-11-06 Thread Tang Chen



On 11/06/2015 12:45 PM, Chris Friesen wrote:

On 11/05/2015 08:33 AM, Andrew Laski wrote:

On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote:


Or more specifically, the migrate and resize API actions both call 
the resize
function in the compute api. As Ed said, they are basically the same 
behind

the scenes. (But the API difference is important.)


Can you be a little more specific on what API difference is important 
to you?
There are two differences currently between migrate and resize in the 
API:


1. There is a different policy check, but this only really protects 
the next bit.


2. Resize passes in a new flavor and migration does not.

Both actions result in an instance being scheduled to a new host.  If 
they were
consolidated into a single action with a policy check to enforce that 
users
specified a new flavor and admins could leave that off would that be 
problematic

for you?



To me, the fact that resize and cold migration share the same 
implementation is just that, an implementation detail.


From the outside they are different things...one is "take this 
instance and move it somewhere else", and the other "take this 
instance and change its resource profile".


To me, the external API would make more sense as:

1) resize

2) migrate (with option of cold or live, and with option to specify a 
destination, and with option to override the scheduler if the 
specified destination doesn't pass filters)


OK. Conceptually speaking, only one case that resize could reuse 
migration code: the current host cannot match the resize condition.

And the VM should be migrated to another host, and do the resize.

So I don't think resize should be one type of migration.

May I understand it like this:  what we are talking about here is a 
3-level conception.


user APInova service driver
migrate  live-migration   o ff compute node 
storage---shared file system
resizecold-migration  o n compute node 
storage---shared file system
rebuild   o n 
compute node storage---nonshared file system

evacuate

Indeed it is a implementation detail. If we can refactor the source code 
as above, maybe it is more clear.





And while we're talking, I don't understand why 
"allow_resize_to_same_host" defaults to False.  The comments in 
https://bugs.launchpad.net/nova/+bug/1251266 say that it's not 
intended to be used in production, but doesn't give a rationale for 
that statement.  If you're using local storage and you just want to 
add some more CPUs/RAM to the instance, wouldn't it be beneficial to 
avoid the need to copy the rootfs?


I'm sorry I don't know why it is False by default. But if we can 
refactor the source code, and split resize and migrate conceptually, I 
think we don't need this option any more.


And another question about resize, shall we think about CPU/memory 
hotplug ?  AFAIK, Linux kerenl and qemu are now supporting memory 
hotplug. CPU hotplug in qemu is still being developed. I was thinking 
resize could use these functionalities.


Thanks.



Chris

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-11-05 Thread Tang Chen



On 11/05/2015 02:36 AM, Jonathan D. Proulx wrote:

On Wed, Nov 04, 2015 at 06:17:17PM +, Murray, Paul (HP Cloud) wrote:
:> From: Jay Pipes [mailto:jaypi...@gmail.com]
:> A fair point. However, I think that a generic update VM API, which would
:> allow changes to the resources consumed by the VM along with capabiities
:> like CPU model or local disk performance (SSD) is a better way to handle this
:> than a resize-specific API.
:
:
:Sorry I am so late to this - but this stuck out for me.
:
:Resize is an operation that a cloud user would do to his VM. Usually the
:cloud user does not know what host the VM is running on so a resize does
:not appear to be a move at all.
:
:Migrate is an operation that a cloud operator does to a VM that is not normally
:available to a cloud user. A cloud operator does not change the VM because
:the operator just provides what the user asked for. He only choses where he is
:going to put it.
:
:It seems clear to me that resize and migrate are very definitely different 
things,
:even if they are implemented using the same code path internally for 
convenience.
:At the very least I believe they need to be kept separate at the API so we can 
apply
:different policy to control access to them.

As an operator I'm with Paul on this.

By all means use the same code path becasue behind the scenes it *is*


Hi Jonathan,

I'm sorry that I cannot understand why resize and migrate are the same 
thing behind.


I have some understanding of my own here, please help to review.
https://blueprints.launchpad.net/nova/+spec/migration-type-refactor

I'm not sure if I understand it right or wrong. In my understanding, 
resizing a VM doesn't need to migrate it.


Thanks.


the same thing.

BUT, at the API level we do need the distinction particularly for access
control policy. The UX 'findablility' is important too, but if that
were the only issue a bit of syntactic sugar in the UI could take care
of it.

-Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-11-03 Thread Tang Chen


Hi all,

Just FYI, the WIP patch-set is now available here:

https://review.openstack.org/241476
https://review.openstack.org/241477
https://review.openstack.org/241478
https://review.openstack.org/241479
https://review.openstack.org/241480

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 
<https://review.openstack.org/#/c/197668/>
https://review.openstack.org/#/c/197669/ 
<https://review.openstack.org/#/c/197669/>



Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Migration type refactor proposal.

2015-10-28 Thread Tang Chen


Hi all,

Please take a look at this BP.

https://blueprints.launchpad.net/nova/+spec/migration-type-refactor

I'm now working on migration state machine, and I think migration type 
also has some problems we need to fix.


This can be part of the whole migration refactoring work.

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-28 Thread Tang Chen


Hi Andrew,

On 10/29/2015 12:53 AM, Andrew Laski wrote:

On 10/27/15 at 09:02am, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent 
as possible in the naming of terminal states like "finished" vs 
"done" etc)


+1.

It seems that there's also some work to do to disambiguate what the 
different operations are and what they actually do.


Migrate/resize share a code path, and then there's a big switch to 
change behavior if the migration is a live-migrate.  I agree with the 
comment below that we should deprecate the resize terminology and 
consolidate cold-migrate and resize under one umbrella with one state 
machine.  Then live-migrate could have a separate state machine.


Rebuild/evacuate share a code path though evacuate involves a 
scheduler decision which is why it seems like a move.  It's actually a 
bit tricky to classify because if the instance is volume backed it is 
essentially just a move operation, if it's image backed it's a 
destructive rebuild.  But I think it makes sense to not consider this 
a move operation and think of it as an administrative rebuild when a 
host is down.


I'm little think of that we should define each migration type clearly 
first, and then improve migration state machine. If we don't agree on 
what type represents what operation,  the state machine won't be good.


Please also help to review this BP, although the idea may be not exactly 
the same as yours.


Thanks.






It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original 
(crap, IMHO) API from Rackspace Cloud Servers API was used for the 
resize functionality in the compute API and it's been a source of 
confusion and frustration ever since. Relying on a manual 
confirmation or revert input from the user was and continues to be a 
horrible idea.


Agreed.  In my experience with operating a public cloud I am not aware 
of anyone benefiting from the manual checkpoint in the middle of 
resize.  But we should solicit feedback from the operator community on 
this.




I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following 
characteristics:


I'm not sure that we should abstract away live vs cold migrate behind 
a single move API, but I strongly agree with consolidate cold-migrate 
and resize.




* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a 
different type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order 
to see the history of state changes or subtask operations for a 
particular request to move a VM


Huge +1 from me.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have 
a look at it.


Best,
-jay

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-28 Thread Tang Chen



On 10/29/2015 09:26 AM, Tang Chen wrote:

Hi Andrew,

On 10/29/2015 12:53 AM, Andrew Laski wrote:

On 10/27/15 at 09:02am, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent 
as possible in the naming of terminal states like "finished" vs 
"done" etc)


+1.

It seems that there's also some work to do to disambiguate what the 
different operations are and what they actually do.


Migrate/resize share a code path, and then there's a big switch to 
change behavior if the migration is a live-migrate.  I agree with the 
comment below that we should deprecate the resize terminology and 
consolidate cold-migrate and resize under one umbrella with one state 
machine.  Then live-migrate could have a separate state machine.


Rebuild/evacuate share a code path though evacuate involves a 
scheduler decision which is why it seems like a move.  It's actually 
a bit tricky to classify because if the instance is volume backed it 
is essentially just a move operation, if it's image backed it's a 
destructive rebuild.  But I think it makes sense to not consider this 
a move operation and think of it as an administrative rebuild when a 
host is down.


I'm little think of that we should define each migration type clearly 
first, and then improve migration state machine. If we don't agree on 
what type represents what operation,  the state machine won't be good.


Please also help to review this BP, although the idea may be not 
exactly the same as yours.


https://blueprints.launchpad.net/nova/+spec/migration-type-refactor

https://blueprints.launchpad.net/nova/+spec/migration-state-field-machine



Thanks.






It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original 
(crap, IMHO) API from Rackspace Cloud Servers API was used for the 
resize functionality in the compute API and it's been a source of 
confusion and frustration ever since. Relying on a manual 
confirmation or revert input from the user was and continues to be a 
horrible idea.


Agreed.  In my experience with operating a public cloud I am not 
aware of anyone benefiting from the manual checkpoint in the middle 
of resize.  But we should solicit feedback from the operator 
community on this.




I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following 
characteristics:


I'm not sure that we should abstract away live vs cold migrate behind 
a single move API, but I strongly agree with consolidate cold-migrate 
and resize.




* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a 
different type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order 
to see the history of state changes or subtask operations for a 
particular request to move a VM


Huge +1 from me.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have 
a look at it.


Best,
-jay

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__ 


OpenStack Development Mailing Li

[openstack-dev] New migration state field and better state machine proposal.

2015-10-28 Thread Tang Chen


Hi all,

Please help to review this BP:

https://blueprints.launchpad.net/nova/+spec/migration-state-field-machine


I have post a BP to improve current migration state machine [1].
Please also refer to the picture of current migration state machine [2].

The current migration state machine has mixed 3 state machines for 
different 3 types of migration all together, which makes the whole 
migration framework unclear and hard to maintain. So a better solution 
is split it into 3 new state machines, one for each migration type.


The state of each migration object is maintained by the Migration.status 
field. After some discussion, we cannot change the current definition of 
status field because it has been exposed through API. So we'd better 
keep the current "status" field, and add an new field "state" to class 
Migration, representing the migration state in the new defined state 
machine. And also redefine the states for each type of migration.


There are also some problems in current migration type. Please refer to 
[3]. I think the migration framework refactor work relates to several 
BPs, and it is a long term work.


Thanks.


[1] https://blueprints.launchpad.net/nova/+spec/migration-state-machine
[2] http://paste.openstack.org/show/476954/
[3] https://blueprints.launchpad.net/nova/+spec/migration-type-refactor


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-27 Thread Tang Chen


Hi Chris,

On 10/27/2015 12:16 PM, Chris Friesen wrote:

On 10/26/2015 06:02 PM, Jay Pipes wrote:

I believe strongly that we should deprecate the existing migrate, 
resize, an
live-migrate APIs in favor of a single consolidated, consistent 
"move" REST API

that would have the following characteristics:

* No manual or wait-input states in any FSM graph


Sounds good.

* Removal of the term "resize" from the API entirely (the target 
resource sizing
is an attribute of the move operation, not a different type of API 
operation in

and of itself)


I disagree on this one.

As an end-user, if my goal is to take an existing instance and give it 
bigger disks, my first instinct isn't going to be to look at the 
"move" operation.  I'm going to look for "scale", or "resize", or 
something like that.


And if an admin wants to migrate an instance away from its current 
host, why would they want to change its disk size in the process?


This is also where I got confused before. So please refer to my previous 
mail. I think to user, nova service, and driver, they are in 3 different 
levels, and migration type may mean different things to them.




I do think it makes sense to combine the external APIs for live and 
cold migration.  Those two are fundamentally similar, logically 
separated only by whether the instance stays running or not.


Yes, this is in nova service level.

Thanks.



And I'm perfectly fine with having the internal implementation of all 
three share a code path, I just don't think it makes sense for the 
*external* API.


* Transition to a task-based API for poll-state requests. This means 
that in

order for a caller to determine the state of a VM the caller would call
something like GET /servers//tasks/ in order to see the 
history of
state changes or subtask operations for a particular request to move 
a VM

enstack.org/cgi-bin/mailman/listinfo/openstack-dev

Sounds good.

Chris

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-27 Thread Tang Chen

Hi Jay, Timofei,

Thank you for the info.

On 10/27/2015 08:02 AM, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).

That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.

Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs
from their original source image.

Well, I just dug into the source code. I think there could be some
difference between evacuate in nova server side and client side. In nova
compute, the evacuate API does call rebuild process as you said. But in
novaclient, there is a command "nova host-evacuate-live", which will
live-migrate all running VMs, which made me believe that evacuate also
migrates VMs. Please refer to:

https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/contrib/host_evacuate_live.py#L72

I think this is also a reason why I always got confused in all these
concepts: cold-migrate, evacuate, evacuate-live, rebuild, resize.

About the migration type, I can see that Timofei has tried to split
live-migration into 3 types:

1. block_live_migrate
2. live_migrate_file_level_storage
3. live_migrate_block_stroage

I think it is in driver level, not the user level. It is based on the
type of the storage the VM is using. And I think migration type should
be a multi-level thing.

Since I'm still a little confused with all the types of migration, I'd
like to share some of my understanding and if they are correct, I think
we can improve it like this.

1. OpenStack is now supporting resize a VM to another compute node. If
we set "allow_resize_to_same-host", it also supports local resize. If we
are not using memory/CPU hotplug, resize will result in a shutdown and
reconfiguration of VM.
So, there should be 2 types of resize: live (using hotplug) and cold
(often resizing the primary disk).

2. Evacuate also has 2 types: live (equals to live-migrate) and cold
(rebuild). But evacuate itself does nothing, I mean there is no actual
process called evacuate. evacuate() is just an API calling
rebuild_instance().

This is from the user level.

So finally, the migration type would be like this:

user computedriver

live-migrate
live-evacuate live-migrate
live-resize memory/CPU hotplug

cold-migrate storage type, etc
clod-evacuate cold-migrate
cold-resize (to self or not)

rebuild rebuild
(this is not a migration)

I mean maybe we should handle different things in different levels. In
compute, if the flow is too complex, we can define some more helper
functions to make the main flow easier to understand.

I support Nikola in that I believe the different migration types
should have different state machines entirely (but be as consistent as
possible in the naming of terminal states like "finished" vs "done" etc)

OK. Agreed. And maybe also introduce state machines for task_state and
vm_state.

It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.

There isn't a designer of the migration process :( The original (crap,
IMHO) API from Rackspace Cloud Servers API was used for the resize
functionality in the compute API and it's been a source of confusion
and frustration ever since. Relying on a manual confirmation or revert
input from the user was and continues to be a horrible idea.

Agreed.

I believe strongly that we should deprecate the existing migrate,
resize, an live-migrate APIs in favor of a single consolidated,
consistent "move" REST API that would have the following characteristics:

* No manual or wait-input states in any FSM graph

Yes.

* Removal of the term "resize" from the API entirely (the target
resource sizing is an attribute of the move operation, not a different
type of API operation in and of itself)

Maybe we can define it in a different level, as I said above. Not sure.

* Transition to a task-based API for poll-state requests. This means
that in order for a caller to determine the state of a VM the caller
would call something like GET /servers//tasks/ in order to
see the history of state changes or subtask operations for a
particular request to move a VM

Yes.

Timofei Durakov (cc'd) has a blueprint for splitting the
live-migration types into separate task

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-26 Thread Tang Chen

On 10/22/2015 05:42 PM, Markus Zoeller wrote:

Tang Chen <tangc...@cn.fujitsu.com> wrote on 10/21/2015 10:22:15 AM:

From: Tang Chen <tangc...@cn.fujitsu.com>
To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev@lists.openstack.org>
Date: 10/21/2015 10:24 AM
Subject: Re: [openstack-dev] [Nova] Migration state machine proposal.

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.

I'd like to make you aware of bp "split-different-live-migration-types":
https://review.openstack.org/#/c/225910/
It intends to split those 3 types you are talking about. This could have
some overlap to your work.

Hi Markus ,

Thank you for the info. Will look at it in detail.

And BTW, the direction came to this:

1. Keep the Migration.status field as it is now. This is because these 
statuses

have been exposed through APIs. Just use Enum to ensure the values are
valid. And introduce a state machine for it. This is just a small 
improvement.

2. The real improvement is here, will be in another BP. Introduce a new
Migration.state field, and a new state machine to fix the problems 
in the

current implementation, I mean remove redundant statuses, and make
the state machine more reasonable.

Thanks.

Regards, Markus Zoeller (markus_z)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-21 Thread Tang Chen


Hi Joshua,

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Question on some things seen in the below paste.

What is with 'finished' -> 'reverted' and 'finished' -> 'confirmed'?


Let's take confirm as an example. Please refer to function confirm_resize().
Resize is one type of migration in the current implement. It needs some
confirmation, and should destroy the source VM after the migration finished.

I guess the 'finished' state means just the migration is finished. Not 
the whole

resize process. Just my guess.



Why does it jump over 'reverting' or 'confirming'? Should it?


Not sure. I think it is just the original design. The confirmation and 
destroying

of the source VM may take some time. So it needs a state to indicate the
operation is on going.



The other question is the difference between 'failed' and 'error' in 
the first diagram, any idea on why/how these are semantically 
different? The difference between 'done' and 'finished' are also in my 
mind semantically confusing.


Yes, they are confusing. That is also why I raised this question.

About 'failed' and 'error', by reading the code, I think 'error' means 
we caught
some known or defined exceptions, and we know how to handle it. 'failed' 
means

unexpected exceptions happened.

And 'finished', 'completed', 'done' are just used by different types of 
migrations.

I cannot tell the difference. So I suggest to unify them.



Overall I'm very much inclined to have three state machines (one for 
each type), vs the mix-mash of all three into one state machine (which 
causes the confusion around states in the first diagram in that paste).


That is an idea. But I would prefer to have one single state machine for 
migration,
because resize and evacuate are reusing migration. They can be in one 
state machine.


It would be very helpful if the designer of the migration process could 
share his idea.
But if it is just some code modified by many people many times, I think 
we should

remove the confusing states and give a easier, better state machine.

Thanks.



Josh

Tang Chen wrote:

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-21 Thread Tang Chen


Hi,

Please help to take a look at this problem. I was trying to raise it in 
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it 
here.

It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-19 Thread Tang Chen


Hi Nikola,

On 10/19/2015 06:52 PM, Nikola Đipanov wrote:

On 10/19/2015 11:13 AM, Tang Chen wrote:

Hi, all,

If you don't mind, how about approve the BP, and I can start this work.


This is IMHO the biggest drawback of the current spec process (as I've
written before).

There is no reason why you should doubt that this particular spec will
get approved, yet due to the combination of extremely limited review
bandwidth and very aggressive deadlines, there is a good chance your
spec will miss Mitaka release purely on process grounds.

This makes you even further dissuaded from actually putting in the
development effort.


Sorry, I didn't realize the limited review bandwidth. Actually I don't
have a deadline of this.

And OK, let's keep its status so that more people could review it,
and minimize the waste of development effort.

Thanks.



N.


Thanks.


On 10/15/2015 04:53 PM, Tang Chen wrote:

Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is
indicating the
status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the
statuses,
which is part of another abandoned BP. And this is also the work I'd
like to go
on with. Please refer to:
https://review.openstack.org/#/c/197668/
<https://review.openstack.org/#/c/197668/>
https://review.openstack.org/#/c/197669/
<https://review.openstack.org/#/c/197669/>


Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave
migration_status
field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-19 Thread Tang Chen


Hi, all,

If you don't mind, how about approve the BP, and I can start this work.

Thanks.


On 10/15/2015 04:53 PM, Tang Chen wrote:

Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 
<https://review.openstack.org/#/c/197668/>
https://review.openstack.org/#/c/197669/ 
<https://review.openstack.org/#/c/197669/>



Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-15 Thread Tang Chen


Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 
<https://review.openstack.org/#/c/197668/>
https://review.openstack.org/#/c/197669/ 
<https://review.openstack.org/#/c/197669/>



Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Host maintenance mode proposal.

2015-10-15 Thread Tang Chen


Hi all,

I tried to implement a common host maintenance mode handling
in nova compute.

BP: https://blueprints.launchpad.net/nova/+spec/host-maintenance-mode
spec: https://review.openstack.org/#/c/228689/
patches: https://review.openstack.org/#/q/topic:bp/host-maintenance-mode,n,z

But according to John (johnthetubaguy), host maintenance mode is
a functionality we are going to remove. So would anybody give me
some advice if I want to go on with this work ?

BTW, if I want to talk about this in IRC meeting, which meeting should I
attend ? Nova API meeting on Tuesday ?

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Host maintenance mode proposal.

2015-10-15 Thread Tang Chen



On 10/15/2015 05:23 PM, John Garbutt wrote:

On 15 October 2015 at 10:08, Tang Chen <tangc...@cn.fujitsu.com> wrote:

Hi all,

I tried to implement a common host maintenance mode handling
in nova compute.

BP: https://blueprints.launchpad.net/nova/+spec/host-maintenance-mode
spec: https://review.openstack.org/#/c/228689/
patches: https://review.openstack.org/#/q/topic:bp/host-maintenance-mode,n,z

But according to John (johnthetubaguy), host maintenance mode is
a functionality we are going to remove. So would anybody give me
some advice if I want to go on with this work ?

BTW, if I want to talk about this in IRC meeting, which meeting should I
attend ? Nova API meeting on Tuesday ?

Sorry, I hadn't spotted there was a spec up for review.
I will go take a look at that, and add my comments on there.


Thank you very much. :)



Certainly the XenAPI host maintenance mode was built for support of
XenServer pools.


Yes, I noticed that. And I was trying to do the same thing in libvirt 
driver.

But after digging into the code, I found that we can do this totally in
nova compute.

So the latest plan is to implement a common one in compute as the
default handling. Xen implemented its own version, that is OK. For those
drivers who don't have one, they can use the default one.

You can also refer to the v1 patch set. I think code is the best 
description.



That support is currently untested, so I want to push for that to be
deprecated, and eventually removed.


Please let me know your latest idea. And I'd like to help.

Thanks.



Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Host maintenance mode proposal.

2015-10-15 Thread Tang Chen


Hi all,

I have update the spec.

https://review.openstack.org/#/c/228689/

I think implement it on nova client side may be better.
Please help to review.

Thanks.

On 10/15/2015 05:08 PM, Tang Chen wrote:

Hi all,

I tried to implement a common host maintenance mode handling
in nova compute.

BP: https://blueprints.launchpad.net/nova/+spec/host-maintenance-mode
spec: https://review.openstack.org/#/c/228689/
patches: 
https://review.openstack.org/#/q/topic:bp/host-maintenance-mode,n,z


But according to John (johnthetubaguy), host maintenance mode is
a functionality we are going to remove. So would anybody give me
some advice if I want to go on with this work ?

BTW, if I want to talk about this in IRC meeting, which meeting should I
attend ? Nova API meeting on Tuesday ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-14 Thread Tang Chen



On 10/14/2015 04:53 PM, Nikola Đipanov wrote:

On 10/14/2015 04:29 AM, Tang Chen wrote:

On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen <tangc...@cn.fujitsu.com
<mailto:tangc...@cn.fujitsu.com>> wrote:

 Hi, all,

 Please help to review this BP.

 https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


 Currently, the migration_status field in Migration object is
 indicating the
 status of migration process. But in the current code, it is
 represented
 by pure string, like 'migrating', 'finished', and so on.

 The strings could be confusing to different developers, e.g. there
 are 3
 statuses representing the migration process is over successfully:
 'finished', 'completed' and 'done'.
 And 2 for migration in process: 'running' and 'migrating'.

 So I think we should use constants or enum for these statuses.


 Furthermore, Nikola has proposed to create a state machine for the
 statuses,
 which is part of another abandoned BP. And this is also the work
 I'd like to go
 on with. Please refer to:
 https://review.openstack.org/#/c/197668/
 https://review.openstack.org/#/c/197669/


This is IMHO a worthwhile effort on it's own. I'd like to see it use a
defined state machine in addition to being a simple enum so that
transitions are clearly defined as well.


Agreed.


 Another proposal is: introduce a new member named "state" into
 Migration.
 Use a state machine to handle this Migration.state, and leave
 migration_status
 field a descriptive human readable free-form.


This is a separate effort IMHO - we should do both if possible.


Yes, I do agree.

And I think migrate_status and migrate_state fields could share the
same state machine if we do both.




On 10/14/2015 11:14 AM, Zhenyu Zheng wrote:

I think it will be better if you can submit a spec for your proposal,
it will be easier for people to give comment.

OK, will submit one soon.

If you plan to just enumerate the possible states - that should not
require a spec. Adding automaton in the mix, and especially adding a new
'state' field probably does deserve some discussion so in that case feel
free to write up a spec.


No, I planed to introduce a state machine for migrate_status field first.
And this needs a spec, I think.

And then we will go on discussing the new state field things.

Thanks.



N.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-13 Thread Tang Chen



On 10/14/2015 11:14 AM, Zhenyu Zheng wrote:
I think it will be better if you can submit a spec for your proposal, 
it will be easier for people to give comment.


OK, will submit one soon.

Thanks.



On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen <tangc...@cn.fujitsu.com 
<mailto:tangc...@cn.fujitsu.com>> wrote:


Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is
indicating the
status of migration process. But in the current code, it is
represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there
are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the
statuses,
which is part of another abandoned BP. And this is also the work
I'd like to go
on with. Please refer to:
https://review.openstack.org/#/c/197668/
https://review.openstack.org/#/c/197669/


Another proposal is: introduce a new member named "state" into
Migration.
Use a state machine to handle this Migration.state, and leave
migration_status
field a descriptive human readable free-form.


So how do you think ?

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Migration state machine proposal.

2015-10-13 Thread Tang Chen


Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is indicating the
status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 

https://review.openstack.org/#/c/197669/ 




Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

2015-10-11 Thread Tang Chen


To all,

Ok, thanks for the advices.

Thanks. :)

On 10/09/2015 08:27 PM, Jeremy Stanley wrote:

On 2015-10-09 17:00:27 +0800 (+0800), Tang Chen wrote:
[...]

I'm not sure if it is a good idea. Please help to review the
following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

The Infra team doesn't rely on blueprints, and instead uses
http://specs.openstack.org/openstack-infra/infra-specs/ to plan
complex implementation work. I only just discovered that you can
disable blueprints on a project in Launchpad, so I've now done that
for the "openstack-ci" project.

That said, I don't expect the proposed feature would get very far.
We're committed to providing test results for all patchsets when
possible; that doesn't mean you are required to pay attention to the
results or even wait for them to be posted on your change while it's
still in a state of initial flux.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

2015-10-09 Thread Tang Chen



On 10/09/2015 05:48 PM, Jordan Pittier wrote:

Hi,
On Fri, Oct 9, 2015 at 11:00 AM, Tang Chen <tangc...@cn.fujitsu.com 
<mailto:tangc...@cn.fujitsu.com>> wrote:


Hi,

CI systems will run tests for each patch once it is submitted or
modified.
But most CI systems occupy a lot of resource, and take a long time to
run tests (1 or 2 hours for one patch).

I think, not all the patches submitted need to be tested. Even
those patches
with an approved BP and spec may be reworked for 20+ versions. So
I think
CI should support a RFC (Require For Comments) mechanism for
developers
to submit and review the code detail and rework. When the patches are
fully ready, I mean all reviewers have agreed on the
implementation detail,
then CI will test the patches. 

So have the humans do the hard work to eventually find out that the 
patch breaks the world ?


No. Developers of course will run some tests themselves before they 
submit patches.
It is just a waste of resource if reviewers are discussing about where 
this function should be,
or what the function should be named. After all these details are agreed 
on, run the CI.



For a 20+ version patch-set, maybe 3 or 4 rounds
of tests are enough. Just test the last 3 or 4 versions.

 How do know, when a new patchset arrives, that it's part of the last 
3 or 4 versions ?


I think it could work like this:
1. At first, developer submits v1 patch-set with RFC tag. CIs don't run.
2. After several versions reworked, like v5, v6, most reviewers have 
agreed on the implementation

is OK. Then submit v7 without RFC tag. Then CIs run.
3. After 3, 4 rounds of tests, v10 patch-set could be merged.

Thanks.



This can significantly reduce CI overload.

This workflow appears in many other OSS communities, such as Linux
kernel,
qemu and libvirt. Testers won't test patches with a [RFC] tag in
the commit message.
So I want to enable CI to support a similar mechanism.

I'm not sure if it is a good idea. Please help to review the
following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I am running a 3rd party for Cinder. The amount of time to setup, 
operate and watch after the CI results cost way more than the 1 or 2 
servers it take to run the jobs. So, I don"t want to be a party pooper 
here, but in my opinion I am not sure it's worth the effort.


Note: I don"t know about nova or neutron.

Jordan



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [CI] Try to introduce RFC mechanism to CI.

2015-10-09 Thread Tang Chen


Hi,

CI systems will run tests for each patch once it is submitted or modified.
But most CI systems occupy a lot of resource, and take a long time to
run tests (1 or 2 hours for one patch).

I think, not all the patches submitted need to be tested. Even those 
patches

with an approved BP and spec may be reworked for 20+ versions. So I think
CI should support a RFC (Require For Comments) mechanism for developers
to submit and review the code detail and rework. When the patches are
fully ready, I mean all reviewers have agreed on the implementation detail,
then CI will test the patches. For a 20+ version patch-set, maybe 3 or 4 
rounds

of tests are enough. Just test the last 3 or 4 versions.

This can significantly reduce CI overload.

This workflow appears in many other OSS communities, such as Linux kernel,
qemu and libvirt. Testers won't test patches with a [RFC] tag in the 
commit message.

So I want to enable CI to support a similar mechanism.

I'm not sure if it is a good idea. Please help to review the following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] Timeout waiting for ssh access

2015-08-25 Thread Tang Chen


Hi all,

Does anybody have any idea about this problem ?

Since ubuntu does not have /etc/sysconfig/network-scripts/ifcfg-*,
obviously it is a fedora like fs structure, we have tried to use centos,
but we still got the same error.

Thanks.


On 08/24/2015 09:19 PM, Xie, Xianshan wrote:


Hi, All

  I`m still strugling to setup nodepool env, and i got following error 
messages:


--

ERROR nodepool.NodeLauncher: Exception launching node id: 13 in 
provider: local_01 error:


  Traceback (most recent call last):

File /home/fujitsu/xiexs/nodepool/nodepool/nodepool.py, line 
405, in _run


  dt = self.launchNode(session)

File /home/fujitsu/xiexs/nodepool/nodepool/nodepool.py, line 
503, in launchNode


timeout=self.timeout):

File /home/fujitsu/xiexs/nodepool/nodepool/nodeutils.py, 
line 50, in ssh_connect


  for count in iterate_timeout(timeout, ssh access):

File /home/fujitsu/xiexs/nodepool/nodepool/nodeutils.py, 
line 42, in iterate_timeout


  raise Exception(Timeout waiting for %s % purpose)

  Exception: Timeout waiting for ssh access

 WARNING nodepool.NodePool: Deleting leaked instance 
d-p-c-local_01-12 (aa6f58d9-f691-4a72-98db-6add9d0edc1f) in local_01 
for node id: 12


--

And meanwhile, in the console.log which records the info for launching 
this instance,


there is also anerror as follows:

--

+ sed -i -e s/^\(DNS[0-9]*=[.0-9]\+\)/#\1/g 
/etc/sysconfig/network-scripts/ifcfg-*^M


sed: can't read /etc/sysconfig/network-scripts/ifcfg-*: No such file 
or directory^M


...

cloud-init-nonet[26.16]: waiting 120 seconds for network device

--

I have tried to figure out what`s causing this error:

1. mounted image.qcow2 and then checked the configuration for the 
network aboutthis instance:


$ cat etc/network/interfaces.d/eth0.cfg

   auto eth0

   iface eth0 inet dhcp

$ cat etc/network/interfaces

   auto lo

   iface lo inet loopback

   source /etc/network/interfaces.d/*.cfg

It seems good.

2. But indeed, the directory named 
/etc/sysconfig/network-scripts/ifcfg-* does not exist.


And i dont understand why it attempts to check this configuration file?

Because my instance is specified to ubuntu not rhel.

So,could you give me some tips to work this out?

Thanks in advance.

Xiexs



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [CI] Cannot get compute endpoint when running nodepool.

2015-08-20 Thread Tang Chen


Hi, all,

I got following error message while running nodepoold with nodepoold 
-d  $DAEMON_ARGS




2015-08-21 20:18:00,336 ERROR nodepool.NodePool: Exception cleaning up 
leaked nodes


Traceback (most recent call last):

  File /home/nodepool/nodepool/nodepool.py, line 2399, in periodicCleanup

self.cleanupLeakedInstances()

  File /home/nodepool/nodepool/nodepool.py, line 2410, in 
cleanupLeakedInstances


servers = manager.listServers()

  File /home/nodepool/nodepool/provider_manager.py, line 570, in 
listServers


self._servers = self.submitTask(ListServersTask())

  File /home/nodepool/nodepool/task_manager.py, line 119, in submitTask

return task.wait()

  File /home/nodepool/nodepool/task_manager.py, line 57, in run

self.done(self.main(client))

  File /home/nodepool/nodepool/provider_manager.py, line 136, in main

servers = client.nova_client.servers.list()

  File /usr/local/lib/python2.7/dist-packages/shade/__init__.py, line 
318, in nova_client


self.get_session_endpoint('compute')

  File /usr/local/lib/python2.7/dist-packages/shade/__init__.py, line 
811, in get_session_endpoint


Error getting %s endpoint: %s % (service_key, str(e)))

OpenStackCloudException: Error getting compute endpoint: Project ID not 
found: admin (Disable debug mode to suppress these details.) (HTTP 401) 
(Request-ID: req-fb986bff-3cad-48e1-9da9-915ac9ef5927)


---

And in my case, the output info with cli listed as follows:

$ openstack service list

| ID   | Name | Type   |

+--+--++

| 213a7ba8f0564523a3d2769f77621fde | nova | compute|

$ openstack project list

+--++

| ID   | Name   |

+--++

| 0a765fdfa79a438aae56202bdd5824e2 | admin  |

$ keystone endpoint-list

+--+---+-+-+-+--+

|id|   region |  
  publicurl| internalurl   | 
adminurl|service_id|


+--+---+-+-+-+--+

| d89b009e81f04a17a26fd07ffbf83efb | regionOne | 
http://controller:8774/v2/%(tenant_id)s 
http://controller:8774/v2/%25%28tenant_id%29s | 
http://controller:8774/v2/%(tenant_id)s 
http://controller:8774/v2/%25%28tenant_id%29s | 
http://controller:8774/v2/%(tenant_id)s 
http://controller:8774/v2/%25%28tenant_id%29s | 
213a7ba8f0564523a3d2769f77621fde |


+--+---+-+-+-+--+

Have you ever seen this error? And could you give me any advice to solve it?

Thanks in advance.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [CI] How to run dsvm_tempest_full without nodepool ?

2015-08-18 Thread Tang Chen


Hi Asselin, Abhishek,
CC: all,

After a long time, I think I failed to setup a nodepool.

I can start nodepool daemon, but when launching nodes with a ready state 
image.




ERROR nodepool.NodeLauncher: Exception launching node id: 7 in provider: 
local_01 error:


Traceback (most recent call last):

  File /usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py, 
line 409, in _run


dt = self.launchNode(session)

  File /usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py, 
line 472, in launchNode


server = self.manager.waitForServer(server_id, self.launch_timeout)

  File 
/usr/local/lib/python2.7/dist-packages/nodepool/provider_manager.py, 
line 458, in waitForServer


return self._waitForResource('server', server_id, timeout)

  File 
/usr/local/lib/python2.7/dist-packages/nodepool/provider_manager.py, 
line 429, in _waitForResource


self.provider.name)):

  File /usr/local/lib/python2.7/dist-packages/nodepool/nodeutils.py, 
line 42, in iterate_timeout


raise Exception(Timeout waiting for %s % purpose)

Exception: Timeout waiting for server 
d7150309-0a33-4338-b4e8-268c719f785b in local_01



Have you ever seen this error before ?


Seeing from the log, I think dsvm-tempest-full test needs nodepool support.

Is dsvm-tempest-full test able to be executed without nodepool ?


Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-08-06 Thread Tang Chen

Hi Asselin,

On 08/06/2015 09:44 PM, Asselin, Ramy wrote:

Hi Tang,

First, I recommend you use os-loganalyze because it significantly
increases the value of the log files by making them easier to consume.

I'm not sure what the issue you encountered is. The link you provide
is to use swift, but that is the newer alternative to the old-fashion
files on disk approach, and not a requirement.

True. And I'm not using swift right now.

That said, you'll find the rules in one of the files located here:
/etc/apache2/sites-enabled/

It is created by this template [1]. As you see, there's not htmlify
directory because it's an alias that invokes os-loganalyze.

Yes, I saw the source. Thank you very much to clarify that.
It is OK now.

But one more thing I want to confirm.

install_log_server.sh should be run on log server, right ?
Then $DOMAIN is configured to the domain name of the log server itself,
and nothing about jenkins master is configured.

So how does the jenkins master find the log server ?
In other word, how does the jenkins master know the domain name of log
server ?

Thanks.

Ramy

[1]
http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/templates/logs.vhost.erb#n85

*From:*Tang Chen [mailto:tangc...@cn.fujitsu.com]
*Sent:* Thursday, August 06, 2015 5:07 AM
*To:* openstack-dev@lists.openstack.org
*Subject:* Re: [openstack-dev] How to use the log server in CI ?

Hi Joshua,

Thanks to reply.

On 08/06/2015 07:45 PM, Joshua Hesketh wrote:

Hi Tang,

For OpenStack's set up, os-loganalyze sits at /htmlify/ and is
used to add markup and filter log lines when viewing in a browser.

On my box, I don't have a /htmlify/ directory, and I don't think I
installed os-loganalyze at all.
But when I accessed to the log site, the URL was just modified, added
/htmlify/.

For your own set up you don't need to use this and could simply
serve anything straight off your disk. It should be safe to remove
the apache matching rules in order to do so.

I'm sorry, how to remove the apache matching rules ? From where ?

Thanks. :)

Hope that helps.

Cheers,

Josh

On Thu, Aug 6, 2015 at 6:50 PM, Tang Chen tangc...@cn.fujitsu.com
mailto:tangc...@cn.fujitsu.com wrote:

Hi Abhishek,

After I setup a log server, if the request ends in .txt.gz,
console.html or console.html.gz rewrite the url to prepend
/htmlify/ .
But actually the log file is on my local machine.

Is this done by os-loganalyze ? Is this included in
install_log_server.sh ? (I don't think so.)
Could I disable it and access my log file locally ?

I found this URL for reference.

http://josh.people.rcbops.com/2014/10/openstack-infrastructure-swift-logs-and-performance/

Thanks. :)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack Development Mailing List (not for usage questions)

Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-08-06 Thread Tang Chen


Hi Joshua,

Thanks to reply.

On 08/06/2015 07:45 PM, Joshua Hesketh wrote:

Hi Tang,

For OpenStack's set up, os-loganalyze sits at /htmlify/ and is used to 
add markup and filter log lines when viewing in a browser.


On my box, I don't have a /htmlify/ directory, and I don't think I 
installed os-loganalyze at all.
But when I accessed to the log site, the URL was just modified, added 
/htmlify/.




For your own set up you don't need to use this and could simply serve 
anything straight off your disk. It should be safe to remove the 
apache matching rules in order to do so.


I'm sorry, how to remove the apache matching rules ? From where ?

Thanks. :)




Hope that helps.

Cheers,
Josh

On Thu, Aug 6, 2015 at 6:50 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Abhishek,

After I setup a log server, if the request ends in .txt.gz,
console.html or console.html.gz rewrite the url to prepend /htmlify/ .
But actually the log file is on my local machine.

Is this done by os-loganalyze ?  Is this included in
install_log_server.sh ? (I don't think so.)
Could I disable it and access my log file locally ?

I found this URL for reference.

http://josh.people.rcbops.com/2014/10/openstack-infrastructure-swift-logs-and-performance/

Thanks. :)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-08-06 Thread Tang Chen


Hi Abhishek,

After I setup a log server, if the request ends in .txt.gz, console.html 
or console.html.gz rewrite the url to prepend /htmlify/ .

But actually the log file is on my local machine.

Is this done by os-loganalyze ?  Is this included in 
install_log_server.sh ? (I don't think so.)

Could I disable it and access my log file locally ?

I found this URL for reference.
http://josh.people.rcbops.com/2014/10/openstack-infrastructure-swift-logs-and-performance/

Thanks. :)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-08-03 Thread Tang Chen

On 08/04/2015 06:44 AM, Asselin, Ramy wrote:

Hi Tang,

Sorry the documentation on github is limited. I've been focusing on
getting an upstream solution working, and that's where it belongs and
should be maintained. I will take a quick look and add some tips as
discussed in this thread.

Of course if you have pull requests, I'm happy to merge them.

I did include publishers in the sample:
https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample#L73

Greate.

Thanks a lot. :)

Ramy

*From:*Tang Chen [mailto:tangc...@cn.fujitsu.com]
*Sent:* Friday, July 31, 2015 12:04 AM
*To:* Abhishek Shrivastava abhis...@cloudbyte.com
*Cc:* OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org

*Subject:* Re: [openstack-dev] How to use the log server in CI ?

Hi Abhishek,

I found out what was wrong.

I didn't configure publisher in my jobs..
Sorry for the trouble.

But I do think the document on github is not suitable for any beginner.
So I think I will post some patches to provide more info for those who
use this tool for the very first time

when I finish the work on my hand.

And about the nodepool you asked last time, I'm working on it.
But due to the proxy in my company, I have to solve lots of trivial
problems first.

Thanks.

On 07/31/2015 09:57 AM, Tang Chen wrote:

Hi Abhishek,

On 07/30/2015 04:54 PM, Abhishek Shrivastava wrote:

Hi Tang,

After completing the logServer installation the logs will go
to your machine automatically after each build run.

I don't quite understand here. I didn't configure anything about
jenkins in install_log_server.sh except the public SSH key.
How could jenkins find and connect to the log server ? Or how
could the log server find the jenkins ?

Now, I can access to my log server index page. But when I run a
build, no new log is put into /srv/static/logs/.

Thanks.

On Thu, Jul 30, 2015 at 2:19 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Asselin, Abhishek,

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my Jenkins
Master, and configured it like this:
$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log
server.

Here are my questions:

1. Is the log server able to be on the same machine with
Jenkins Master ?
I think maybe the apache in the log server conflicts
with the jenkins server.

The answer to your question is no, as the logs generated

each time will become large in size so it is recommended to
use logServer on a separate machine having a public IP.

2. Is the log server able to upload the logs to Gerrit
automatically ?
Or it is just a server for you to view the logs ?

The logs is not uploaded to Gerrit, only success or failure is
reported to Gerrit. Also, when you click on the job in the
gerrit with either of the message, you will be redirected to
the logServer page.

I raised an issue on github. You can also discuss this on
github if you like.
(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it
seems that very few people are using os-ext-testing.

Thanks.

*Image removed by sender.*

*Thanks Regards,*

*Abhishek*

/_Cloudbyte Inc. http://www.cloudbyte.com_/

OpenStack Development Mailing List (not for usage questions)

Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-07-31 Thread Tang Chen


Hi Abhishek,

I found out what was wrong.

I didn't configure publisher in my jobs..
Sorry for the trouble.

But I do think the document on github is not suitable for any beginner.
So I think I will post some patches to provide more info for those who 
use this tool for the very first time

when I finish the work on my hand.

And about the nodepool you asked last time, I'm working on it.
But due to the proxy in my company, I  have to solve lots of trivial 
problems first.


Thanks.


On 07/31/2015 09:57 AM, Tang Chen wrote:

Hi Abhishek,


On 07/30/2015 04:54 PM, Abhishek Shrivastava wrote:

Hi Tang,

After completing the logServer installation the logs will go to your 
machine automatically after each build run.


I don't quite understand here. I didn't configure anything about 
jenkins in install_log_server.sh except the public SSH key.
How could jenkins find and connect to the log server ? Or how could 
the log server find the jenkins ?


Now, I can access to my log server index page. But when I run a build, 
no new log is put into /srv/static/logs/.


Thanks.





On Thu, Jul 30, 2015 at 2:19 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Asselin, Abhishek,

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my Jenkins
Master, and configured it like this:
$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log server.

Here are my questions:

1. Is the log server able to be on the same machine with Jenkins
Master ?
I think maybe the apache in the log server conflicts with the
jenkins server.


The answer to your question is no, as the logs generated
each time will become large in size so it is recommended to use 
logServer on a separate machine having a public IP.



2. Is the log server able to upload the logs to Gerrit
automatically ?
Or it is just a server for you to view the logs ?


The logs is not uploaded to Gerrit, only success or failure is 
reported to Gerrit. Also, when you click on the job in the gerrit 
with either of the message, you will be redirected to the logServer page.



I raised an issue on github.  You can also discuss this on github
if you like.
(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it seems
that very few people are using os-ext-testing.

Thanks.




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-07-30 Thread Tang Chen


Hi Abhishek,


On 07/30/2015 04:54 PM, Abhishek Shrivastava wrote:

Hi Tang,

After completing the logServer installation the logs will go to your 
machine automatically after each build run.


I don't quite understand here. I didn't configure anything about jenkins 
in install_log_server.sh except the public SSH key.
How could jenkins find and connect to the log server ? Or how could the 
log server find the jenkins ?


Now, I can access to my log server index page. But when I run a build, 
no new log is put into /srv/static/logs/.


Thanks.





On Thu, Jul 30, 2015 at 2:19 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Asselin, Abhishek,

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my Jenkins Master,
and configured it like this:
$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log server.

Here are my questions:

1. Is the log server able to be on the same machine with Jenkins
Master ?
I think maybe the apache in the log server conflicts with the
jenkins server.


 The answer to your question is no, as the logs generated
 each time will become large in size so it is recommended to use 
logServer on a separate machine having a public IP.



2. Is the log server able to upload the logs to Gerrit automatically ?
Or it is just a server for you to view the logs ?


 The logs is not uploaded to Gerrit, only success or failure is 
reported to Gerrit. Also, when you click on the job in the gerrit with 
either of the message, you will be redirected to the logServer page.



I raised an issue on github.  You can also discuss this on github
if you like.
(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it seems that
very few people are using os-ext-testing.

Thanks.




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] How to use the log server in CI ?

2015-07-30 Thread Tang Chen


Hi Asselin, Abhishek,**

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my Jenkins Master, and 
configured it like this:

$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log server.

Here are my questions:

1. Is the log server able to be on the same machine with Jenkins Master ?
I think maybe the apache in the log server conflicts with the 
jenkins server.


2. Is the log server able to upload the logs to Gerrit automatically ?
Or it is just a server for you to view the logs ?

I raised an issue on github.  You can also discuss this on github if you 
like.

(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it seems that very 
few people are using os-ext-testing.


Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-07-30 Thread Tang Chen



On 07/30/2015 04:54 PM, Abhishek Shrivastava wrote:

Hi Tang,

After completing the logServer installation the logs will go to your 
machine automatically after each build run.




But I cannot find anything new under /srv/static/logs/.
Shouldn't the new logs be under this directory ?




On Thu, Jul 30, 2015 at 2:19 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Asselin, Abhishek,

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my Jenkins Master,
and configured it like this:
$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log server.

Here are my questions:

1. Is the log server able to be on the same machine with Jenkins
Master ?
I think maybe the apache in the log server conflicts with the
jenkins server.


 The answer to your question is no, as the logs generated
 each time will become large in size so it is recommended to use 
logServer on a separate machine having a public IP.


What if I just want to setup it in a local environment and try to use it ?

What should I configure the $DOMAIN ?  I don't have a public hostname now.

Thanks.



2. Is the log server able to upload the logs to Gerrit automatically ?
Or it is just a server for you to view the logs ?


 The logs is not uploaded to Gerrit, only success or failure is 
reported to Gerrit. Also, when you click on the job in the gerrit with 
either of the message, you will be redirected to the logServer page.



I raised an issue on github.  You can also discuss this on github
if you like.
(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it seems that
very few people are using os-ext-testing.

Thanks.




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to use the log server in CI ?

2015-07-30 Thread Tang Chen



On 07/30/2015 05:48 PM, Abhishek Shrivastava wrote:

I am talking about /srv/static/logs folder.


Yes, I see.

But, I run the install_log_server.sh on my Jenkins Master, and 
configured $DOMAIN to another machine's hostname.


Did I do this correctly ?

After that, I can find /srv/static/logs on my Jenkins Master, and when 
I run a build, nothing new was created under that directory.

And on the other machine, there is nothing.

I'm wondering if I should run the script on the other machine ?

I'm a little confused.

Thanks.



On Thu, Jul 30, 2015 at 3:18 PM, Abhishek Shrivastava 
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:


Hi Tang,

You can give any name in $DOMAIN, also try to run a build then see
if the logs are going to the LogServer, but before that clear that
folder so that you can see the difference.

On Thu, Jul 30, 2015 at 3:15 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:


On 07/30/2015 04:54 PM, Abhishek Shrivastava wrote:

Hi Tang,

After completing the logServer installation the logs will go
to your machine automatically after each build run.



But I cannot find anything new under /srv/static/logs/.
Shouldn't the new logs be under this directory ?




On Thu, Jul 30, 2015 at 2:19 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Asselin, Abhishek,

Sorry, it is about the CI again.

I run install_log_server.sh to setup a log server.

I setup the log server on the same machine with my
Jenkins Master, and configured it like this:
$DOMAIN=localhost
$|JENKINS_SSH_PUBLIC_KEY| = path to my ssh key

The script completed. But I don't know how to use the log
server.

Here are my questions:

1. Is the log server able to be on the same machine with
Jenkins Master ?
I think maybe the apache in the log server conflicts
with the jenkins server.


 The answer to your question is no, as the logs generated
 each time will become large in size so it is recommended to
use logServer on a separate machine having a public IP.


What if I just want to setup it in a local environment and try
to use it ?

What should I configure the $DOMAIN ?  I don't have a public
hostname now.

Thanks.



2. Is the log server able to upload the logs to Gerrit
automatically ?
Or it is just a server for you to view the logs ?


 The logs is not uploaded to Gerrit, only success or failure
is reported to Gerrit. Also, when you click on the job in the
gerrit with either of the message, you will be redirected to
the logServer page.


I raised an issue on github.  You can also discuss this
on github if you like.
(https://github.com/rasselin/os-ext-testing/issues/19)

I also asked about this in IRC #openstack-infra, but it
seems that very few people are using os-ext-testing.

Thanks.




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/





-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-27 Thread Tang Chen


Hi Zaro,

On 07/24/2015 11:44 PM, Zaro wrote:
Tang, Have you tried to look at the function registrations and gearman 
queue on the gearman server as i suggested?  The instructions are in 
the wiki i referenced (in 'Starting the gearman workers' section).  
What you are seeing from the gear_client could also be from passing in 
a function name that does not exist.  Unfortunately the simple 
gear_client does not check whether the passed in functions are valid 
so what you want to do is verify that the gearman functions have been 
registered by using the 'workers' command.




Thank you for your suggestion. I have not seen the inside of Gearman.

But I set up a slave node, and the job can run on the slave node.

This is also acceptable for me. So I think I have to go on with my work 
first.

And I'll check what you said later.

Thanks very much. :)



On Fri, Jul 24, 2015 at 1:09 AM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/24/2015 03:36 PM, Tang Chen wrote:

Hi Zaro,

Thanks to reply.

On 07/24/2015 03:06 PM, Zaro wrote:

That is correctly, the Jenkins UI will not show any info about
the gearman queue.  If job is pending in Jenkins that means it's
been added to the gearman queue and has been moved to the
jenkins build queue.  The jenkins UI will only show the job once
it's in the jenkins queue.

Hmm.  What happens when you manually start a build by click on
the build icon in the jenkins UI?


If I click the build icon in the jenkins UI, the job can run.
Wired, isn't it ?

Thanks.



I'm wondering if maybe your labels are configured correctly for
your jobs and slaves.  Are there any labels on the job or the
nodes (master or slaves)? I think the jobs would be in a pending
state if there are labels on nodes and jobs but they do not
match.  I would check to see if there are any labels on the jobs
and if there are then I would remove all of the labels and try
again. There are step by step directions on how to setup the
gearman-plugin in the jenkins wiki[1] which also shows you how
to debug by viewing the gearman server queue and function
registrations.  Try to make sure the functions are registered
correctly and that the job is in the queue after you run the
gear-client command.

A couple of questions:
What version of Jenkins are you running? I think openstack runs
on 1.565.3
Also I don't think we've ever tested Jenkins and the gearman
server running on the same machine or VM.  Maybe one thing you
can try is to run them on separate VMs?


I don't have any label, and only a master, no slave.

My jenkins is 1.609.1.

The whole problem is described here:

http://paste.openstack.org/show/405051/

Please help to check and see.


BTW, I post the problem to #openstack-infra IRC channel. You can
join into that meeting if you'd like to.


Thanks.



[1] https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-24 Thread Tang Chen



On 07/24/2015 12:59 PM, Zaro wrote:
Hello Tang,  Openstack slaves only have a single executor so what you 
are probably seeing is due to using build slaves that have multiple 
executors.  There were a few bugs[1] that was fixed recently around 
these types of deadlock issues. The new gearman-plugin release[2] 
contains fixes for those issues.  Also if you want to test the 
gearman-plugin with Jenkins independently of zuul you can use the 
simple gearman-plugin-client[3] to send jobs your gearman server to 
see if the jobs get built.


[1] https://issues.jenkins-ci.org/browse/JENKINS-28891 and 
https://issues.jenkins-ci.org/browse/JENKINS-25867
[2] 
http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/gearman-plugin/0.1.2/

[3] https://github.com/zaro0508/gearman-plugin-client


Hi Zaro,

Thanks for the reply.

But I updated the gearman plugin to 0.1.2, and used the 
gearman-plugin-client to submit jobs.


# python gear_client.py -s localhost 
--function=build:noop-check-communication


Sat Jul 25 14:16:57 2015
Sending job: build:noop-check-communication to localhost with 
params={'OFFLINE_NODE_WHEN_COMPLETE': 'false', 'uuid': 
'08ad7a195237493d91eea55789e76128'}


Waiting for jobs to finish.


It doesn't work. The job submitted by the client is also pending.

BTW, I cannot see the job submitted by client in my Jenkins GUI. Is that 
correct ?


Thanks.



-Khai

On Thu, Jul 23, 2015 at 9:13 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/24/2015 12:00 PM, Tang Chen wrote:


On 07/24/2015 10:08 AM, Tang Chen wrote:


On 07/23/2015 11:44 PM, Asselin, Ramy wrote:

Are you running on 'master' nodes? I remember seeing
an issue where with a recent version of Jenkins or a
plugin where it doesn't execute jobs on the master node.
But when run on non-master jenkins slaves, it works fine.


I checked my configuration, and made sure these things:
1. I have only a master node, no slave node.
2. I have 20 idle executors on master node.
3. My master node is online.
4. My master node is set to Utilize this node as much as
possible.
5. zuul is able to be notified by Gerrit, and tell Jenkins
to start jobs.

But the jobs are always pending.


And my Gearman reports this error sometimes.

2015-07-25 10:50:44,914 ERROR gear.Server: Exception in poll loop:
Traceback (most recent call last):
  File
/usr/local/lib/python2.7/dist-packages/gear/__init__.py,
line 2614, in _doPollLoop
self._pollLoop()
  File
/usr/local/lib/python2.7/dist-packages/gear/__init__.py,
line 2626, in _pollLoop
ret = self.poll.poll()
IOError: [Errno 4] Interrupted system call

Not sure if it has anything to do with this problem.

In Jenkins GUI, Gearman connection is tested successfully on
127.0.0.1:4730 http://127.0.0.1:4730.


Seeing from zuul debug log, Gearman has successfully submitted the
jobs.

2015-07-25 11:42:09,255 DEBUG zuul.Scheduler: Adding trigger
event: TriggerEvent patchset-created openstack-dev/sandbox master
205360,1
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Done adding trigger
event: TriggerEvent patchset-created openstack-dev/sandbox master
205360,1
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Run handler awake
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Fetching trigger event
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Processing trigger
event TriggerEvent patchset-created openstack-dev/sandbox master
205360,1
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:
Event TriggerEvent patchset-created openstack-dev/sandbox master
205360,1 for change Change 0x7ff518312c10 205360,1 matched
EventFilter types: patchset-created in pipeline
IndependentPipelineManager check
2015-07-25 11:42:09,257 INFO zuul.Scheduler: Adding
openstack-dev/sandbox, Change 0x7ff518312c10 205360,1 to
Pipeline check
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:
Considering adding change Change 0x7ff518312c10 205360,1
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:
Checking for changes needed by Change 0x7ff518312c10 205360,1:
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: 
 No changes needed

2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:
Adding change Change 0x7ff518312c10 205360,1 to queue
ChangeQueue check: openstack-dev/sandbox
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager:
Event TriggerEvent patchset-created openstack-dev/sandbox master
205360,1 for change Change 0x7ff518312c10 205360,1 matched
EventFilter types: patchset-created in pipeline

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-24 Thread Tang Chen


Hi Zaro,

Thanks to reply.

On 07/24/2015 03:06 PM, Zaro wrote:
That is correctly, the Jenkins UI will not show any info about the 
gearman queue.  If job is pending in Jenkins that means it's been 
added to the gearman queue and has been moved to the jenkins build 
queue.  The jenkins UI will only show the job once it's in the jenkins 
queue.


Hmm.  What happens when you manually start a build by click on the 
build icon in the jenkins UI?  I'm wondering if maybe your labels are 
configured correctly for your jobs and slaves.  Are there any labels 
on the job or the nodes (master or slaves)? I think the jobs would be 
in a pending state if there are labels on nodes and jobs but they do 
not match.  I would check to see if there are any labels on the jobs 
and if there are then I would remove all of the labels and try again.  
There are step by step directions on how to setup the gearman-plugin 
in the jenkins wiki[1] which also shows you how to debug by viewing 
the gearman server queue and function registrations.  Try to make sure 
the functions are registered correctly and that the job is in the 
queue after you run the gear-client command.


A couple of questions:
What version of Jenkins are you running?  I think openstack runs on 
1.565.3
Also I don't think we've ever tested Jenkins and the gearman server 
running on the same machine or VM.  Maybe one thing you can try is to 
run them on separate VMs?


I don't have any label, and only a master, no slave.

My jenkins is 1.609.1.

The whole problem is described here:

http://paste.openstack.org/show/405051/

Please help to check and see.


BTW, I post the problem to #openstack-infra IRC channel. You can join 
into that meeting if you'd like to.



Thanks.



[1] https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin


On Thu, Jul 23, 2015 at 11:20 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/24/2015 12:59 PM, Zaro wrote:

Hello Tang,  Openstack slaves only have a single executor so what
you are probably seeing is due to using build slaves that have
multiple executors.  There were a few bugs[1] that was fixed
recently around these types of deadlock issues.   The new
gearman-plugin release[2] contains fixes for those issues.  Also
if you want to test the gearman-plugin with Jenkins independently
of zuul you can use the simple gearman-plugin-client[3] to send
jobs your gearman server to see if the jobs get built.

[1] https://issues.jenkins-ci.org/browse/JENKINS-28891 and
https://issues.jenkins-ci.org/browse/JENKINS-25867
[2]
http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/gearman-plugin/0.1.2/
[3] https://github.com/zaro0508/gearman-plugin-client


Hi Zaro,

Thanks for the reply.

But I updated the gearman plugin to 0.1.2, and used the
gearman-plugin-client to submit jobs.

# python gear_client.py -s localhost
--function=build:noop-check-communication

Sat Jul 25 14:16:57 2015
Sending job: build:noop-check-communication to localhost with
params={'OFFLINE_NODE_WHEN_COMPLETE': 'false', 'uuid':
'08ad7a195237493d91eea55789e76128'}

Waiting for jobs to finish.


It doesn't work. The job submitted by the client is also pending.

BTW, I cannot see the job submitted by client in my Jenkins GUI.
Is that correct ?

Thanks.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-24 Thread Tang Chen



On 07/24/2015 03:36 PM, Tang Chen wrote:

Hi Zaro,

Thanks to reply.

On 07/24/2015 03:06 PM, Zaro wrote:
That is correctly, the Jenkins UI will not show any info about the 
gearman queue.  If job is pending in Jenkins that means it's been 
added to the gearman queue and has been moved to the jenkins build 
queue.  The jenkins UI will only show the job once it's in the 
jenkins queue.


Hmm.  What happens when you manually start a build by click on the 
build icon in the jenkins UI?


If I click the build icon in the jenkins UI, the job can run. Wired, 
isn't it ?


Thanks.

I'm wondering if maybe your labels are configured correctly for your 
jobs and slaves.  Are there any labels on the job or the nodes 
(master or slaves)? I think the jobs would be in a pending state if 
there are labels on nodes and jobs but they do not match.  I would 
check to see if there are any labels on the jobs and if there are 
then I would remove all of the labels and try again.  There are step 
by step directions on how to setup the gearman-plugin in the jenkins 
wiki[1] which also shows you how to debug by viewing the gearman 
server queue and function registrations.  Try to make sure the 
functions are registered correctly and that the job is in the queue 
after you run the gear-client command.


A couple of questions:
What version of Jenkins are you running?  I think openstack runs on 
1.565.3
Also I don't think we've ever tested Jenkins and the gearman server 
running on the same machine or VM.  Maybe one thing you can try is to 
run them on separate VMs?


I don't have any label, and only a master, no slave.

My jenkins is 1.609.1.

The whole problem is described here:

http://paste.openstack.org/show/405051/

Please help to check and see.


BTW, I post the problem to #openstack-infra IRC channel. You can join 
into that meeting if you'd like to.



Thanks.



[1] https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin


On Thu, Jul 23, 2015 at 11:20 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/24/2015 12:59 PM, Zaro wrote:

Hello Tang,  Openstack slaves only have a single executor so
what you are probably seeing is due to using build slaves that
have multiple executors.  There were a few bugs[1] that was
fixed recently around these types of deadlock issues.   The new
gearman-plugin release[2] contains fixes for those issues.  Also
if you want to test the gearman-plugin with Jenkins
independently of zuul you can use the simple
gearman-plugin-client[3] to send jobs your gearman server to see
if the jobs get built.

[1] https://issues.jenkins-ci.org/browse/JENKINS-28891 and
https://issues.jenkins-ci.org/browse/JENKINS-25867
[2]
http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/gearman-plugin/0.1.2/
[3] https://github.com/zaro0508/gearman-plugin-client


Hi Zaro,

Thanks for the reply.

But I updated the gearman plugin to 0.1.2, and used the
gearman-plugin-client to submit jobs.

# python gear_client.py -s localhost
--function=build:noop-check-communication

Sat Jul 25 14:16:57 2015
Sending job: build:noop-check-communication to localhost with
params={'OFFLINE_NODE_WHEN_COMPLETE': 'false', 'uuid':
'08ad7a195237493d91eea55789e76128'}

Waiting for jobs to finish.


It doesn't work. The job submitted by the client is also pending.

BTW, I cannot see the job submitted by client in my Jenkins GUI.
Is that correct ?

Thanks.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-23 Thread Tang Chen



On 07/23/2015 11:44 PM, Asselin, Ramy wrote:

Are you running on 'master' nodes? I remember seeing an issue where with a 
recent version of Jenkins or a plugin where it doesn't execute jobs on the 
master node.
But when run on non-master jenkins slaves, it works fine.


I checked my configuration, and made sure these things:
1. I have only a master node, no slave node.
2. I have 20 idle executors on master node.
3. My master node is online.
4. My master node is set to Utilize this node as much as possible.
5. zuul is able to be notified by Gerrit, and tell Jenkins to start jobs.

But the jobs are always pending.

I'm now trying to setup a slave node and try again. :)

And anyone has any idea of this, please let me know.

Thanks.


-Original Message-
From: Tang Chen [mailto:tangc...@cn.fujitsu.com]
Sent: Thursday, July 23, 2015 12:38 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up 
a new CI system.

Hi, all

The Jenkins jobs submitted by zuul are always pending-Waiting for next available 
executor .

And Jenkins log shows the following:

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl
submitFunction
WARNING:  Worker _exec-0 exception while executing function 
hudson.plugins.gearman.StartJobWorker
java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.CancellationException
at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:136)
at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:22)
at
hudson.plugins.gearman.MyGearmanWorkerImpl.submitFunction(MyGearmanWorkerImpl.java:590)
at
hudson.plugins.gearman.MyGearmanWorkerImpl.work(MyGearmanWorkerImpl.java:374)
at
hudson.plugins.gearman.AbstractWorkerThread.run(AbstractWorkerThread.java:166)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
java.util.concurrent.CancellationException
at
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:116)
at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:125)
... 5 more
Caused by: java.util.concurrent.CancellationException
at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:77)
at
hudson.plugins.gearman.StartJobWorker.safeExecuteFunction(StartJobWorker.java:196)
at
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:114)
... 6 more

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-0 sending initial grab job Jul 24, 2015 11:09:04 AM 
hudson.plugins.gearman.MyGearmanWorkerImpl
handleSessionEvent
INFO:  Worker _exec-1 received unique job assignment Jul 24, 2015 11:09:04 
AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-1 executing function Jul 24, 2015 11:09:04 AM 
hudson.plugins.gearman.StartJobWorker
safeExecuteFunction
INFO:  Worker _exec-1 scheduling devstack-vm-test build #14 on with UUID 
83d4fd10c1ad4f1ead51beb2adf23ccd and build params
[(StringParameterValue) BASE_LOG_PATH='03/204803/1',
(StringParameterValue) ZUUL_PIPELINE='check', (StringParameterValue) 
ZUUL_UUID='83d4fd10c1ad4f1ead51beb2adf23ccd', (StringParameterValue) 
LOG_PATH='03/204803/1/check/devstack-vm-test/83d4fd1',
(StringParameterValue) ZUUL_CHANGE_IDS='204803,1',
(StringParameterValue) ZUUL_PATCHSET='1', (StringParameterValue) 
ZUUL_BRANCH='master', (StringParameterValue) 
ZUUL_REF='refs/zuul/master/Z07d022076a68448d842bd1a47dd42e19',
(StringParameterValue)
ZUUL_COMMIT='174cac545549f086e07f32edbe34b70c4155a7fc',
(StringParameterValue) ZUUL_URL='http://10.124.196.205/p/',
(StringParameterValue) ZUUL_CHANGE='204803', (StringParameterValue) 
ZUUL_CHANGES='openstack-dev/sandbox:master:refs/changes/03/204803/1',
(StringParameterValue) ZUUL_PROJECT='openstack-dev/sandbox']


It seems that Gearman doesn't work properly.

Do you guys have any idea of this ?

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-23 Thread Tang Chen



On 07/24/2015 12:00 PM, Tang Chen wrote:


On 07/24/2015 10:08 AM, Tang Chen wrote:


On 07/23/2015 11:44 PM, Asselin, Ramy wrote:
Are you running on 'master' nodes? I remember seeing an issue where 
with a recent version of Jenkins or a plugin where it doesn't 
execute jobs on the master node.

But when run on non-master jenkins slaves, it works fine.


I checked my configuration, and made sure these things:
1. I have only a master node, no slave node.
2. I have 20 idle executors on master node.
3. My master node is online.
4. My master node is set to Utilize this node as much as possible.
5. zuul is able to be notified by Gerrit, and tell Jenkins to start 
jobs.


But the jobs are always pending.


And my Gearman reports this error sometimes.

2015-07-25 10:50:44,914 ERROR gear.Server: Exception in poll loop:
Traceback (most recent call last):
  File /usr/local/lib/python2.7/dist-packages/gear/__init__.py, line 
2614, in _doPollLoop

self._pollLoop()
  File /usr/local/lib/python2.7/dist-packages/gear/__init__.py, line 
2626, in _pollLoop

ret = self.poll.poll()
IOError: [Errno 4] Interrupted system call

Not sure if it has anything to do with this problem.

In Jenkins GUI, Gearman connection is tested successfully on 
127.0.0.1:4730.


Seeing from zuul debug log, Gearman has successfully submitted the jobs.

2015-07-25 11:42:09,255 DEBUG zuul.Scheduler: Adding trigger event: 
TriggerEvent patchset-created openstack-dev/sandbox master 205360,1
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Done adding trigger event: 
TriggerEvent patchset-created openstack-dev/sandbox master 205360,1

2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Run handler awake
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Fetching trigger event
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Processing trigger event 
TriggerEvent patchset-created openstack-dev/sandbox master 205360,1
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Event 
TriggerEvent patchset-created openstack-dev/sandbox master 205360,1 
for change Change 0x7ff518312c10 205360,1 matched EventFilter types: 
patchset-created in pipeline IndependentPipelineManager check
2015-07-25 11:42:09,257 INFO zuul.Scheduler: Adding 
openstack-dev/sandbox, Change 0x7ff518312c10 205360,1 to Pipeline check
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: 
Considering adding change Change 0x7ff518312c10 205360,1
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Checking 
for changes needed by Change 0x7ff518312c10 205360,1:
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:   No 
changes needed
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Adding 
change Change 0x7ff518312c10 205360,1 to queue ChangeQueue check: 
openstack-dev/sandbox
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: Event 
TriggerEvent patchset-created openstack-dev/sandbox master 205360,1 
for change Change 0x7ff518312c10 205360,1 matched EventFilter types: 
patchset-created in pipeline IndependentPipelineManager silent
2015-07-25 11:42:09,258 INFO zuul.Scheduler: Adding 
openstack-dev/sandbox, Change 0x7ff518312c10 205360,1 to Pipeline silent
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: 
Considering adding change Change 0x7ff518312c10 205360,1
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: Unable to 
find change queue for change Change 0x7ff518312c10 205360,1 in project 
openstack-dev/sandbox
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: check
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Checking 
for changes needed by Change 0x7ff518312c10 205360,1:
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager:   No 
changes needed
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Preparing 
ref for: Change 0x7ff518312c10 205360,1
2015-07-25 11:42:09,307 INFO zuul.IndependentPipelineManager: Change 
Change 0x7ff518312c10 205360,1 depends on changes []
2015-07-25 11:42:09,307 DEBUG zuul.MergeClient: Submitting job gear.Job 
0x7ff518325490 handle: None name: merger:merge unique: 
b425daae0cec4ff3b0d4920ee5533c9c with data {'items': [{'oldrev': None, 
'newrev': None, 'refspec': u'refs/changes/60/205360/1', 'merge_mode': 2, 
'number': u'205360', 'project': 'openstack-dev/sandbox', 'url': 
'ssh://f...@review.openstack.org:29418/openstack-dev/sandbox', 'branch': 
u'master', 'patchset': u'1', 'ref': 'Z81183af13086459d9d9fb8c06af86c09'}]}
2015-07-25 11:42:09,308 DEBUG zuul.IndependentPipelineManager: Finished 
queue processor: check (changed: False)
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: silent
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Finished 
queue processor: silent (changed: False)
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: patch
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Finished

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-23 Thread Tang Chen



On 07/24/2015 10:08 AM, Tang Chen wrote:


On 07/23/2015 11:44 PM, Asselin, Ramy wrote:
Are you running on 'master' nodes? I remember seeing an issue where 
with a recent version of Jenkins or a plugin where it doesn't execute 
jobs on the master node.

But when run on non-master jenkins slaves, it works fine.


I checked my configuration, and made sure these things:
1. I have only a master node, no slave node.
2. I have 20 idle executors on master node.
3. My master node is online.
4. My master node is set to Utilize this node as much as possible.
5. zuul is able to be notified by Gerrit, and tell Jenkins to start jobs.

But the jobs are always pending.


And my Gearman reports this error sometimes.

2015-07-25 10:50:44,914 ERROR gear.Server: Exception in poll loop:
Traceback (most recent call last):
  File /usr/local/lib/python2.7/dist-packages/gear/__init__.py, line 
2614, in _doPollLoop

self._pollLoop()
  File /usr/local/lib/python2.7/dist-packages/gear/__init__.py, line 
2626, in _pollLoop

ret = self.poll.poll()
IOError: [Errno 4] Interrupted system call

Not sure if it has anything to do with this problem.

In Jenkins GUI, Gearman connection is tested successfully on 127.0.0.1:4730.

Thanks.



I'm now trying to setup a slave node and try again. :)

And anyone has any idea of this, please let me know.

Thanks.


-Original Message-
From: Tang Chen [mailto:tangc...@cn.fujitsu.com]
Sent: Thursday, July 23, 2015 12:38 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI] Jenkins jobs are not executed when 
setting up a new CI system.


Hi, all

The Jenkins jobs submitted by zuul are always pending-Waiting for 
next available executor .


And Jenkins log shows the following:

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl
submitFunction
WARNING:  Worker _exec-0 exception while executing function 
hudson.plugins.gearman.StartJobWorker

java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.CancellationException
at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:136) 


at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:22) 


at
hudson.plugins.gearman.MyGearmanWorkerImpl.submitFunction(MyGearmanWorkerImpl.java:590) 


at
hudson.plugins.gearman.MyGearmanWorkerImpl.work(MyGearmanWorkerImpl.java:374) 


at
hudson.plugins.gearman.AbstractWorkerThread.run(AbstractWorkerThread.java:166) 


at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
java.util.concurrent.CancellationException
at
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:116) 


at
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:125) 


... 5 more
Caused by: java.util.concurrent.CancellationException
at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:77)
at
hudson.plugins.gearman.StartJobWorker.safeExecuteFunction(StartJobWorker.java:196) 


at
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:114) 


... 6 more

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-0 sending initial grab job Jul 24, 2015 
11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl

handleSessionEvent
INFO:  Worker _exec-1 received unique job assignment Jul 24, 2015 
11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-1 executing function Jul 24, 2015 11:09:04 AM 
hudson.plugins.gearman.StartJobWorker

safeExecuteFunction
INFO:  Worker _exec-1 scheduling devstack-vm-test build #14 on 
with UUID 83d4fd10c1ad4f1ead51beb2adf23ccd and build params

[(StringParameterValue) BASE_LOG_PATH='03/204803/1',
(StringParameterValue) ZUUL_PIPELINE='check', (StringParameterValue) 
ZUUL_UUID='83d4fd10c1ad4f1ead51beb2adf23ccd', (StringParameterValue) 
LOG_PATH='03/204803/1/check/devstack-vm-test/83d4fd1',

(StringParameterValue) ZUUL_CHANGE_IDS='204803,1',
(StringParameterValue) ZUUL_PATCHSET='1', (StringParameterValue) 
ZUUL_BRANCH='master', (StringParameterValue) 
ZUUL_REF='refs/zuul/master/Z07d022076a68448d842bd1a47dd42e19',

(StringParameterValue)
ZUUL_COMMIT='174cac545549f086e07f32edbe34b70c4155a7fc',
(StringParameterValue) ZUUL_URL='http://10.124.196.205/p/',
(StringParameterValue) ZUUL_CHANGE='204803', (StringParameterValue) 
ZUUL_CHANGES='openstack-dev/sandbox:master:refs/changes/03/204803/1',

(StringParameterValue) ZUUL_PROJECT='openstack-dev/sandbox']


It seems that Gearman doesn't work properly.

Do you guys have any idea of this ?

Thanks.

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__ 


OpenStack

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-23 Thread Tang Chen


Hi, all

The Jenkins jobs submitted by zuul are always pending—Waiting for next 
available executor .


And Jenkins log shows the following:

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl 
submitFunction
WARNING:  Worker _exec-0 exception while executing function 
hudson.plugins.gearman.StartJobWorker
java.lang.RuntimeException: java.lang.RuntimeException: 
java.util.concurrent.CancellationException
at 
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:136)
at 
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:22)
at 
hudson.plugins.gearman.MyGearmanWorkerImpl.submitFunction(MyGearmanWorkerImpl.java:590)
at 
hudson.plugins.gearman.MyGearmanWorkerImpl.work(MyGearmanWorkerImpl.java:374)
at 
hudson.plugins.gearman.AbstractWorkerThread.run(AbstractWorkerThread.java:166)

at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: 
java.util.concurrent.CancellationException
at 
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:116)
at 
org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:125)

... 5 more
Caused by: java.util.concurrent.CancellationException
at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:77)
at 
hudson.plugins.gearman.StartJobWorker.safeExecuteFunction(StartJobWorker.java:196)
at 
hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:114)

... 6 more

Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-0 sending initial grab job
Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl 
handleSessionEvent

INFO:  Worker _exec-1 received unique job assignment
Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
INFO:  Worker _exec-1 executing function
Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.StartJobWorker 
safeExecuteFunction
INFO:  Worker _exec-1 scheduling devstack-vm-test build #14 on with 
UUID 83d4fd10c1ad4f1ead51beb2adf23ccd and build params 
[(StringParameterValue) BASE_LOG_PATH='03/204803/1', 
(StringParameterValue) ZUUL_PIPELINE='check', (StringParameterValue) 
ZUUL_UUID='83d4fd10c1ad4f1ead51beb2adf23ccd', (StringParameterValue) 
LOG_PATH='03/204803/1/check/devstack-vm-test/83d4fd1', 
(StringParameterValue) ZUUL_CHANGE_IDS='204803,1', 
(StringParameterValue) ZUUL_PATCHSET='1', (StringParameterValue) 
ZUUL_BRANCH='master', (StringParameterValue) 
ZUUL_REF='refs/zuul/master/Z07d022076a68448d842bd1a47dd42e19', 
(StringParameterValue) 
ZUUL_COMMIT='174cac545549f086e07f32edbe34b70c4155a7fc', 
(StringParameterValue) ZUUL_URL='http://10.124.196.205/p/', 
(StringParameterValue) ZUUL_CHANGE='204803', (StringParameterValue) 
ZUUL_CHANGES='openstack-dev/sandbox:master:refs/changes/03/204803/1', 
(StringParameterValue) ZUUL_PROJECT='openstack-dev/sandbox']



It seems that Gearman doesn't work properly.

Do you guys have any idea of this ?

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-22 Thread Tang Chen



On 07/22/2015 12:49 PM, Tang Chen wrote:

Hi all,

When I send a patch to gerrit, my zuul is notified, but jenkins jobs 
are not run.


My CI always reports the following error:

Merge Failed.

This change was unable to be automatically merged with the current state of the 
repository. Please rebase your change and upload a new patchset.

I think, because the patch cannot be merged, the jobs are not run.

Referring tohttps://www.mediawiki.org/wiki/Gerrit/Advanced_usage,I did update 
my master branch and make sure it is up-to-date. But it doesn't work.And other 
CIs from other companies didn't report this error.


process_event_queue()
 |-- pipeline.manager.addChange()
 |-- report Unable to find change queue for change Change 
0x7fa7ef8b6250 204446,1 in project openstack-dev/sandbox


204446 is my patch number.

Anyone knows why is that ?

Thanks.





And also, when zuul tries to get the patch from gerrit, it executes:

gerrit query --format json --all-approvals --comments --commit-message 
--current-patch-set --dependencies --files --patch-sets --submit-records 204337


When I try to execute it myself, it reports:Permission denied (publickey).

I updated my ssh key, and uploaded the new public key to gerrit, but it doesn't 
work.


Does anyone have any idea what's going on here ?

Thanks.







__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-22 Thread Tang Chen


Hi Abhishek,


On 07/22/2015 03:56 PM, Abhishek Shrivastava wrote:

Hi Tang,
Reboot your master vm and then try the same. Also after restarting 
check the status of zuul and zuul-merger.




I found the problem. zuul could not fetch repo because of the proxy..
And as a result, merge failed. So the job was not run.

Now it is OK. :)

Thanks. :)

On Wed, Jul 22, 2015 at 12:14 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/22/2015 12:49 PM, Tang Chen wrote:

Hi all,

When I send a patch to gerrit, my zuul is notified, but jenkins
jobs are not run.

My CI always reports the following error:

Merge Failed.

This change was unable to be automatically merged with the current state of 
the repository. Please rebase your change and upload a new patchset.

I think, because the patch cannot be merged, the jobs are not run.

Referring tohttps://www.mediawiki.org/wiki/Gerrit/Advanced_usage,I did 
update my master branch and make sure it is up-to-date. But it doesn't work.And 
other CIs from other companies didn't report this error.


process_event_queue()
 |-- pipeline.manager.addChange()
 |-- report Unable to find change queue for change
Change 0x7fa7ef8b6250 204446,1 in project openstack-dev/sandbox

204446 is my patch number.

Anyone knows why is that ?

Thanks.





And also, when zuul tries to get the patch from gerrit, it executes:

gerrit query --format json --all-approvals --comments --commit-message 
--current-patch-set --dependencies --files --patch-sets --submit-records 204337


When I try to execute it myself, it reports:Permission denied 
(publickey).

I updated my ssh key, and uploaded the new public key to gerrit, but it 
doesn't work.


Does anyone have any idea what's going on here ?

Thanks.







__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-21 Thread Tang Chen


hi Asselin,

On 07/21/2015 11:52 PM, Asselin, Ramy wrote:


Tang,

#openstack-meeting is for people to come together for officially 
scheduled meetings [1]


Did you try to join the 3^rd party meeting on Monday[2]? I was 
chairing the meeting but did not see you. That would be a great forum 
to ask these questions.




Sorry, I joined #openstack-meting yesterday.

Will try to join 3rd party meeting next Monday.

Otherwise, you can ask in #openstack-infra. If you do ask, remember to 
stay logged in, otherwise you'll miss any responses  people are not 
likely to respond to your question if they see you're not logged in 
(because you'll miss the response). :)




OK, Thanks. :)


Ramy

[1] https://wiki.openstack.org/wiki/Meetings

[2] http://eavesdrop.openstack.org/#Third_Party_Meeting

*From:*Tang Chen [mailto:tangc...@cn.fujitsu.com]
*Sent:* Tuesday, July 21, 2015 12:50 AM
*To:* openstack-dev@lists.openstack.org
*Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul.

On 07/21/2015 03:35 PM, Abhishek Shrivastava wrote:

Hi Tang,

Can you please send me the whole job snippet you wrote.


In /etc/jenkins_jobs/config/ examples.yaml, that's all.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

#publishers:
#  - devstack-logs  # In macros.yaml from os-ext-testing
#  - console-log  # In macros.yaml from os-ext-testing



noop-check-communication was setup by default. I didn't change 
anything else.



BTW, I tried to ask this in #openstack-meeting IRC.
But no one seems to be active. :)

Thanks.



On Tue, Jul 21, 2015 at 12:52 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Asselin, Abhishek,

I got some problems when I was trying to write a jenkins job.

I found that when zuul received the notification from gerrit,
jenkins didn't run the test.

I added something to noop-check-communication in
/etc/jenkins_jobs/config/ examples.yaml,
just touched a file under /tmp.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication # I added
something here.
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

And I flushed the jobs, using jenkins-jobs --flush-cache
update /etc/jenkins_jobs/config/.
I can build the job in jenkins web UI, and the file was touched.


But when I send a patch, the file was not touched. But CI
really works.
I can see it on the web site. (
https://review.openstack.org/#/c/203941/)

How do you think of this ?




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 


**

*Thanks  Regards,*

*Abhishek*

/_Cloudbyte Inc. http://www.cloudbyte.com_/




__

OpenStack Development Mailing List (not for usage questions)

Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-21 Thread Tang Chen



On 07/21/2015 11:42 PM, Asselin, Ramy wrote:


Hi Tang,

Please use a new thread for this new question.  I'd like to keep the 
current thread focused on How to set a proxy for zuul.




Sure, will start a new thread if I cannot get it through.

Thanks. :)



Ramy

*From:*Tang Chen [mailto:tangc...@cn.fujitsu.com]
*Sent:* Tuesday, July 21, 2015 12:23 AM
*To:* openstack-dev@lists.openstack.org
*Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul.

Hi Asselin, Abhishek,

I got some problems when I was trying to write a jenkins job.

I found that when zuul received the notification from gerrit, jenkins 
didn't run the test.


I added something to noop-check-communication in 
/etc/jenkins_jobs/config/ examples.yaml,

just touched a file under /tmp.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication # I added something here.
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

And I flushed the jobs, using jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/.

I can build the job in jenkins web UI, and the file was touched.


But when I send a patch, the file was not touched. But CI really works.
I can see it on the web site. ( https://review.openstack.org/#/c/203941/)

How do you think of this ?




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [CI] Jenkins jobs are not executed when setting up a new CI system.

2015-07-21 Thread Tang Chen


Hi all,

When I send a patch to gerrit, my zuul is notified, but jenkins jobs are 
not run.


My CI always reports the following error:

Merge Failed.

This change was unable to be automatically merged with the current state of the 
repository. Please rebase your change and upload a new patchset.

I think, because the patch cannot be merged, the jobs are not run.

Referring to https://www.mediawiki.org/wiki/Gerrit/Advanced_usage,I did update 
my master branch and make sure it is up-to-date. But it doesn't work.And other 
CIs from other companies didn't report this error.


And also, when zuul tries to get the patch from gerrit, it executes:

gerrit query --format json --all-approvals --comments --commit-message 
--current-patch-set --dependencies --files --patch-sets --submit-records 204337


When I try to execute it myself, it reports:Permission denied (publickey).

I updated my ssh key, and uploaded the new public key to gerrit, but it doesn't 
work.


Does anyone have any idea what's going on here ?

Thanks.





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-21 Thread Tang Chen



On 07/21/2015 03:35 PM, Abhishek Shrivastava wrote:

Hi Tang,

Can you please send me the whole job snippet you wrote.


In /etc/jenkins_jobs/config/ examples.yaml, that's all.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

#publishers:
#  - devstack-logs  # In macros.yaml from os-ext-testing
#  - console-log  # In macros.yaml from os-ext-testing



noop-check-communication was setup by default. I didn't change anything 
else.



BTW, I tried to ask this in #openstack-meeting IRC.
But no one seems to be active. :)

Thanks.




On Tue, Jul 21, 2015 at 12:52 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Asselin, Abhishek,

I got some problems when I was trying to write a jenkins job.

I found that when zuul received the notification from gerrit,
jenkins didn't run the test.

I added something to noop-check-communication in
/etc/jenkins_jobs/config/ examples.yaml,
just touched a file under /tmp.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication # I added something
here.
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

And I flushed the jobs, using jenkins-jobs --flush-cache update
/etc/jenkins_jobs/config/.
I can build the job in jenkins web UI, and the file was touched.


But when I send a patch, the file was not touched. But CI really
works.
I can see it on the web site. (
https://review.openstack.org/#/c/203941/)

How do you think of this ?




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-21 Thread Tang Chen


Hi Asselin, Abhishek,

I got some problems when I was trying to write a jenkins job.

I found that when zuul received the notification from gerrit, jenkins 
didn't run the test.


I added something to noop-check-communication in 
/etc/jenkins_jobs/config/ examples.yaml,

just touched a file under /tmp.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication # I added something here.
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

And I flushed the jobs, using jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/.

I can build the job in jenkins web UI, and the file was touched.


But when I send a patch, the file was not touched. But CI really works.
I can see it on the web site. ( https://review.openstack.org/#/c/203941/)

How do you think of this ?



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-21 Thread Tang Chen



On 07/21/2015 04:05 PM, Abhishek Shrivastava wrote:
If you want to create a new job then refer 
*dsvm-cinder-driver.yaml.sample*, and regarding 
*noop-check-communication* its just for testing the first time don't 
modify it.




Well, I understand. But I don't think this little change will cause a 
problem.


I think, if noop-check-communication was executed, the file should be 
created under /tmp.


I just asked the  same question in IRC meeting, but not resolved yet.

Thanks. :)

On Tue, Jul 21, 2015 at 1:19 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/21/2015 03:35 PM, Abhishek Shrivastava wrote:

Hi Tang,

Can you please send me the whole job snippet you wrote.


In /etc/jenkins_jobs/config/ examples.yaml, that's all.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

#publishers:
#  - devstack-logs  # In macros.yaml from os-ext-testing
#  - console-log  # In macros.yaml from os-ext-testing



noop-check-communication was setup by default. I didn't change
anything else.


BTW, I tried to ask this in #openstack-meeting IRC.
But no one seems to be active. :)

Thanks.




On Tue, Jul 21, 2015 at 12:52 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Asselin, Abhishek,

I got some problems when I was trying to write a jenkins job.

I found that when zuul received the notification from gerrit,
jenkins didn't run the test.

I added something to noop-check-communication in
/etc/jenkins_jobs/config/ examples.yaml,
just touched a file under /tmp.

- job-template:
name: 'noop-check-communication'
node: '{node}'

builders:
  - shell: |
  #!/bin/bash -xe
  touch /tmp/noop-check-communication # I added
something here.
  echo Hello world, this is the {vendor} Testing System
  - link-logs  # In macros.yaml from os-ext-testing

And I flushed the jobs, using jenkins-jobs --flush-cache
update /etc/jenkins_jobs/config/.
I can build the job in jenkins web UI, and the file was touched.


But when I send a patch, the file was not touched. But CI
really works.
I can see it on the web site. (
https://review.openstack.org/#/c/203941/)

How do you think of this ?





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-20 Thread Tang Chen


Hi Abhishek,

On 07/20/2015 02:09 PM, Abhishek Shrivastava wrote:

..



Quite a nice explanation +1 for all the things you have done. Just one 
more thing to ask, have you made your slave node static?




Not yet. And I haven't made my nodepool. :)
These are my next step.

But before I start to do that, I'm have to learn how to write a Jenkins 
job.

And make my CI test program work properly.

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-20 Thread Tang Chen



On 07/19/2015 02:41 AM, Asselin, Ramy wrote:


HI Abhi  Tang,

Sorry I missed this thread. Let me know if you've resolved your issues.

My repo is undergoing migrations to reuse components in 
openstack-infra/puppet-openstackci.


For single-use-nodes, the file you need has been removed here [1]: But 
I see now that it is still needed, or a different function is needed 
based on this version used by infra: [2]. I will explore a solution.


A couple other notes, please use ci-sandbox  [3] instead of sandbox.



BTW, in /etc/zuul/layout.yaml, it is ci-sandbox.

But in /etc/jenkins_jobs/config/projects.yaml, it is configured to sandbox.

Not a big deal, but for someone who uses CI for the first time, it costs 
some time to solve it.


Thanks. :)

Zuul use behind a proxy: seems you got past this? Could you share your 
solution?


Also, feel free to join 3^rd party ci IRC  meetings on freenode [4]. 
It's a great place to ask questions and meet others setting up or 
maintaining these systems.


Thanks,

Ramy

IRC: asselin

[1] 
https://github.com/rasselin/os-ext-testing/commit/dafe822be7813522a6c7361993169da20b37ffb7


[2] 
https://github.com/openstack-infra/project-config/blob/master/zuul/openstack_functions.py


[3] http://git.openstack.org/cgit/openstack-dev/ci-sandbox/

[4] http://eavesdrop.openstack.org/#Third_Party_Meeting

*From:*Abhishek Shrivastava [mailto:abhis...@cloudbyte.com]
*Sent:* Monday, July 13, 2015 11:51 PM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul.

Also if you want to change it you will need to talk with Asselin Ramy 
who is the owner of the repo you followed.


On Tue, Jul 14, 2015 at 12:18 PM, Abhishek Shrivastava 
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:


Basically it is not required, and if you see
/etc/jenkins_jobs/config folder you will find one
dsvm-cinder-tempest.yaml which is to be used basically not
examples.yaml. So its not an issue.

On Tue, Jul 14, 2015 at 12:07 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

On 07/14/2015 01:46 PM, Abhishek Shrivastava wrote:

Instead of it use reusable_node option.


Thanks. Problem resolved. :)

BTW, single_use_node is written in layout.yaml by default.
If it doesn't exist anymore, do we need a patch to fix it ?

For someone who uses CI for the first time, it is really a
problem..

And also, if I want to post patch for zuul, where should I
post the patch ?

Thanks.




On Tue, Jul 14, 2015 at 9:12 AM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com
wrote:

Hi Abhishek, All,

I found the problem.

My /etc/zuul/layout/layout.yaml has the following config:

jobs:
  - name: ^dsvm-tempest.*$
parameter-function: single_use_node

But in _parseConfig() in zuul/scheduler.py, it failed
to find single_use_node().

fname =
config_job.get('parameter-function', None)
if fname:
func = config_env.get(fname, None)
if not func:
raise Exception(Unable to find
function %s % fname)

So projects section was not parsed.

Does anyone know why ?

Thanks.




On 07/14/2015 10:54 AM, Tang Chen wrote:

Hi Abhishek,

I printed the self.layout.projects in
zuul/scheduler.py, it is empty.
So the project was not found.

But I did do the *jenkins-jobs --flush-cache
update* */etc/jenkins_jobs/config/*
And I did configure openstack-dev/sandbox in
layout.yaml.

Do you have any idea what's wrong here ?

Thanks.

On 07/13/2015 05:58 PM, Tang Chen wrote:

On 07/13/2015 04:35 PM, Abhishek Shrivastava
wrote:

Updating jobs using *sudo jenkins-jobs
--flush-cache update
/etc/jenkins_jobs/config/. *Also update
the myvendor in examples.yaml


Sorry, I updated the jobs, restart the whole
machine. But it still doesn't work.

By the way, there is no vendor in examples.yaml.

It is still this error: Project
openstack-dev/sandbox not found

Anything else should I pay attention

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-19 Thread Tang Chen


Hi Asselin, Abhishek,

Thanks for the reply. :)


On 07/19/2015 02:41 AM, Asselin, Ramy wrote:


HI Abhi  Tang,

Sorry I missed this thread. Let me know if you've resolved your issues.

My repo is undergoing migrations to reuse components in 
openstack-infra/puppet-openstackci.


For single-use-nodes, the file you need has been removed here [1]: But 
I see now that it is still needed, or a different function is needed 
based on this version used by infra: [2]. I will explore a solution.


A couple other notes, please use ci-sandbox  [3] instead of sandbox.



OK.

Zuul use behind a proxy: seems you got past this? Could you share your 
solution?




The root cause is that zuul uses a library named paramiko to create 
connection

with low level socket APIs. And it doesn't provide proxy functionary.

I tried to use tools like proxychain, to redirect the zuul connection to 
go through
my proxy. But it doesn't work. If I use proxychain to run zuul service, 
it doesn't

output anything to the log file, and the service will die soon.

I think, there are two solutions:
1. Add proxy functionality to paramiko.
2. Add proxy functionality to zuul. That means, maybe,
zuul does not use paramiko to create connection, but by itself.

Solution 1 is much simpler, so for now, I just modified the source code 
of paramiko.


I'm using python-socksipy package, and modified
/usr/local/lib/python2.7/dist-packages/paramiko/client.py like this:

diff --git a/client.py b/client.py
index 15ff696..d7225ed 100644
--- a/client.py
+++ b/client.py
@@ -24,6 +24,7 @@ from binascii import hexlify
 import getpass
 import os
 import socket
+import socks
 import warnings
 import pdb

@@ -235,6 +236,7 @@ class SSHClient (ClosingContextManager):
 ``gss_deleg_creds`` and ``gss_host`` arguments.
 

+   
 if not sock:
 for (family, socktype, proto, canonname, sockaddr) in 
socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM):

 if socktype == socket.SOCK_STREAM:
@@ -251,6 +253,13 @@ class SSHClient (ClosingContextManager):
 except:
 pass
 retry_on_signal(lambda: sock.connect(addr))
+   
+
+   if not sock:
+   sock = socks.socksocket()
+   sock.setproxy(socks.PROXY_TYPE_SOCKS5, MY_PROXY_IP, 
MY_PROXY_PORT, username=XXX, password=XXX)


  # This is review.openstack.org
+   addr = ('104.130.159.134', 29418)
+   retry_on_signal(lambda: sock.connect(addr))

 t = self._transport = Transport(sock, gss_kex=gss_kex, 
gss_deleg_creds=gss_deleg_creds)

 t.use_compression(compress=compress)


Of course, this is just a draft. It is only for my case, not for all.


BTW, I'm now working on Fujitsu CI System, and really want to join into 
the development of openstack-infra.
I think the proxy functionality is necessary for many companies. So if 
you are planing to add the proxy support,

I think I can help.

Thanks. :)

Also, feel free to join 3^rd party ci IRC  meetings on freenode [4]. 
It's a great place to ask questions and meet others setting up or 
maintaining these systems.


Thanks,

Ramy

IRC: asselin

[1] 
https://github.com/rasselin/os-ext-testing/commit/dafe822be7813522a6c7361993169da20b37ffb7


[2] 
https://github.com/openstack-infra/project-config/blob/master/zuul/openstack_functions.py


[3] http://git.openstack.org/cgit/openstack-dev/ci-sandbox/

[4] http://eavesdrop.openstack.org/#Third_Party_Meeting

*From:*Abhishek Shrivastava [mailto:abhis...@cloudbyte.com]
*Sent:* Monday, July 13, 2015 11:51 PM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul.

Also if you want to change it you will need to talk with Asselin Ramy 
who is the owner of the repo you followed.


On Tue, Jul 14, 2015 at 12:18 PM, Abhishek Shrivastava 
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:


Basically it is not required, and if you see
/etc/jenkins_jobs/config folder you will find one
dsvm-cinder-tempest.yaml which is to be used basically not
examples.yaml. So its not an issue.

On Tue, Jul 14, 2015 at 12:07 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

On 07/14/2015 01:46 PM, Abhishek Shrivastava wrote:

Instead of it use reusable_node option.


Thanks. Problem resolved. :)

BTW, single_use_node is written in layout.yaml by default.
If it doesn't exist anymore, do we need a patch to fix it ?

For someone who uses CI for the first time, it is really a
problem..

And also, if I want to post patch for zuul, where should I
post the patch ?

Thanks.




On Tue, Jul 14, 2015 at 9:12 AM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com
wrote:

Hi Abhishek, All

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-14 Thread Tang Chen



On 07/14/2015 01:46 PM, Abhishek Shrivastava wrote:

Instead of it use reusable_node option.


Thanks. Problem resolved. :)

BTW, single_use_node is written in layout.yaml by default.
If it doesn't exist anymore, do we need a patch to fix it ?

For someone who uses CI for the first time, it is really a problem..

And also, if I want to post patch for zuul, where should I post the patch ?

Thanks.



On Tue, Jul 14, 2015 at 9:12 AM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Abhishek, All,

I found the problem.

My /etc/zuul/layout/layout.yaml has the following config:

jobs:
  - name: ^dsvm-tempest.*$
parameter-function: single_use_node

But in _parseConfig() in zuul/scheduler.py, it failed to find
single_use_node().

fname = config_job.get('parameter-function', None)
if fname:
func = config_env.get(fname, None)
if not func:
raise Exception(Unable to find function %s %
fname)

So projects section was not parsed.

Does anyone know why ?

Thanks.




On 07/14/2015 10:54 AM, Tang Chen wrote:

Hi Abhishek,

I printed the self.layout.projects in zuul/scheduler.py, it is empty.
So the project was not found.

But I did do the jenkins-jobs --flush-cache update
/etc/jenkins_jobs/config/
And I did configure openstack-dev/sandbox in layout.yaml.

Do you have any idea what's wrong here ?

Thanks.


On 07/13/2015 05:58 PM, Tang Chen wrote:


On 07/13/2015 04:35 PM, Abhishek Shrivastava wrote:

Updating jobs using sudo jenkins-jobs --flush-cache update
/etc/jenkins_jobs/config/. Also update the myvendor in
examples.yaml



Sorry, I updated the jobs, restart the whole machine. But it
still doesn't work.

By the way, there is no vendor in examples.yaml.

It is still this error: Project openstack-dev/sandbox not found

Anything else should I pay attention to?

Thanks.



On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:


On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote:

Use tester or something, also are you updating the jobs or
not?


I used tester as my vendor. It doesn't work.

And what do you mean by updating the jobs ? I built the
noop-check-cimmunitication
job once in Jenkins UI. Does it matter ? All the others are
not touched.

And referring to the error, Project openstack-dev/sandbox
not found, it seems like
somewhere the project name was wrong.

right ?


Thanks.



On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com
wrote:

Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through
Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek
Shrivastava abhis...@cloudbyte.com
mailto:abhis...@cloudbyte.com wrote:

First of all, change the vendor to your vendor
name in /etc/jenkins_jobs/config/projects.yaml
file. Also, restart the zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins
UI, I tested the connection, and it succeeded.

And also, I restarted zuul and zuul merger every time
I modified the yaml files.

But it doesn't work.

And the vendor, does that matter ?  And what vendor
name should I provide ?
I cannot find any vendor info in my Gerrit service
account profile.
For example, is XXX OK ?

Thanks.





On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com
mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I
posted a patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler:
Run handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler:
Fetching trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Processing trigger event TriggerEvent
comment-added openstack-dev/sandbox master
201002,1 Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Project openstack-dev/sandbox not found

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen



On 07/13/2015 04:35 PM, Abhishek Shrivastava wrote:
Updating jobs using sudo jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/. Also update the myvendor in examples.yaml




Sorry, I updated the jobs, restart the whole machine. But it still 
doesn't work.


By the way, there is no vendor in examples.yaml.

It is still this error: Project openstack-dev/sandbox not found

Anything else should I pay attention to?

Thanks.



On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote:

Use tester or something, also are you updating the jobs or not?


I used tester as my vendor. It doesn't work.

And what do you mean by updating the jobs ? I built the
noop-check-cimmunitication
job once in Jenkins UI. Does it matter ? All the others are not
touched.

And referring to the error, Project openstack-dev/sandbox not
found, it seems like
somewhere the project name was wrong.

right ?


Thanks.



On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:

First of all, change the vendor to your vendor name in
/etc/jenkins_jobs/config/projects.yaml file. Also,
restart the zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins UI, I
tested the connection, and it succeeded.

And also, I restarted zuul and zuul merger every time I
modified the yaml files.

But it doesn't work.

And the vendor, does that matter ?  And what vendor name
should I provide ?
I cannot find any vendor info in my Gerrit service account
profile.
For example, is XXX OK ?

Thanks.





On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com
mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a
patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run
handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler:
Fetching trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Processing trigger event TriggerEvent comment-added
openstack-dev/sandbox master 201002,1 Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Project openstack-dev/sandbox not found
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run
handler sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks like
this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave ||
devstack-precise-check || d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here ?


Thanks.






__
OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/



__
OpenStack Development Mailing List (not for usage questions

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen


Hi Abhishek,

I printed the self.layout.projects in zuul/scheduler.py, it is empty.
So the project was not found.

But I did do the jenkins-jobs --flush-cache update /etc/jenkins_jobs/config/
And I did configure openstack-dev/sandbox in layout.yaml.

Do you have any idea what's wrong here ?

Thanks.


On 07/13/2015 05:58 PM, Tang Chen wrote:


On 07/13/2015 04:35 PM, Abhishek Shrivastava wrote:
Updating jobs using sudo jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/. Also update the myvendor in examples.yaml




Sorry, I updated the jobs, restart the whole machine. But it still 
doesn't work.


By the way, there is no vendor in examples.yaml.

It is still this error: Project openstack-dev/sandbox not found

Anything else should I pay attention to?

Thanks.



On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote:

Use tester or something, also are you updating the jobs or not?


I used tester as my vendor. It doesn't work.

And what do you mean by updating the jobs ? I built the
noop-check-cimmunitication
job once in Jenkins UI. Does it matter ? All the others are not
touched.

And referring to the error, Project openstack-dev/sandbox not
found, it seems like
somewhere the project name was wrong.

right ?


Thanks.



On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through
Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:

First of all, change the vendor to your vendor name
in /etc/jenkins_jobs/config/projects.yaml file. Also,
restart the zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins UI, I
tested the connection, and it succeeded.

And also, I restarted zuul and zuul merger every time I
modified the yaml files.

But it doesn't work.

And the vendor, does that matter ?  And what vendor name
should I provide ?
I cannot find any vendor info in my Gerrit service account
profile.
For example, is XXX OK ?

Thanks.





On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com
mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a
patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run
handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler:
Fetching trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Processing trigger event TriggerEvent
comment-added openstack-dev/sandbox master 201002,1
Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Project openstack-dev/sandbox not found
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run
handler sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks
like this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave ||
devstack-precise-check || d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here ?


Thanks.






__
OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*

*
*Thanks  Regards,
*
*Abhishek

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen


Hi Abhishek, All,

I found the problem.

My /etc/zuul/layout/layout.yaml has the following config:

jobs:
  - name: ^dsvm-tempest.*$
parameter-function: single_use_node

But in _parseConfig() in zuul/scheduler.py, it failed to find 
single_use_node().


fname = config_job.get('parameter-function', None)
if fname:
func = config_env.get(fname, None)
if not func:
raise Exception(Unable to find function %s % fname)

So projects section was not parsed.

Does anyone know why ?

Thanks.



On 07/14/2015 10:54 AM, Tang Chen wrote:

Hi Abhishek,

I printed the self.layout.projects in zuul/scheduler.py, it is empty.
So the project was not found.

But I did do the jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/

And I did configure openstack-dev/sandbox in layout.yaml.

Do you have any idea what's wrong here ?

Thanks.


On 07/13/2015 05:58 PM, Tang Chen wrote:


On 07/13/2015 04:35 PM, Abhishek Shrivastava wrote:
Updating jobs using sudo jenkins-jobs --flush-cache update 
/etc/jenkins_jobs/config/. Also update the myvendor in examples.yaml




Sorry, I updated the jobs, restart the whole machine. But it still 
doesn't work.


By the way, there is no vendor in examples.yaml.

It is still this error: Project openstack-dev/sandbox not found

Anything else should I pay attention to?

Thanks.



On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:



On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote:

Use tester or something, also are you updating the jobs or not?


I used tester as my vendor. It doesn't work.

And what do you mean by updating the jobs ? I built the
noop-check-cimmunitication
job once in Jenkins UI. Does it matter ? All the others are not
touched.

And referring to the error, Project openstack-dev/sandbox not
found, it seems like
somewhere the project name was wrong.

right ?


Thanks.



On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through
Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com
wrote:

First of all, change the vendor to your vendor name
in /etc/jenkins_jobs/config/projects.yaml file. Also,
restart the zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins UI,
I tested the connection, and it succeeded.

And also, I restarted zuul and zuul merger every time I
modified the yaml files.

But it doesn't work.

And the vendor, does that matter ?  And what vendor name
should I provide ?
I cannot find any vendor info in my Gerrit service account
profile.
For example, is XXX OK ?

Thanks.





On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com
mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a
patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run
handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler:
Fetching trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Processing trigger event TriggerEvent
comment-added openstack-dev/sandbox master
201002,1 Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler:
Project openstack-dev/sandbox not found
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run
handler sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks
like this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave || devstack-precise-check ||
d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen


Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Fetching trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Processing trigger event 
TriggerEvent comment-added openstack-dev/sandbox master 201002,1 
Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Project 
openstack-dev/sandbox not found

2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run handler sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks like this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave || devstack-precise-check || d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here ?

Thanks.





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen


Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava 
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:


First of all, change the vendor to your vendor name in
/etc/jenkins_jobs/config/projects.yaml file. Also, restart the
zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins UI, I tested the 
connection, and it succeeded.


And also, I restarted zuul and zuul merger every time I modified the 
yaml files.


But it doesn't work.

And the vendor, does that matter ?  And what vendor name should I provide ?
I cannot find any vendor info in my Gerrit service account profile.
For example, is XXX OK ?

Thanks.




On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a patch to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run handler awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Fetching trigger
event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Processing
trigger event TriggerEvent comment-added
openstack-dev/sandbox master 201002,1 Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Project
openstack-dev/sandbox not found
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run handler sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks like this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave || devstack-precise-check ||
d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here ?


Thanks.






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] How to set a proxy for zuul.

2015-07-13 Thread Tang Chen



On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote:

Use tester or something, also are you updating the jobs or not?


I used tester as my vendor. It doesn't work.

And what do you mean by updating the jobs ? I built the 
noop-check-cimmunitication

job once in Jenkins UI. Does it matter ? All the others are not touched.

And referring to the error, Project openstack-dev/sandbox not found, 
it seems like

somewhere the project name was wrong.

right ?

Thanks.



On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen tangc...@cn.fujitsu.com 
mailto:tangc...@cn.fujitsu.com wrote:


Hi Abhishek,

Thanks for the quick reply.

On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote:

Also check that Gearman is connecting or not through Jenkins UI.

On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava
abhis...@cloudbyte.com mailto:abhis...@cloudbyte.com wrote:

First of all, change the vendor to your vendor name in
/etc/jenkins_jobs/config/projects.yaml file. Also, restart
the zuul and zuul merger.



I did the check. Gearman plugin works find. In Jenkins UI, I
tested the connection, and it succeeded.

And also, I restarted zuul and zuul merger every time I modified
the yaml files.

But it doesn't work.

And the vendor, does that matter ?  And what vendor name should I
provide ?
I cannot find any vendor info in my Gerrit service account profile.
For example, is XXX OK ?

Thanks.





On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen
tangc...@cn.fujitsu.com mailto:tangc...@cn.fujitsu.com wrote:

Hi all,

I have constructed my CI system.
When I tested it with sandbox project, I posted a patch
to Gerrit.

https://review.openstack.org/201002

But I got this error in zuul scheduler:

2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run handler
awake
2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Fetching
trigger event
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Processing
trigger event TriggerEvent comment-added
openstack-dev/sandbox master 201002,1 Verified:1
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Project
openstack-dev/sandbox not found
2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run handler
sleeping


My /etc/zuul/layout/layout.yaml looks like this:

projects:
  - name: openstack-dev/sandbox
check:
  - noop-check-communication


My /etc/jenkins_jobs/config/projects.yaml looks like this:

- project:
name: sandbox
github-org: openstack-dev
node: master
vendor: myvendor

jobs:
- noop-check-communication
- dsvm-tempest-full:
node: 'devstack_slave ||
devstack-precise-check || d-p-c'


And Jenkins master works fine.


Does anyone know what is going on here ?


Thanks.






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/




-- 
*

*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
*
*
*Thanks  Regards,
*
*Abhishek*
/_Cloudbyte Inc. http://www.cloudbyte.com_/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

65 matches

Mail list logo