date:20160401

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1449#issuecomment-204634361
  
The debut of `upr`, so let me know if you want me to change the format.  
Also if you want me to add more tests in future runs, please let me know...

Given these results and the fact that the one failure consistently fails in 
my environment, I think we are ready to merge...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: speedup iptables setup

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1449#issuecomment-204633944
  

### CI Results




### 84/85 SUCCESSFUL

**1 Failure** (this and another similar test seem to fail consistently in 
my CI currently)
```
FAIL: Test redundant router internals
-
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_network_ops.py", 
line 483, in test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
"Attempt to retrieve google.com index page should be successful once 
rule is added!"
AssertionError: Attempt to retrieve google.com index page should be 
successful once rule is added!
```

Ran the following tests...
```
echo "Running tests with required_hardware=true"
nosetests --with-marvin --marvin-config=${marvinCfg} -s -a 
tags=advanced,required_hardware=true \
smoke/test_password_server.py \
smoke/test_vpc_redundant.py \
smoke/test_routers_iptables_default_policy.py \
smoke/test_routers_network_ops.py \
smoke/test_vpc_router_nics.py \
smoke/test_router_dhcphosts.py \
smoke/test_loadbalance.py \
smoke/test_internal_lb.py \
smoke/test_ssvm.py \
smoke/test_vpc_vpn.py \
smoke/test_privategw_acl.py \
smoke/test_network.py

echo "Running tests with required_hardware=false"
nosetests --with-marvin --marvin-config=${marvinCfg} -s -a 
tags=advanced,required_hardware=false \
smoke/test_routers.py \
smoke/test_network_acl.py \
smoke/test_reset_vm_on_reboot.py \
smoke/test_vm_life_cycle.py \
smoke/test_service_offerings.py \
smoke/test_network.py \
component/test_vpc_offerings.py \
component/test_vpc_routers.py
```


**Associated Uploads**

**`test_network_1WIKLA`:**

* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1449/test_network_1WIKLA/results.txt)

**`test_vpc_routers_8ZWKPC`:**

* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1449/test_vpc_routers_8ZWKPC/results.txt)


Uploads will be available until `2016-06-02 00:00:00 + GMT`

*Comment created by [`upr comment`](https://github.com/swill/upr).*




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9333: Exclude clusters from OV...

2016-04-01 Thread rafaelweingartner

Github user rafaelweingartner commented on the pull request:

https://github.com/apache/cloudstack/pull/1457#issuecomment-204624408
  
@nvazquez,
I find the âAlertManagerâ class terrible for that. Its name gives the 
idea of something that manages alerts and not clusters 
parameters/configurations. 

To add that property, I was thinking about the 
âcom.cloud.resource.ResourceManagerImpl.discoverCluster(AddClusterCmd)â 
method (lines 491 and 492 are already doing that for over-commitment factors), 
and then you could create a static entry at âCapacityManagerâ that seems to 
be the best option to hold such configurations.

If you change the âdiscoverClusterâ, you would still need a script to 
add that parameter to those clusters that the environment already has. You have 
to do that to enable the users to change those parameters and use that 
functionality through the web interface.

What do you think?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Redundant Router Interfaces

2016-04-01 Thread Dean Close

Hi Wei,

It looks like your patch will prevent redundant routers from bringing up public 
interfaces. We also need to:

1. shift the VRRP instance over to eth0 as eth2 will be down on the backup.
2. Ensure that eth0 and eth1 are up on both redundant virtual routers.

Your patch might already solve point 2. What do you think?

Point 1 will require modifying the keepalived.conf template. Do you want to 
handle this in your PR or shall I arrange that?

Kind Regards,

Dean Close
iCloudHosting.com
http://www.icloudhosting.com
Tel: 01582 227927

Spectrum House, Dunstable Road, Redbourn, AL3 7PR

**

The names iCloudHosting and iCloudHosting.com are trading styles of BBS 
Commerce Ltd which is registered in England and Wales, Company Number 04837714. 
Please use our trading address above for mail. Our registered office is 5 
Theale Lakes Business Park, Moulden Way, Sulhamstead, Reading, Berkshire, RG7 
4GB. VAT Registration Number GB 982 8230 94.

This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. If 
you are not the intended recipient you are not authorised to and must not 
disclose, copy, distribute, or retain this message or any part of it.

iCloudHosting accepts no responsibility for information, errors or omissions in 
this email.

**

On Fri, Apr 1, 2016 at 8:34 pm, Wei ZHOU  wrote:
actually not only this issue, but more
(1) restart vpc router will break the network if there are multiple tiers
in the vpc.
(2) check_heartbeat.sh does not work.

I will create tickets for them next Monday, and create PR on github.

-Wei

2016-04-01 21:14 GMT+02:00 Wei ZHOU :

> Dean,
>
> I just fixed it yesterday.
>
> the commit is
>
> ---
>
> diff --git a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> index 5f63c06..5256d03 100755
> --- a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> +++ b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> @@ -27,7 +27,7 @@ from CsRoute import CsRoute
> from CsRule import CsRule
>
> VRRP_TYPES = ['guest']
> -PUBLIC_INTERFACE = ['eth1']
> +VPC_PUBLIC_INTERFACE = ['eth1']
>
> class CsAddress(CsDataBag):
>
> @@ -323,7 +323,7 @@ class CsIP:
> # If redundant only bring up public interfaces that are
> not eth1.
> # Reason: private gateways are public interfaces.
> # master.py and keepalived will deal with eth1 public
> interface.
> - if self.cl.is_redundant() and (not self.is_public() or
> self.getDevice() not in PUBLIC_INTERFACE):
> + if self.cl.is_redundant() and (not self.is_public() or
> (self.config.is_vpc() and self.getDevice() not in VPC_PUBLIC_INTERFACE)):
> CsHelper.execute(cmd2)
> # if not redundant bring everything up
> if not self.cl.is_redundant():
>
> ---
>
> - Wei
>
> 2016-04-01 20:13 GMT+02:00 Dean Close :
>
>> Hi guys,
>>
>> I had been investigating a possible bug with the way interfaces are
>> managed on virtual routers. The public interfaces are being brought up on
>> backup routers and (because they boot second) they arp the IPs away from
>> the master. I'd been examining an idea for a fix but whilst doing so I
>> found that the system appears to be designed to bring up these interfaces.
>>
>> I suspect that a few things need to be reworked - but the changes
>> necessary go so far against what has been implemented that I wanted to open
>> this up before doing the work.
>>
>> Hopefully if I go through my findings you guys can help me see what I
>> might be getting wrong.
>>
>> The following was correct for pre-4.6 redundant routers:
>>
>> 1. Both routers get configured with IP addresses, routes and iptables
>> rules.
>> 2. Public interfaces are initially set as DOWN.
>> 3. Keepalived runs a VRRP instance on the private interface (eth0) to
>> negotiate MASTER/BACKUP roles.
>> 4. Keepalived manages the virtual IP on eth0 used as the public gateway
>> for the guest VMs.
>> 5. Keepalived uses a master notify script to bring up the public
>> interfaces.
>>
>> The above was true for pre-4.6 routers. Now, however, things appear to
>> work differently:
>>
>> 1. Both routers get configured as before.
>> 2. All interfaces apart from eth1 (the Hypervisor-link interface) are
>> set as UP.
>> 3. Keepalived runs a VRRP instance on the first public interface (eth2)
>> to negotiate MASTER/BACKUP roles.
>> 4. Keepalived manages the virtual IP as before.
>> 5. Keepalived uses a master notify script to bring up the public
>> interfaces (unnecessary)
>> 6. Keepalived uses a backup notify script to bring down the public
>> interfaces (unused)
>>
>> This is unexpected for the following reasons:
>>
>> 1. The keepalived notify script brings the public interfaces down when
>> transitioning to

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-01 Thread cristofolini

Github user cristofolini commented on the pull request:

https://github.com/apache/cloudstack/pull/1458#issuecomment-204588967
  
Shouldn't the path shown in line 55 of your config file be OS-independent?
I'd also suggest fixing the typo in the file name. At least I believe it 
was meant to be Barem**e**tal_AdvZone_Vmware.cfg instead of 
`Baremtal_AdvZone_Vmware.cfg`.
If you decide to fix these things, it would be nice if you squashed your 
commits into a single one, since they're all implementing the same feature. :)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9323: Fix cancel host maintena...

2016-04-01 Thread cristofolini

Github user cristofolini commented on the pull request:

https://github.com/apache/cloudstack/pull/1454#issuecomment-20457
  
@abhinandanprateek Looks like jenkins found some problems with your index. 
I'd suggest rebasing against the current master making all the needed merges 
and pushing again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack-8611:Handle SSH if server "for...

Github user GabrielBrascher commented on the pull request:

https://github.com/apache/cloudstack/pull/1459#issuecomment-204571209
  
Correcting myself, the history from the SshHelper 
(https://github.com/likitha/cloudstack/commits/CLOUDSTACK-8611/utils/src/com/cloud/utils/ssh/SshHelper.java)
 is not in the history of the SshHelper that I commited.

Either way, I am willing to push to any direction. As it is now, add the 
work done by the cherry-picked commit 
(b9181c689e0e7b5f1e28c81d73710196dfabd0ba), but also, this way seems that I did 
more than wat it is.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Redundant Router Interfaces

2016-04-01 Thread Wei ZHOU

actually not only this issue, but more
(1) restart vpc router will break the network if there are multiple tiers
in the vpc.
(2) check_heartbeat.sh does not work.

I will create tickets for them next Monday, and create PR on github.

-Wei

2016-04-01 21:14 GMT+02:00 Wei ZHOU :

> Dean,
>
> I just fixed it yesterday.
>
> the commit is
>
> ---
>
> diff --git a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> index 5f63c06..5256d03 100755
> --- a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> +++ b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
> @@ -27,7 +27,7 @@ from CsRoute import CsRoute
>  from CsRule import CsRule
>
>  VRRP_TYPES = ['guest']
> -PUBLIC_INTERFACE = ['eth1']
> +VPC_PUBLIC_INTERFACE = ['eth1']
>
>  class CsAddress(CsDataBag):
>
> @@ -323,7 +323,7 @@ class CsIP:
>  # If redundant only bring up public interfaces that are
> not eth1.
>  # Reason: private gateways are public interfaces.
>  # master.py and keepalived will deal with eth1 public
> interface.
> -if self.cl.is_redundant() and (not self.is_public() or
> self.getDevice() not in PUBLIC_INTERFACE):
> +if self.cl.is_redundant() and (not self.is_public() or
> (self.config.is_vpc() and self.getDevice() not in VPC_PUBLIC_INTERFACE)):
>  CsHelper.execute(cmd2)
>  # if not redundant bring everything up
>  if not self.cl.is_redundant():
>
> ---
>
> - Wei
>
> 2016-04-01 20:13 GMT+02:00 Dean Close :
>
>> Hi guys,
>>
>> I had been investigating a possible bug with the way interfaces are
>> managed on virtual routers. The public interfaces are being brought up on
>> backup routers and (because they boot second) they arp the IPs away from
>> the master. I'd been examining an idea for a fix but whilst doing so I
>> found that the system appears to be designed to bring up these interfaces.
>>
>> I suspect that a few things need to be reworked - but the changes
>> necessary go so far against what has been implemented that I wanted to open
>> this up before doing the work.
>>
>> Hopefully if I go through my findings you guys can help me see what I
>> might be getting wrong.
>>
>> The following was correct for pre-4.6 redundant routers:
>>
>>   1. Both routers get configured with IP addresses, routes and iptables
>> rules.
>>   2. Public interfaces are initially set as DOWN.
>>   3. Keepalived runs a VRRP instance on the private interface (eth0) to
>> negotiate MASTER/BACKUP roles.
>>   4. Keepalived manages the virtual IP on eth0 used as the public gateway
>> for the guest VMs.
>>   5. Keepalived uses a master notify script to bring up the public
>> interfaces.
>>
>> The above was true for pre-4.6 routers. Now, however, things appear to
>> work differently:
>>
>>   1. Both routers get configured as before.
>>   2. All interfaces apart from eth1 (the Hypervisor-link interface) are
>> set as UP.
>>   3. Keepalived runs a VRRP instance on the first public interface (eth2)
>> to negotiate MASTER/BACKUP roles.
>>   4. Keepalived manages the virtual IP as before.
>>   5. Keepalived uses a master notify script to bring up the public
>> interfaces (unnecessary)
>>   6. Keepalived uses a backup notify script to bring down the public
>> interfaces (unused)
>>
>> This is unexpected for the following reasons:
>>
>>   1. The keepalived notify script brings the public interfaces down when
>> transitioning to BACKUP - so how can we expect to run a VRRP instance over
>> eth2?
>>   2. If interfaces are down when transitioning to BACKUP, why are they
>> not expected to be down to begin with? (Before the router has become MASTER)
>>   3. Why are we running a VRRP instance over an interface with an IP that
>> will clash with another host on the network?
>>
>> The following method from the CsIP class in
>> /opt/cloud/bin/cs/CsAddress.py confuses matters futher:
>>
>> def check_is_up(self):
>>  """ Ensure device is up """
>>  cmd = "ip link show %s | grep 'state DOWN'" % self.getDevice()
>>  for i in CsHelper.execute(cmd):
>>  if " DOWN " in i:
>>  cmd2 = "ip link set %s up" % self.getDevice()
>>  # If redundant only bring up public interfaces that are
>> not eth1.
>>  # Reason: private gateways are public interfaces.
>>  # master.py and keepalived will deal with eth1 public
>> interface.
>>  if self.cl.is_redundant() and (not self.is_public() or
>> self.getDevice() not in PUBLIC_INTERFACE):
>>  CsHelper.execute(cmd2)
>>  # if not redundant bring everything up
>>  if not self.cl.is_redundant():
>>  CsHelper.execute(cmd2)
>>
>> The comments refer to eth1 as a public interface when this is the link

[GitHub] cloudstack pull request: speedup iptables setup

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1449#issuecomment-204537583
  
@remibergsma The test have already been started.  :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: speedup iptables setup

2016-04-01 Thread remibergsma

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1449#issuecomment-204532720
  
@kollyma Please squash the commits.
@swill Let's run integration tests on this before merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Hooking into the SecurityGroups

Oh sorry, I misunderstood what you needed to do.  The events stream may be
able to give you a way to hook in and have some code called on Create and
Delete, but you are right, it is probably not the best approach for what
you need to do.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Apr 1, 2016 at 12:47 PM, Nux!  wrote:

> Will,
>
> What I want to do is enable some sort of accounting based on iptables
> counters (I only want to measure traffic to certain destinations, i.e. not
> count inter-vm traffic in a SG zone).
> I doubt it's something the events stream can help, but it's a good thing
> to be aware of nonetheless.
>
> Thanks
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Will Stevens" 
> > To: dev@cloudstack.apache.org
> > Cc: "Pierre-Luc Dion" 
> > Sent: Friday, 1 April, 2016 15:38:19
> > Subject: Re: Hooking into the SecurityGroups
>
> > Patrick, did you guys end up doing anything with these events?  I know we
> > were messing with that at one point.  Did we learn anything interesting?
> > Is it a viable solution for what Nux is trying to do?
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Apr 1, 2016 at 9:59 AM, Patrick Dube 
> > wrote:
> >
> >> @swill. You can configure CloudStack to push it's events to both mysql
> and
> >> RabbitMQ or Kafka (
> >>
> >>
> http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/events.html
> >> )
> >>
> >> On Fri, Apr 1, 2016 at 8:37 AM, Will Stevens 
> >> wrote:
> >>
> >> > Pierre-Luc may have gotten something like this working. I think he
> tried
> >> > something similar.
> >> >
> >> > I think you can do something like redirect the events to redis and
> then
> >> > with a tool that uses the API and redis, build an event based overlay.
> >> >
> >> > I would have to look into this deeper, but I think PLD did some work
> on
> >> > this.
> >> > Thanks a lot Jayapal!
> >> >
> >> > Will, that sounds nice but I have not seen anything like this.
> >> >
> >> > Another interesting thing might be - as Wido suggested - to use
> libvirt
> >> > filters instead of our python scripts and I believe libvirt has it's
> own
> >> > way of hooking stuff into it. Could be another thing to explore,
> though
> >> it
> >> > sounds like a level lower than what you're proposing.
> >> >
> >> > Lucian
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > - Original Message -
> >> > > From: "Will Stevens" 
> >> > > To: dev@cloudstack.apache.org
> >> > > Sent: Friday, 1 April, 2016 12:50:16
> >> > > Subject: Re: Hooking into the SecurityGroups
> >> >
> >> > > Slightly off topic, but relevant. Ideally we could easily hook into
> the
> >> > > event logging and build added logic by simply tying them to specific
> >> > > events. This would limit the hackery and would provide a system that
> >> > others
> >> > > could use without having to change the core. Has anyone done
> something
> >> > like
> >> > > this?
> >> > > On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
> >> > >
> >> > >> Hi,
> >> > >>
> >> > >> I want to hook into the SGs and add a few iptables rules every
> time a
> >> VM
> >> > >> is spawned and delete them when the VM is moved/deleted.
> >> > >> Has anyone done this before? Any pointers before I go and butcher
> it?
> >> > :-)
> >> > >>
> >> > >> Lucian
> >> > >>
> >> > >> --
> >> > >> Sent from the Delta quadrant using Borg technology!
> >> > >>
> >> > >> Nux!
> >> > >> www.nux.ro
> >> >
>

Re: Redundant Router Interfaces

2016-04-01 Thread Wei ZHOU

Dean,

I just fixed it yesterday.

the commit is

---

diff --git a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
index 5f63c06..5256d03 100755
--- a/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
+++ b/systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py
@@ -27,7 +27,7 @@ from CsRoute import CsRoute
 from CsRule import CsRule

 VRRP_TYPES = ['guest']
-PUBLIC_INTERFACE = ['eth1']
+VPC_PUBLIC_INTERFACE = ['eth1']

 class CsAddress(CsDataBag):

@@ -323,7 +323,7 @@ class CsIP:
 # If redundant only bring up public interfaces that are
not eth1.
 # Reason: private gateways are public interfaces.
 # master.py and keepalived will deal with eth1 public
interface.
-if self.cl.is_redundant() and (not self.is_public() or
self.getDevice() not in PUBLIC_INTERFACE):
+if self.cl.is_redundant() and (not self.is_public() or
(self.config.is_vpc() and self.getDevice() not in VPC_PUBLIC_INTERFACE)):
 CsHelper.execute(cmd2)
 # if not redundant bring everything up
 if not self.cl.is_redundant():

---

- Wei

2016-04-01 20:13 GMT+02:00 Dean Close :

> Hi guys,
>
> I had been investigating a possible bug with the way interfaces are
> managed on virtual routers. The public interfaces are being brought up on
> backup routers and (because they boot second) they arp the IPs away from
> the master. I'd been examining an idea for a fix but whilst doing so I
> found that the system appears to be designed to bring up these interfaces.
>
> I suspect that a few things need to be reworked - but the changes
> necessary go so far against what has been implemented that I wanted to open
> this up before doing the work.
>
> Hopefully if I go through my findings you guys can help me see what I
> might be getting wrong.
>
> The following was correct for pre-4.6 redundant routers:
>
>   1. Both routers get configured with IP addresses, routes and iptables
> rules.
>   2. Public interfaces are initially set as DOWN.
>   3. Keepalived runs a VRRP instance on the private interface (eth0) to
> negotiate MASTER/BACKUP roles.
>   4. Keepalived manages the virtual IP on eth0 used as the public gateway
> for the guest VMs.
>   5. Keepalived uses a master notify script to bring up the public
> interfaces.
>
> The above was true for pre-4.6 routers. Now, however, things appear to
> work differently:
>
>   1. Both routers get configured as before.
>   2. All interfaces apart from eth1 (the Hypervisor-link interface) are
> set as UP.
>   3. Keepalived runs a VRRP instance on the first public interface (eth2)
> to negotiate MASTER/BACKUP roles.
>   4. Keepalived manages the virtual IP as before.
>   5. Keepalived uses a master notify script to bring up the public
> interfaces (unnecessary)
>   6. Keepalived uses a backup notify script to bring down the public
> interfaces (unused)
>
> This is unexpected for the following reasons:
>
>   1. The keepalived notify script brings the public interfaces down when
> transitioning to BACKUP - so how can we expect to run a VRRP instance over
> eth2?
>   2. If interfaces are down when transitioning to BACKUP, why are they not
> expected to be down to begin with? (Before the router has become MASTER)
>   3. Why are we running a VRRP instance over an interface with an IP that
> will clash with another host on the network?
>
> The following method from the CsIP class in /opt/cloud/bin/cs/CsAddress.py
> confuses matters futher:
>
> def check_is_up(self):
>  """ Ensure device is up """
>  cmd = "ip link show %s | grep 'state DOWN'" % self.getDevice()
>  for i in CsHelper.execute(cmd):
>  if " DOWN " in i:
>  cmd2 = "ip link set %s up" % self.getDevice()
>  # If redundant only bring up public interfaces that are
> not eth1.
>  # Reason: private gateways are public interfaces.
>  # master.py and keepalived will deal with eth1 public
> interface.
>  if self.cl.is_redundant() and (not self.is_public() or
> self.getDevice() not in PUBLIC_INTERFACE):
>  CsHelper.execute(cmd2)
>  # if not redundant bring everything up
>  if not self.cl.is_redundant():
>  CsHelper.execute(cmd2)
>
> The comments refer to eth1 as a public interface when this is the link to
> the hypervisor. Indeed, PUBLIC_INTERFACE is defined on line 31 as ['eth1'].
> But keepalived and master.py don't influence eth1 at all. This looks like a
> mistake.
>
> Lastly, the logic of this line looks flawed:
>
>   if self.cl.is_redundant() and (not self.is_public() or self.getDevice()
> not in PUBLIC_INTERFACE)
>
> As PUBLIC_INTERFACE is limited to eth1, the `not self.is_public()` will be
> ignored. Public IPs will never be assigned

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204527828
  
I get the same behavior with Java 7.

```
$ mvn --version
Apache Maven 3.0.5 (Red Hat 3.0.5-16)
Maven home: /usr/share/maven
Java version: 1.7.0_99, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.10.1.el7.x86_64", arch: "amd64", 
family: "unix"
```

It fails here:
```
[INFO] Apache CloudStack Framework - Jobs  SUCCESS [2.429s]
[INFO] Apache CloudStack Cloud Engine Internal Components API  SUCCESS 
[1.062s]
[INFO] Apache CloudStack Server .. FAILURE 
[1:01.548s]
[INFO] Apache CloudStack Framework - Quota ... SKIPPED
[INFO] Apache CloudStack Usage Server  SKIPPED
```

When running this:
```
Running org.apache.cloudstack.network.lb.CertServiceTest
Tests run: 13, Failures: 2, Errors: 1, Skipped: 0, Time elapsed: 0.325 sec 
<<< FAILURE! - in org.apache.cloudstack.network.lb.CertServiceTest

runUploadSslCertWithCAChain(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0.05 sec  <<< ERROR!
com.cloud.utils.exception.CloudRuntimeException: Error parsing certificate 
data Parsing certificate/key failed: Could not parse certificate: 
java.io.IOException: Empty input
at 
org.apache.cloudstack.network.lb.CertServiceImpl.uploadSslCert(CertServiceImpl.java:150)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertWithCAChain(CertServiceTest.java:143)

runUploadSslCertBadChain(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0.016 sec  <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertBadChain(CertServiceTest.java:290)


runUploadSslCertNoRootCert(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0.023 sec  <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertNoRootCert(CertServiceTest.java:342)
```

We need to better understand why this is happening...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204520010
  
I have changed the way my CI environment is being built, so it is built to 
use Java 7 now.  I am testing this PR now, so I will update with status when I 
have more...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: speedup iptables setup

2016-04-01 Thread resmo

Github user resmo commented on the pull request:

https://github.com/apache/cloudstack/pull/1449#issuecomment-204500797
  
LGTM! please merge


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Redundant Router Interfaces

2016-04-01 Thread Dean Close

Hi guys,

I had been investigating a possible bug with the way interfaces are managed on 
virtual routers. The public interfaces are being brought up on backup routers 
and (because they boot second) they arp the IPs away from the master. I'd been 
examining an idea for a fix but whilst doing so I found that the system appears 
to be designed to bring up these interfaces.

I suspect that a few things need to be reworked - but the changes necessary go 
so far against what has been implemented that I wanted to open this up before 
doing the work.

Hopefully if I go through my findings you guys can help me see what I might be 
getting wrong.

The following was correct for pre-4.6 redundant routers:
 
  1. Both routers get configured with IP addresses, routes and iptables rules.
  2. Public interfaces are initially set as DOWN.
  3. Keepalived runs a VRRP instance on the private interface (eth0) to 
negotiate MASTER/BACKUP roles.
  4. Keepalived manages the virtual IP on eth0 used as the public gateway for 
the guest VMs.
  5. Keepalived uses a master notify script to bring up the public interfaces.

The above was true for pre-4.6 routers. Now, however, things appear to work 
differently:

  1. Both routers get configured as before.
  2. All interfaces apart from eth1 (the Hypervisor-link interface) are set as 
UP.
  3. Keepalived runs a VRRP instance on the first public interface (eth2) to 
negotiate MASTER/BACKUP roles.
  4. Keepalived manages the virtual IP as before.
  5. Keepalived uses a master notify script to bring up the public interfaces 
(unnecessary)
  6. Keepalived uses a backup notify script to bring down the public interfaces 
(unused)

This is unexpected for the following reasons:

  1. The keepalived notify script brings the public interfaces down when 
transitioning to BACKUP - so how can we expect to run a VRRP instance over eth2?
  2. If interfaces are down when transitioning to BACKUP, why are they not 
expected to be down to begin with? (Before the router has become MASTER)
  3. Why are we running a VRRP instance over an interface with an IP that will 
clash with another host on the network?

The following method from the CsIP class in /opt/cloud/bin/cs/CsAddress.py 
confuses matters futher:

def check_is_up(self):
 """ Ensure device is up """
 cmd = "ip link show %s | grep 'state DOWN'" % self.getDevice()
 for i in CsHelper.execute(cmd):
 if " DOWN " in i:
 cmd2 = "ip link set %s up" % self.getDevice()
 # If redundant only bring up public interfaces that are not 
eth1.
 # Reason: private gateways are public interfaces.
 # master.py and keepalived will deal with eth1 public 
interface. 
 if self.cl.is_redundant() and (not self.is_public() or 
self.getDevice() not in PUBLIC_INTERFACE):
 CsHelper.execute(cmd2)
 # if not redundant bring everything up
 if not self.cl.is_redundant():
 CsHelper.execute(cmd2)
 
The comments refer to eth1 as a public interface when this is the link to the 
hypervisor. Indeed, PUBLIC_INTERFACE is defined on line 31 as ['eth1']. But 
keepalived and master.py don't influence eth1 at all. This looks like a mistake.

Lastly, the logic of this line looks flawed:

  if self.cl.is_redundant() and (not self.is_public() or self.getDevice() not 
in PUBLIC_INTERFACE)

As PUBLIC_INTERFACE is limited to eth1, the `not self.is_public()` will be 
ignored. Public IPs will never be assigned to eth1, so this line evaluates as:

  
  if self.cl.is_redundant() and (self.getDevice() not in PUBLIC_INTERFACE)

which reduces even further to:

  if self.cs.is_redundant() and self.is_control()


What would need doing
-

  1. The keepalived.conf template would need to be changed to run the VRRP 
instance over eth0.
  2. The check_is_up method of the CsIP class should be renamed to 
'bring_up_interfaces'. For redundant routers it should ignore IPs that pass 
is_public or needs_vrrp.
  3. The arpPing method should do nothing if the interface is down.
  4. The PUBLIC_INTERFACE constant should be either renamed or dropped 
altogether.
  5. Other things that I haven't considered?


I'd really appreciate any feedback on this. It's possible that I've got it all 
wrong but I'm suspecting not. I just don't want to tread on anyone's toes by 
submitting a PR that goes against what appears to be an explicit design 
decision.


Kind regards,

Dean Close
iCloudHosting.com
http://www.icloudhosting.com
Tel: 01582 227927

Unit 2, Smallmead Road, Reading RG2 0QS

**
The names iCloudHosting and iCloudHosting.com are trading styles of BBS 
Commerce Ltd which is registered in England and Wales, Company Number 04837714. 
Please use our trading address above for mail. Our registered office is 5 
Theale Lakes Business Park, Moulden Way,

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

2016-04-01 Thread jburwell

Github user jburwell commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204494886
  
@swill Java8 is not currently supported by the management server.  We know 
that the process will collapse after running for 24-48 hours.  Part of the 
issue is that Spring 3 has known issues on Java8.

Per my earlier feedback, I think the Bouncy Castle upgrade should be split 
into a separate PR.  The issue could be with the changes to the certificate 
handling code or an issue between the new Bouncy Castle version and one of the 
transitive dependency updates.  Breaking up this PR would make it much easier 
to understand whether or not this issue is caused by the dependency upgrades or 
the code change.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204490175
  
There were quite a few changes to the way certificates are handled in this 
PR.  Changes to versions of bouncycastle as well as changing the way the Pem 
files are handles.  In light of that, the errors I am getting are pretty 
relevant.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204486265
  
What about this PR forces me to downgrade to Java 7?  My Java 8 build 
environment has been working fine against master until this PR.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack-8611:Handle SSH if server "for...

Github user GabrielBrascher commented on the pull request:

https://github.com/apache/cloudstack/pull/1459#issuecomment-204484447

@DaanHoogland thanks for the hint, but the problem is not related to line
ending.

The problem is that there are two (2) different SshHelper classes in the
ACS history. Although the content is the same, they were moved.

The master SshHelper
(https://github.com/apache/cloudstack/commits/75b68c68298dab270782be68f8f9f28e551fc5d0/utils/src/main/java/com/cloud/utils/ssh/SshHelper.java)
is located at
"cloudstack/utils/src/main/java/com/cloud/utils/ssh/SshHelper.java". The file
that I got to edit is the predecessor of the master SshHelper and is located at
"cloudstack/utils/src/com/cloud/utils/ssh/SshHelper.java".

I am not sure about how to proceed.

On the one hand, by keeping this PR as it is, it will preserve the history
of all commits of this (SshHelper) class
(https://github.com/likitha/cloudstack/commits/CLOUDSTACK-8611/utils/src/com/cloud/utils/ssh/SshHelper.java)
and the path will remain as the master; on the other hand, if I add these
changes in the SshHelper at the current master, it will appear that I made all
the changes in this class.

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

2016-04-01 Thread bhaisaab

Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204474193
  
@swill based on the output you've shared from your CI env -- `Java version: 
1.8.0_77, vendor: Oracle Corporation`; you're using Java 8. Please switch to 
Java 7 (openjdk or oraclejava) and try again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204470246
  
The above results are from my CI environment which is running the following:
```
$ mvn --version
Apache Maven 3.0.5 (Red Hat 3.0.5-16)
Maven home: /usr/share/maven
Java version: 1.8.0_77, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.10.1.el7.x86_64", arch: "amd64", 
family: "unix"
```

I just tried on my laptop which has the following config:
```
$ mvn --version
Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 
11:22:22-0400)
Maven home: /Users/swill/Dropbox/code/src/apache-maven-3.1.1
Java version: 1.7.0_40, vendor: Oracle Corporation
Java home: 
/Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
```

When building, I get the following:
```
---
 T E S T S
---
Running com.cloud.network.bigswitch.BigSwitchApiTest
log4j:WARN No appenders could be found for logger 
(org.apache.commons.httpclient.HttpClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.647 sec 
- in com.cloud.network.bigswitch.BigSwitchApiTest
Running com.cloud.network.bigswitch.BigSwitchBcfUtilsTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.538 sec - 
in com.cloud.network.bigswitch.BigSwitchBcfUtilsTest
Running com.cloud.network.resource.BigSwitchBcfResourceTest
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.084 sec 
- in com.cloud.network.resource.BigSwitchBcfResourceTest

Results :

Tests run: 64, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ 
cloud-plugin-network-bigswitch ---
[INFO] Building jar: 
/Users/swill/Dropbox/code/cs/cloudstack/plugins/network-elements/bigswitch/target/cloud-plugin-network-bigswitch-4.9.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ 
cloud-plugin-network-bigswitch ---
[INFO] 
[INFO] >>> maven-pmd-plugin:3.3:check (cloudstack-pmd) @ 
cloud-plugin-network-bigswitch >>>
[INFO] 
[INFO] --- maven-pmd-plugin:3.3:pmd (pmd) @ cloud-plugin-network-bigswitch 
---
[WARNING] Unable to locate Source XRef to link to - DISABLED
[WARNING] Unable to locate Source XRef to link to - DISABLED
Apr 01, 2016 12:41:07 PM net.sourceforge.pmd.PMD removeBrokenRules
WARNING: Removed misconfigured rule: LoosePackageCoupling  cause: No 
packages or classes specified
Apr 01, 2016 12:41:08 PM net.sourceforge.pmd.PMD removeBrokenRules
WARNING: Removed misconfigured rule: LoosePackageCoupling  cause: No 
packages or classes specified
Apr 01, 2016 12:41:08 PM net.sourceforge.pmd.PMD removeBrokenRules
WARNING: Removed misconfigured rule: LoosePackageCoupling  cause: No 
packages or classes specified
Apr 01, 2016 12:41:08 PM net.sourceforge.pmd.PMD removeBrokenRules
WARNING: Removed misconfigured rule: LoosePackageCoupling  cause: No 
packages or classes specified
Apr 01, 2016 12:41:08 PM net.sourceforge.pmd.PMD removeBrokenRules
WARNING: Removed misconfigured rule: LoosePackageCoupling  cause: No 
packages or classes specified
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00010ee34f21, pid=38330, tid=12547
#
# JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 
1.7.0_40-b43)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode bsd-amd64 
compressed oops)
# Problematic frame:
# V  [libjvm.dylib+0x234f21]  oopDesc::size()+0x8d
#
# Failed to write core dump. Core dumps have been disabled. To enable core 
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/swill/Dropbox/code/cs/cloudstack/hs_err_pid38330.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#
Abort trap: 6
```

As suggested, I ran: `ulimit -c unlimited`

After making this change, it builds on my laptop...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204472158
  
Here is the core dump I had originally when I tried to build on my mac.


[hs_err_pid38330.log.zip](https://github.com/apache/cloudstack/files/200015/hs_err_pid38330.log.zip)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Hooking into the SecurityGroups

Will,

What I want to do is enable some sort of accounting based on iptables counters 
(I only want to measure traffic to certain destinations, i.e. not count 
inter-vm traffic in a SG zone).
I doubt it's something the events stream can help, but it's a good thing to be 
aware of nonetheless.

Thanks

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Will Stevens" 
> To: dev@cloudstack.apache.org
> Cc: "Pierre-Luc Dion" 
> Sent: Friday, 1 April, 2016 15:38:19
> Subject: Re: Hooking into the SecurityGroups

> Patrick, did you guys end up doing anything with these events?  I know we
> were messing with that at one point.  Did we learn anything interesting?
> Is it a viable solution for what Nux is trying to do?
> 
> *Will STEVENS*
> Lead Developer
> 
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
> 
> On Fri, Apr 1, 2016 at 9:59 AM, Patrick Dube 
> wrote:
> 
>> @swill. You can configure CloudStack to push it's events to both mysql and
>> RabbitMQ or Kafka (
>>
>> http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/events.html
>> )
>>
>> On Fri, Apr 1, 2016 at 8:37 AM, Will Stevens 
>> wrote:
>>
>> > Pierre-Luc may have gotten something like this working. I think he tried
>> > something similar.
>> >
>> > I think you can do something like redirect the events to redis and then
>> > with a tool that uses the API and redis, build an event based overlay.
>> >
>> > I would have to look into this deeper, but I think PLD did some work on
>> > this.
>> > Thanks a lot Jayapal!
>> >
>> > Will, that sounds nice but I have not seen anything like this.
>> >
>> > Another interesting thing might be - as Wido suggested - to use libvirt
>> > filters instead of our python scripts and I believe libvirt has it's own
>> > way of hooking stuff into it. Could be another thing to explore, though
>> it
>> > sounds like a level lower than what you're proposing.
>> >
>> > Lucian
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > - Original Message -
>> > > From: "Will Stevens" 
>> > > To: dev@cloudstack.apache.org
>> > > Sent: Friday, 1 April, 2016 12:50:16
>> > > Subject: Re: Hooking into the SecurityGroups
>> >
>> > > Slightly off topic, but relevant. Ideally we could easily hook into the
>> > > event logging and build added logic by simply tying them to specific
>> > > events. This would limit the hackery and would provide a system that
>> > others
>> > > could use without having to change the core. Has anyone done something
>> > like
>> > > this?
>> > > On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> I want to hook into the SGs and add a few iptables rules every time a
>> VM
>> > >> is spawned and delete them when the VM is moved/deleted.
>> > >> Has anyone done this before? Any pointers before I go and butcher it?
>> > :-)
>> > >>
>> > >> Lucian
>> > >>
>> > >> --
>> > >> Sent from the Delta quadrant using Borg technology!
>> > >>
>> > >> Nux!
>> > >> www.nux.ro
>> >

Re: Hooking into the SecurityGroups

2016-04-01 Thread Patrick Dube

They are the same events that get pushed to the cloud.event MySQL table. So
you get vm creation, vm deletion, etc. Most API driven actions will end up
in that table. Using the RabbitMQ configuration, I got it to push the
events successfully, but haven't expanded on it yet. Soon™

On Fri, Apr 1, 2016 at 10:38 AM, Will Stevens  wrote:

> Patrick, did you guys end up doing anything with these events?  I know we
> were messing with that at one point.  Did we learn anything interesting?
> Is it a viable solution for what Nux is trying to do?
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Apr 1, 2016 at 9:59 AM, Patrick Dube 
> wrote:
>
> > @swill. You can configure CloudStack to push it's events to both mysql
> and
> > RabbitMQ or Kafka (
> >
> >
> http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/events.html
> > )
> >
> > On Fri, Apr 1, 2016 at 8:37 AM, Will Stevens 
> > wrote:
> >
> > > Pierre-Luc may have gotten something like this working. I think he
> tried
> > > something similar.
> > >
> > > I think you can do something like redirect the events to redis and then
> > > with a tool that uses the API and redis, build an event based overlay.
> > >
> > > I would have to look into this deeper, but I think PLD did some work on
> > > this.
> > > Thanks a lot Jayapal!
> > >
> > > Will, that sounds nice but I have not seen anything like this.
> > >
> > > Another interesting thing might be - as Wido suggested - to use libvirt
> > > filters instead of our python scripts and I believe libvirt has it's
> own
> > > way of hooking stuff into it. Could be another thing to explore, though
> > it
> > > sounds like a level lower than what you're proposing.
> > >
> > > Lucian
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > - Original Message -
> > > > From: "Will Stevens" 
> > > > To: dev@cloudstack.apache.org
> > > > Sent: Friday, 1 April, 2016 12:50:16
> > > > Subject: Re: Hooking into the SecurityGroups
> > >
> > > > Slightly off topic, but relevant. Ideally we could easily hook into
> the
> > > > event logging and build added logic by simply tying them to specific
> > > > events. This would limit the hackery and would provide a system that
> > > others
> > > > could use without having to change the core. Has anyone done
> something
> > > like
> > > > this?
> > > > On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I want to hook into the SGs and add a few iptables rules every time
> a
> > VM
> > > >> is spawned and delete them when the VM is moved/deleted.
> > > >> Has anyone done this before? Any pointers before I go and butcher
> it?
> > > :-)
> > > >>
> > > >> Lucian
> > > >>
> > > >> --
> > > >> Sent from the Delta quadrant using Borg technology!
> > > >>
> > > >> Nux!
> > > >> www.nux.ro
> > >
> >
>

[GitHub] cloudstack pull request: CLOUDSTACK-9333: Exclude clusters from OV...

2016-04-01 Thread nvazquez

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1457#issuecomment-204431280
  
@rafaelweingartner thanks for reviewing! I couldn't find any document with 
all cluster_details, however we could find out that this property could be 
defined like in 
cloudstack/engine/components-api/src/com/cloud/alert/AlertManager.java,
 those properties are inserted in configuration table, specifying 
the class which defines it in component field. 
I think this new property cluster.storage.operations.exclude 
could be done that way, do you agree? In that case, in which component could it 
be defined?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

2016-04-01 Thread bhaisaab

Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204429261
  
@swill I'm guessing an environment issue, likely an empty java keystore; 
what is the version of your maven, java? Are you using Oracle Java or OpenJDK?

This works in my local env that has maven 3.3.3, oracle-java 1.7.0_80 (I 
had issues with distro provided openjdk 7u95, and I found other folks on ML 
have reported issues with latest openjdk as well, therefore I had switched to 
oracle-java).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Hooking into the SecurityGroups

Patrick, did you guys end up doing anything with these events?  I know we
were messing with that at one point.  Did we learn anything interesting?
Is it a viable solution for what Nux is trying to do?

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Apr 1, 2016 at 9:59 AM, Patrick Dube 
wrote:

> @swill. You can configure CloudStack to push it's events to both mysql and
> RabbitMQ or Kafka (
>
> http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/events.html
> )
>
> On Fri, Apr 1, 2016 at 8:37 AM, Will Stevens 
> wrote:
>
> > Pierre-Luc may have gotten something like this working. I think he tried
> > something similar.
> >
> > I think you can do something like redirect the events to redis and then
> > with a tool that uses the API and redis, build an event based overlay.
> >
> > I would have to look into this deeper, but I think PLD did some work on
> > this.
> > Thanks a lot Jayapal!
> >
> > Will, that sounds nice but I have not seen anything like this.
> >
> > Another interesting thing might be - as Wido suggested - to use libvirt
> > filters instead of our python scripts and I believe libvirt has it's own
> > way of hooking stuff into it. Could be another thing to explore, though
> it
> > sounds like a level lower than what you're proposing.
> >
> > Lucian
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> > > From: "Will Stevens" 
> > > To: dev@cloudstack.apache.org
> > > Sent: Friday, 1 April, 2016 12:50:16
> > > Subject: Re: Hooking into the SecurityGroups
> >
> > > Slightly off topic, but relevant. Ideally we could easily hook into the
> > > event logging and build added logic by simply tying them to specific
> > > events. This would limit the hackery and would provide a system that
> > others
> > > could use without having to change the core. Has anyone done something
> > like
> > > this?
> > > On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
> > >
> > >> Hi,
> > >>
> > >> I want to hook into the SGs and add a few iptables rules every time a
> VM
> > >> is spawned and delete them when the VM is moved/deleted.
> > >> Has anyone done this before? Any pointers before I go and butcher it?
> > :-)
> > >>
> > >> Lucian
> > >>
> > >> --
> > >> Sent from the Delta quadrant using Borg technology!
> > >>
> > >> Nux!
> > >> www.nux.ro
> >
>

Re: Hooking into the SecurityGroups

2016-04-01 Thread Patrick Dube

@swill. You can configure CloudStack to push it's events to both mysql and
RabbitMQ or Kafka (
http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/events.html
)

On Fri, Apr 1, 2016 at 8:37 AM, Will Stevens 
wrote:

> Pierre-Luc may have gotten something like this working. I think he tried
> something similar.
>
> I think you can do something like redirect the events to redis and then
> with a tool that uses the API and redis, build an event based overlay.
>
> I would have to look into this deeper, but I think PLD did some work on
> this.
> Thanks a lot Jayapal!
>
> Will, that sounds nice but I have not seen anything like this.
>
> Another interesting thing might be - as Wido suggested - to use libvirt
> filters instead of our python scripts and I believe libvirt has it's own
> way of hooking stuff into it. Could be another thing to explore, though it
> sounds like a level lower than what you're proposing.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Will Stevens" 
> > To: dev@cloudstack.apache.org
> > Sent: Friday, 1 April, 2016 12:50:16
> > Subject: Re: Hooking into the SecurityGroups
>
> > Slightly off topic, but relevant. Ideally we could easily hook into the
> > event logging and build added logic by simply tying them to specific
> > events. This would limit the hackery and would provide a system that
> others
> > could use without having to change the core. Has anyone done something
> like
> > this?
> > On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
> >
> >> Hi,
> >>
> >> I want to hook into the SGs and add a few iptables rules every time a VM
> >> is spawned and delete them when the VM is moved/deleted.
> >> Has anyone done this before? Any pointers before I go and butcher it?
> :-)
> >>
> >> Lucian
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
>

[GitHub] cloudstack pull request: CLOUDSTACK-9298: Improve performance of r...

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1425#issuecomment-204397261
  
Yes I meant to post the results, but I was too quick trying to get thing 
setup for the next run and I blew away my environment.  :(  Im a noob...  I 
will be posting the results going forward.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: maven: Upgrade dependency versions

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1397#issuecomment-204396178
  
This is failing to build in my CI environment.  The following tests are 
causing the build to fail.

It is failing here:
```
[INFO] Apache CloudStack Framework - Event Notification .. SUCCESS [0.485s]
[INFO] Apache CloudStack Cloud Engine Schema Component ... SUCCESS [4.719s]
[INFO] Apache CloudStack Framework - Jobs  SUCCESS [2.225s]
[INFO] Apache CloudStack Cloud Engine Internal Components API  SUCCESS 
[0.951s]
[INFO] Apache CloudStack Server .. FAILURE [47.577s]
[INFO] Apache CloudStack Framework - Quota ... SKIPPED
[INFO] Apache CloudStack Usage Server  SKIPPED
[INFO] Apache CloudStack Cloud Engine Orchestration Component  SKIPPED
```

With these failed tests:
```
Tests run: 13, Failures: 2, Errors: 1, Skipped: 0, Time elapsed: 0.348 sec 
<<< FAILURE! - in org.apache.cloudstack.network.lb.CertServiceTest

runUploadSslCertWithCAChain(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0.026 sec  <<< ERROR!
com.cloud.utils.exception.CloudRuntimeException: Error parsing certificate 
data Parsing certificate/key failed: Could not parse certificate: 
java.io.IOException: Empty input
at 
org.apache.cloudstack.network.lb.CertServiceImpl.uploadSslCert(CertServiceImpl.java:150)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertWithCAChain(CertServiceTest.java:143)

runUploadSslCertBadChain(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0.011 sec  <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertBadChain(CertServiceTest.java:290)


runUploadSslCertNoRootCert(org.apache.cloudstack.network.lb.CertServiceTest)  
Time elapsed: 0 sec  <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.cloudstack.network.lb.CertServiceTest.runUploadSslCertNoRootCert(CertServiceTest.java:342)
```

Any ideas???



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9298: Improve performance of r...

2016-04-01 Thread remibergsma

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1425#issuecomment-204382745
  
@swill Congrats on the first bubble-tested PR :-) I would recommend posting 
the results though, the results.txt file shows which tests were run. If later 
we want to check, it's documented here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Hooking into the SecurityGroups

Pierre-Luc may have gotten something like this working. I think he tried
something similar.

I think you can do something like redirect the events to redis and then
with a tool that uses the API and redis, build an event based overlay.

I would have to look into this deeper, but I think PLD did some work on
this.
Thanks a lot Jayapal!

Will, that sounds nice but I have not seen anything like this.

Another interesting thing might be - as Wido suggested - to use libvirt
filters instead of our python scripts and I believe libvirt has it's own
way of hooking stuff into it. Could be another thing to explore, though it
sounds like a level lower than what you're proposing.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Will Stevens" 
> To: dev@cloudstack.apache.org
> Sent: Friday, 1 April, 2016 12:50:16
> Subject: Re: Hooking into the SecurityGroups

> Slightly off topic, but relevant. Ideally we could easily hook into the
> event logging and build added logic by simply tying them to specific
> events. This would limit the hackery and would provide a system that
others
> could use without having to change the core. Has anyone done something
like
> this?
> On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
>
>> Hi,
>>
>> I want to hook into the SGs and add a few iptables rules every time a VM
>> is spawned and delete them when the VM is moved/deleted.
>> Has anyone done this before? Any pointers before I go and butcher it? :-)
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro

Re: Hooking into the SecurityGroups

Thanks a lot Jayapal!

Will, that sounds nice but I have not seen anything like this.

Another interesting thing might be - as Wido suggested - to use libvirt filters 
instead of our python scripts and I believe libvirt has it's own way of hooking 
stuff into it. Could be another thing to explore, though it sounds like a level 
lower than what you're proposing.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Will Stevens" 
> To: dev@cloudstack.apache.org
> Sent: Friday, 1 April, 2016 12:50:16
> Subject: Re: Hooking into the SecurityGroups

> Slightly off topic, but relevant. Ideally we could easily hook into the
> event logging and build added logic by simply tying them to specific
> events. This would limit the hackery and would provide a system that others
> could use without having to change the core. Has anyone done something like
> this?
> On Apr 1, 2016 6:42 AM, "Nux!"  wrote:
> 
>> Hi,
>>
>> I want to hook into the SGs and add a few iptables rules every time a VM
>> is spawned and delete them when the VM is moved/deleted.
>> Has anyone done this before? Any pointers before I go and butcher it? :-)
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro

Re: Hooking into the SecurityGroups

Slightly off topic, but relevant. Ideally we could easily hook into the
event logging and build added logic by simply tying them to specific
events. This would limit the hackery and would provide a system that others
could use without having to change the core. Has anyone done something like
this?
On Apr 1, 2016 6:42 AM, "Nux!"  wrote:

> Hi,
>
> I want to hook into the SGs and add a few iptables rules every time a VM
> is spawned and delete them when the VM is moved/deleted.
> Has anyone done this before? Any pointers before I go and butcher it? :-)
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>

Re: Hooking into the SecurityGroups

2016-04-01 Thread Jayapal Uradi

Hi Nux,

You need to look at the vmops file for xenserver and security_group.py for the 
kvm.

Look at the below methods in these file.
 destroy_network_rules_for_vm, default_network_rules.

Thanks,
jayapal


On 01-Apr-2016, at 4:12 pm, Nux! > wrote:

Hi,

I want to hook into the SGs and add a few iptables rules every time a VM is 
spawned and delete them when the VM is moved/deleted.
Has anyone done this before? Any pointers before I go and butcher it? :-)

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro




DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Hooking into the SecurityGroups

Hi,

I want to hook into the SGs and add a few iptables rules every time a VM is 
spawned and delete them when the VM is moved/deleted.
Has anyone done this before? Any pointers before I go and butcher it? :-)

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

Github user cloudsadhu commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58188259
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
+}
+],
+"clustertype": "CloudManaged"
+}
+],
+"gateway": "10.x.x.x"
+}
+],
+"internaldns1": "10.x.x.x",
+   "baremetalrcturl":

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

Github user cloudsadhu commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58188294
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
+}
+],
+"clustertype": "CloudManaged"
+}
+],
+"gateway": "10.x.x.x"
+}
+],
+"internaldns1": "10.x.x.x",
+   "baremetalrcturl":

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

Github user cloudsadhu commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58188213
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
--- End diff --

Done, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-01 Thread shwetaag

Github user shwetaag commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58182115
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
+}
+],
+"clustertype": "CloudManaged"
+}
+],
+"gateway": "10.x.x.x"
+}
+],
+"internaldns1": "10.x.x.x",
+   "baremetalrcturl":

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-01 Thread shwetaag

Github user shwetaag commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58181661
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
+}
+],
+"clustertype": "CloudManaged"
+}
+],
+"gateway": "10.x.x.x"
+}
+],
+"internaldns1": "10.x.x.x",
+   "baremetalrcturl":

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-01 Thread shwetaag

Github user shwetaag commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1458#discussion_r58180544
  
--- Diff: tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg ---
@@ -0,0 +1,191 @@
+{
+"zones": [
+{
+"name": "ZoneBaremetal",
+"guestcidraddress": "10.1.1.0/24",
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+],
+"dns1": "10.x.x.x",
+"vlan": "1020-1025",
+"physical_networks": [
+{
+"broadcastdomainrange": "Zone",
+"name": "Sandbox-pnet",
+"traffictypes": [
+{
+"typ": "Guest"
+},
+{
+"typ": "Management"
+},
+{
+"typ": "Public"
+}
+],
+"providers": [
+{
+"broadcastdomainrange": "ZONE",
+"name": "VirtualRouter"
+}
+]
+}
+],
+   "logger": {
+"LogFolderPath": "C:/Marvin"
+   },
+"ipranges": [
+{
+"startip": "10.x.x.x",
+"endip": "10.x.x.x",
+"netmask": "255.255.255.0",
+"vlan": "1220",
+"gateway": "10.x.x.1"
+}
+],
+"networktype": "Advanced",
+"pods": [
+{
+"endip": "10.x.x.x",
+"name": "POD01",
+"startip": "10.x.x.x",
+"netmask": "255.255.255.192",
+   
"vmwaredc":{"name":"dc-bare","vcenter":"10.x.x.x","username":"Administrator@vsphere.local","password":"Password"},
+"clusters": [
+{
+"clustername": "10.x.x.x/dc-bar/cluster1",
+"hypervisor": "VmWare",
+"username": "Administrator@vsphere.local",
+"clustertype": "ExternalManaged",
+"password": "Password",
+"url": "http://10.x.x.x/dc-bare/cluster1;,
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+},
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password"
+}
+],
+"primaryStorages": [
+{
+"url": 
"nfs://10.x.x.x:/export/home/sadhu/primrc2",
+"name": "PS0"
+}
+]
+},
+   {
+"clustername": "C1",
+"hypervisor": "BareMetal",
+"hosts": [
+{
+"username": "root",
+"url": "http://10.x.x.x;,
+"password": "password",
+"hostmac": "b2:ac:21:s2:1c:20",
+"cpunumber": "1",
+"cpuspeed": "1000",
+"memory": "1024",
+   
"hosttags": "host15"
--- End diff --

indentation issues


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [SSL CERTS] Importing ROOT and INTERMEDIATE certs for SSVM

Ilya,

See the last few lines of this post, I  had a similar problem a while back:
http://www.nux.ro/archive/2014/03/Run_your_own_realhostip.html

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "ilya" 
> To: dev@cloudstack.apache.org
> Sent: Friday, 1 April, 2016 01:09:56
> Subject: Re: [SSL CERTS] Importing ROOT and INTERMEDIATE certs for SSVM

> I have a web-service that serves CloudStack templates, the SSL on the
> download web service is signed by internal CA. This means i need to
> inject the intermediate CA as well as ROOT CA into SSVM's java keystore
> - for java client to be able to recognize the Certs and download the
> template from remote repository.
> 
> 
> 
> 
> 
> On 3/29/16 4:48 AM, Daan Hoogland wrote:
>> Ilya, to my knowledge the certificate won't be saved on file. It will be
>> loaded from the command coming from the MS in the agent directly. Why are
>> you looking to update the ssvm? I thought these are only used in the
>> consoleproxy.
>> 
>> On Tue, Mar 29, 2016 at 12:17 AM, ilya  wrote:
>> 
>>> I'm having difficulty getting ROOT and INTERMEDIATE certificates to show
>>> up in SSVM java keystore.
>>>
>>>
>>> I've followed the procedure on
>>>
>>> http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/systemvm.html?highlight=pkcs
>>>
>>> and
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replace+realhostip.com+with+Your+Own+Domain+Name
>>>
>>> But after restart of SSVM and MS - the keystore still has default Go
>>> Daddy certs.
>>>
>>> Would any know how to troubleshoot it?
>>>
>>> Also, one thing to note, i'm not uploading the actual wild card cert -
>>> is its against security policy. It will be impossible for me to get a
>>> wildcard cert.
>>>
>>> Regards
>>> ilya
>>>
>> 
>>

[GitHub] cloudstack pull request: Cloudstack 8611 Handle SSH if server "for...

2016-04-01 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1459#issuecomment-204303829
  
@GabrielBrascher what shows is that unchanged line are changed. This could 
mean that you are using different line endings. Are you working on a windows 
machine?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-01 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1458#issuecomment-204301035
  
tools/marvin/marvin/config/Baremtal_AdvZone_Vmware.cfg does not have a 
license. @cloudsadhu please add it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 8611 Handle SSH if server "for...

Github user GabrielBrascher commented on the pull request:

https://github.com/apache/cloudstack/pull/1459#issuecomment-204293184
  
Not sure why it shows as if I had changed all the SshHelper class (the  
diff shows  removed lines that I have not changed). I will check on that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 8611 Handle SSH if server "for...

GitHub user GabrielBrascher opened a pull request:

https://github.com/apache/cloudstack/pull/1459

Cloudstack 8611 Handle SSH if server "forget" to send exit status

 Continuing the work started by @likitha, I cherry-picked the 
commit (b9181c689e0e7b5f1e28c81d73710196dfabd0ba) from PR 
. 

I made some changes from the cherry-picked commit adding @wilderrodrigues 
suggestions (create simple methods that could be reused and make unit tests 
feasible).

Also, tried to simplify the logic by assuming that 

if ((conditions & ChannelCondition.EXIT_STATUS) != 0) {
if ((conditions & (ChannelCondition.STDOUT_DATA | 
ChannelCondition.STDERR_DATA)) == 0) {
break;
}
}

... is the same as `((conditions & ChannelCondition.EXIT_STATUS) != 0) && 
((conditions & (ChannelCondition.STDOUT_DATA | ChannelCondition.STDERR_DATA)) 
== 0)`. This expression has the following results according to each possible 
condition.

|Condition|Value|result
|-|---|--|
TIMEOUT  | 001|false 
CLOSED  | 010 |false 
STDERR_DATA | 100 | false 
STDERR_DATA | 0001000 | false
EOF | 001 | false 
EXIT_STATUS | 010 | **true**
EXIT_SIGNAL | 100 | false 

Testing all the possibilities, it is possible to note that the condition of 
(newConditions & ChannelCondition.EXIT_STATUS) != 0)`` is sufficient; thus the 
simplified "if" conditional can be:

`if ((conditions & ChannelCondition.EXIT_STATUS) != 0) {
break;
}`

This proposed work can be explained by quoting @likitha:
>CheckS2SVpnConnectionsCommand execution involves executing a script 
(checkbatchs2svpn.sh) in the virtual router. Once CS has opened a session to a 
virtual router and executed a script in the router, it waits indefinitely till 
the session either times out or the exit status of the remote process is 
available. But it is possible that an EOF is reached by the process in the 
router and the router never set the exit status.

>References -
>1. Some servers never send the exit status, or occasionally "forget" to do 
so 
(http://grepcode.com/file/repo1.maven.org/maven2/org.jvnet.hudson/trilead-ssh2/build212-hudson-1/com/trilead/ssh2/ChannelCondition.java).
>2. Get the exit code/status from the remote command - if available. Be 
careful - not all server implementations return this value - 
(http://grepcode.com/file/repo1.maven.org/maven2/org.jvnet.hudson/trilead-ssh2/build212-hudson-1/com/trilead/ssh2/Session.java#Session.waitForCondition%28int%2Clong%29).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/GabrielBrascher/cloudstack CLOUDSTACK-8611

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1459.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1459


commit 1409f981f298bfe60a1791222ca0e62276659cad
Author: Likitha Shetty 
Date:   2015-05-01T11:14:51Z

CLOUDSTACK-8611. CS waits indefinitely for CheckS2SVpnConnectionsCommand to 
return.
While remote executing commands through ssh, handle channel condition of 
EOF because we wait for the the condition.

commit 96d76d41ab38ab5459e3b3c685633fb8f41cf609
Author: gabrascher 
Date:   2016-04-01T06:21:16Z

Handle SSH if server "forget" to send exit status.

Cherry-picked the work started by https://github.com/likitha
commit (b9181c689e0e7b5f1e28c81d73710196dfabd0ba) from PR
https://github.com/apache/cloudstack/pull/561.

CS waits indefinitely for CheckS2SVpnConnectionsComm and to return.
While remote executing commands through ssh, handle channel condition of
EOF because we wait for the the condition.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Permissions issues on cwiki with my login credentials

2016-04-01 Thread Daan Hoogland

On Fri, Apr 1, 2016 at 7:50 AM, Shweta Agarwal <
shweta.agarw...@accelerite.com> wrote:

> shwetaag


is added



-- 
Daan

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...