[GitHub] cloudstack pull request #1605: CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Im...

[sateesh-chodapuneedi]

Github user sateesh-chodapuneedi commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1605#discussion_r69677623
  
--- Diff: 
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 ---
@@ -2015,19 +2016,20 @@ protected StartAnswer execute(StartCommand cmd) {
 }
 
 /**
- * Sets video card memory to the one provided in detail svga.vramSize 
(if provided).
+ * Sets video card memory to the one provided in detail svga.vramSize 
(if provided) on {@code vmConfigSpec}.
  * 64MB was always set before.
  * Size must be in KB.
  * @param vmMo virtual machine mo
  * @param vmSpec virtual machine specs
+ * @param vmConfigSpec virtual machine config spec
  */
-protected void postVideoCardMemoryConfigBeforeStart(VirtualMachineMO 
vmMo, VirtualMachineTO vmSpec) {
+protected void videoCardMemoryConfig(VirtualMachineMO vmMo, 
VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
--- End diff --

Although the method name is apt for current scenario, i.e. configuring 
video RAM, this method could be used for different kinds of Video card 
configurations. Probably it is better to rename this to videoCardConfig or 
videoCardReconfig or so.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1605: CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Im...

[sateesh-chodapuneedi]

Github user sateesh-chodapuneedi commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1605#discussion_r69677503
  
--- Diff: 
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 ---
@@ -2015,19 +2016,20 @@ protected StartAnswer execute(StartCommand cmd) {
 }
 
 /**
- * Sets video card memory to the one provided in detail svga.vramSize 
(if provided).
+ * Sets video card memory to the one provided in detail svga.vramSize 
(if provided) on {@code vmConfigSpec}.
  * 64MB was always set before.
  * Size must be in KB.
  * @param vmMo virtual machine mo
  * @param vmSpec virtual machine specs
+ * @param vmConfigSpec virtual machine config spec
  */
-protected void postVideoCardMemoryConfigBeforeStart(VirtualMachineMO 
vmMo, VirtualMachineTO vmSpec) {
+protected void videoCardMemoryConfig(VirtualMachineMO vmMo, 
VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
 String paramVRamSize = "svga.vramSize";
--- End diff --

👍 Thanks @nvazquez for this PR. It indeed reduces additional API calls.
I'd suggest few minor changes though.
Please move this string constant "svga.vramSize" to VmDetailConstants which 
is place holder for VM detail constants.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1605: CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Improve p...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1605
  
In general, a good idea. I've not tested it though. @nvazquez can you 
re-kick Jenkins by pushing -f on the PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Roadmap for 4.x and 5.0

[ilya]

Marc

You are correct that my shell script is not most robust - it should  be
re-written in java - and called upon on "graceful" shutdown - this
script should be treated as POC i guess.

What it guards against - is more than just snapshots though. Basically -
any async operation that would be harmful to end user experience if i
was to take down one of the MS servers.

I front my MS servers with a VIP, as i take down one of the MS servers
gracefully via script below, the agents all reconnect to next MS.



The current "Cold Cross Cluster" migration as it stands is done by
copying the data disk to secondary and then back to primary. If you have
a VMs with 4TB data disks - thats not feasible for several reasons (1
NFS export for SSVM may not be as large, its pretty slow to copy to NFS
and back to Primary - even if you have a robust network). Hence direct
migration bypassing the secondary store would be far more efficient.

In regards to secure KVM migration, each migrate call, establishes a
one-time SSH key pair between 2 KVM host that will be used only for the
duration of that migration. It is cleared once the operation completes
and avoids a possibility of someone exploiting the cloud user ssh keys.

This is not a big deal to Cloud Hosting companies - but is a big deal to
enterprise security folks who run cloudstack as private cloud. We don't
want cloud user keys littered everywhere - not very ideal in terms of
security.

Regards
ilya



On 7/3/16 10:41 PM, ma...@exoscale.ch wrote:
> Hi Ilya,
> 
> Regarding the live migration, we are using it in production and did migrate a 
> couple of VMs until we reach some corner cases, for which I wrote a few 
> fixes. We'll verify them during the following weeks. The code is based on CS 
> 4.4 but I started porting it to master. I have to finish that and merge the 
> fixes too. For the cold migration, it's already in CS and we are usign it 
> since a while.
> What do you mean by secure KVM migration? My code reads configuration values 
> for which you can have TLS peer-2-peer connection between the agents to 
> transfert over it all the data using the features in libvirt. That the setup 
> we have in production.
> 
> For the graceful shutdown, we have a HA proxy in front so we just edit the 
> configuration to turn off one MS. We are also checking manually if there 
> aren't any snapshot ongoing before launching the stop-start. But I don't find 
> this very robust. Therefore I read a lot of the code managing the agent and 
> how the agents are connected to the MS. There is already a command to 
> rebalance agents between MS, so I'm developping a solution around that.
> 
> Kind regards,
> Marc-Aurèle
> 
> 
>> On 02 Jul 2016, at 02:03, ilya  wrote:
>>
>> Marco,
>>
>> I written a tiny shell script that does following:
>>
>> Make's sure there are async_jobs that arent running, also block 8080 via
>> iptables - to avoid user connecting to MS thats about to go down.
>>
>> It needs a bit of enhancement - and should lookup the MSID of that
>> specific server, it looks something like this - consider borrowing
>> concepts if applicable..
>>
>>> #!/bin/bash
>>> DATESTAMP=$(date +%m%d%y-%H%M%S)
>>> DBPASS=$(java -classpath /usr/share/cloudstack-common/lib/jasypt-1.9.0.jar 
>>> org.jasypt.intf.cli.JasyptPBEStringDecryptionCLI input="$(cat 
>>> /etc/cloudstack/management/db.properties | grep db.cloud.password | awk 
>>> -F'(' '{print $2}' | sed 's/)//g')" password="$(cat 
>>> /etc/cloudstack/management/key)" | grep -A2 OUTPUT | tail -1)
>>> DBHOST=$(cat /etc/cloudstack/management/db.properties | grep db.cloud.host 
>>> | awk -F'=' '{print $2}' | tail -1 )
>>> DBUSER=$(cat /etc/cloudstack/management/db.properties | grep 
>>> db.cloud.username | awk -F'=' '{print $2}')
>>> DB=$(cat /etc/cloudstack/management/db.properties | grep db.cloud.name | 
>>> awk -F'=' '{print $2}')
>>> DBPORT=$(cat /etc/cloudstack/management/db.properties | grep db.cloud.port 
>>> | awk -F'=' '{print $2}')
>>> MYSQLCMD="mysql -h $DBHOST -u $DBUSER -P $DBPORT -p$DBPASS $DB"
>>> #echo $DBPASS $DBHOST $DBUSER $DB $DBPORT
>>>
>>>
>>> JOBS=$(echo 'SELECT * FROM cloud.async_job where job_status=0 and 
>>> job_dispatcher not like "pseudoJobDispatcher"' | $MYSQLCMD | wc -l)
>>>
>>> if [ $JOBS -gt 0 ]
>>>then
>>>echo "WARN: Looks like i have active jobs in flight, please 
>>> try again later"
>>>echo 'SELECT * FROM cloud.async_job where job_status=0 and 
>>> job_dispatcher not like "pseudoJobDispatcher"' | $MYSQLCMD
>>>exit
>>>else
>>>echo "NOTE: No jobs running, good to go!"
>>>echo "NOTE: Blocking incoming 8080"
>>>/sbin/iptables -A INPUT -p tcp --destination-port 8080 -j 
>>> DROP
>>>service cloudstack-management stop
>>>service cloudstack-management stop:wq
>>>CSPID=$(cat /var/run/cloudstack-management.pid )
>>>

[GitHub] cloudstack pull request #1605: CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Im...

[nvazquez]

GitHub user nvazquez opened a pull request:

https://github.com/apache/cloudstack/pull/1605

CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Improve performance

JIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9428

### Introduction

On #1310 passing vRAM size to support 3D GPU problem was addressed on 
VMware. It was found out that it could be improved to increase performance by 
reducing extra API calls, as we'll describe later

### Improvement
On WMware, `VmwareResource` manages execution of `StartCommand.` Before 
sending power on command to ESXi hypervisor, vm is configured by calling 
`reconfigVMTask` web method on vSphere's client `VimPortType` web service.
It was found out that we were using this method 2 times when passing vRAM 
size, as it implied creating a new vm config spec only editing video card specs 
and making an extra call to `reconfigVMTask.`

We propose reducing the extra web service call by adjusting vm's config 
spec. This way video card gets properly configured (when passing vRAM size) in 
the same configure call, increasing performance.

### Use case (passing vRAM size)
* Deploy a new VM, let its id be X
* Stop VM
* Execute SQL, where X is vm's id and Z is vRAM size (in kB): 

INSERT INTO cloud.user_vm_details (vm_id, name, value) VALUES (X, 
'mks.enable3d', 'true');
INSERT INTO cloud.user_vm_details (vm_id, name, value) VALUES (X, 
'mks.use3dRenderer', 'automatic');
INSERT INTO cloud.user_vm_details (vm_id, name, value) VALUES (X, 
'svga.autodetect', 'false');
INSERT INTO cloud.user_vm_details (vm_id, name, value) VALUES (X, 
'svga.vramSize', Z);

* Start VM

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nvazquez/cloudstack fixVram

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1605.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1605


commit 99436e0ed63286ea537069c2c58c1a83e2ce25ea
Author: nvazquez 
Date:   2016-07-05T22:21:26Z

CLOUDSTACK-9428: Fix for CLOUDSTACK-9211 - Improve performance




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1604: improvement: systemvm slow operations

[romain-dartigues]

Github user romain-dartigues commented on the issue:

https://github.com/apache/cloudstack/pull/1604
  
@syed

> there could be a possibility of having an ipv6 gateway

Right, but IPv6 does not use ARP but 
[NDP](https://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol) ([RFC 
4861](https://tools.ietf.org/html/rfc4861)).

> do we know why the arping is failing?

Apparently because the file "/etc/cloudstack/ips.json" is generated with a 
gateway at "None" even when the gateway is present in the input "cmd_line.json" 
(but my memory could fail me at this time of the day).

One month ago we found the "problem" and wanted a fix; trying to understand 
*why* it did happen made me start #1575 (now #1603).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [PROPOSAL] 2016-2017 Release Cycle and Calendar

[John Burwell]

Wido,

I don’t think that Rohit is advocating expanding the 4.9.0 scope.  Instead, he 
is saying that proceeding with LTS on 4.8.0.1 would be counterproductive.  
4.8.0 was released in January meaning that 4.9 will contain nearly 6 months of 
defect fixes and enhancements.  Had we cut LTS last Friday, we would have faced 
the choice of attempting to backport all of the defect changes from master to 
LTS or simply not including them.  Therefore, I think we all agree that we 
should release 4.9.0.0 as soon as possible.  My hope is that we can complete 
testing and RC voting by 15 July 2016 in order to cut LTS and move forward with 
the development of 4.10.0.0.  Am I correct in my understanding Rohit and your 
thoughts regarding 4.9?

Thanks,
-John

> 
john.burw...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
  
 

On Jul 5, 2016, at 5:25 AM, Wido den Hollander  wrote:
> 
>> 
>> Op 4 juli 2016 om 18:17 schreef Rohit Yadav :
>> 
>> 
>> I think 4.9 brings some new features, enhancements and improvements that 
>> people would appreciate in a LTS release. Many of them cannot be simply 
>> backported and 2 weeks will be far shorter to identify, backport and 
>> regression-test all those changes.
>> 
> 
> But that way we keep finding reasons to delay 4.9 and make that the LTS. 4.10 
> afterwards will have other cool features, etc. That will stay.
> 
>> 
>> From my personal experience, I've done a lot of backporting in 4.3, then did 
>> maintenance work in 4.5. I think it's still worth it to wait for few weeks 
>> to get 4.9.0 out of the door and have LTS based on 4.9.0 release.
>> 
>> 
>> If Will is not able to find time to work on 4.9 release, maybe someone else 
>> can takeover and help with the 4.9 release by 15th July (at least get some 
>> RCs up for testing and voting)? LTS can be 4.9.1 etc.
>> 
>> 
>> Regards.
>> 
>> 
>> From: Wido den Hollander 
>> Sent: 04 July 2016 15:32:45
>> To: dev@cloudstack.apache.org; John Burwell
>> Subject: Re: [PROPOSAL] 2016-2017 Release Cycle and Calendar
>> 
>> 
>>> Op 4 juli 2016 om 6:57 schreef John Burwell :
>>> 
>>> 
>>> All,
>>> 
>>> Following up on a our discussions about LTS [1], post 4.9 releases [2], and 
>>> 5.0.0/6.0.0 [3], I have assembled a release plan for the next 18-20 months 
>>> in the wiki [4].  My original proposal assumed by that 4.9 would be 
>>> released by 1 July.  I have adjusted the timelines assuming that we will 
>>> complete the release by 15 July.  My thinking is that the number of defect 
>>> fixes present in 4.9 and 4.8.1 would require more than 2 weeks of work to 
>>> backport to a 4.8-based LTS.  Therefore, I have shifted the LTS cutoff 
>>> dates from 1 Jan and 1 July to 15 Jan and 15 July.
>>> 
>>> Once we gain consensus on the release cycle and calendar, I will cleanup 
>>> the wiki to move completed release plans to an archive area and remove 
>>> obsolete release process documentation.
>>> 
>> 
>> Looks good to me. It's time that 4.9 is released and shifting to July 15th 
>> looks good.
>> 
>> Overall your release schedule proposal looks good to me, so I'd say +1 on 
>> the proposal.
>> 
>> Wido
>> 
>>> Thanks,
>>> -John
>>> 
>>> [1]: http://markmail.org/thread/zzrjldiryhy3ygfk
>>> [2]: http://markmail.org/thread/7rv3jwz4hzf3yplf
>>> [3]: http://markmail.org/thread/c24hvshz3z4babjb
>>> [4]: 
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+Release+Cycle+and+Calendar
>>> 
>>> john.burw...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>>> @shapeblue
>>> 
>>> 
>>> 
>> 
>> rohit.ya...@shapeblue.com 
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue

[GitHub] cloudstack issue #1600: Support Backup of Snapshots for Managed Storage

[syed]

Github user syed commented on the issue:

https://github.com/apache/cloudstack/pull/1600
  
@mike-tutkowski I've fixed the problem where the volume was not being 
detached after the copy. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1602: CLOUDSTACK-9422: Granular 'vmware.create.full.clone'...

[nvazquez]

Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1602
  
Thanks @syed for your comments!
I agree with @serg38 that extension to other hypervisors could be done in a 
separate PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1602: CLOUDSTACK-9422: Granular 'vmware.create.full.clone'...

[serg38]

Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1602
  
@syed. Absolutely it can although the effort seems to be well beyond and 
scope of this PR. @nvazquez and I have only access to vmware hypervisors at 
this point. If there is an interest to expand it let's open another PR focusing 
just on that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1603: Vrouter fixes pylint

[The-Loeki]

Github user The-Loeki commented on the issue:

https://github.com/apache/cloudstack/pull/1603
  
Hi :) sorry I was unable to process your request to merge your PR with 
mine, as you can probably tell by now this would've oversized & muddied both 
PR's ;)
Depending on the uptake of the CS team here I'm quite willing to help you 
with it, because as I mentioned IMHO this should've been done from the start on.

So the `.format()` syntax is less compatible as you mention. It is however 
only incompatible with long-deprecated Python versions and CS SysVM's are 
compatible so that's not really an issue.
While of course it is a matter of personal taste, the official message from 
Python pplz however is that `.format()` is intended to replace the `%s` 
formatting: https://docs.python.org/2/library/stdtypes.html#str.format , which 
is why I and the other OSS projects I work on prefer it.

The logging one is an excellent remark though, wading through the docs I 
notice that they've added the `.format()` syntax as per 3.2: 
https://docs.python.org/3/howto/logging-cookbook.html#use-of-alternative-formatting-styles
  

I'm not sure from the top of my head which version the SysVM's are using, 
but if it's not Py3 it oughta be (here comes the next big-ass PR ;) ) 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1586: CLOUDSTACK-9410 Data Disk shown as detached in XS

[syed]

Github user syed commented on the issue:

https://github.com/apache/cloudstack/pull/1586
  
agree with @yvsubhash. I don't see an need to set the name-label to 
`detached` the code LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1602: CLOUDSTACK-9422: Granular 'vmware.create.full.clone'...

[syed]

Github user syed commented on the issue:

https://github.com/apache/cloudstack/pull/1602
  
Thanks @serg38 my question is more along the lines of can this `fullClone`/ 
`linkedClone` functionality be extended to other hypervisors, in that case, we 
can make the code hypervisor agnostic. If not then I am happy with the current 
implementation as well. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1604: improvement: systemvm slow operations

[syed]

Github user syed commented on the issue:

https://github.com/apache/cloudstack/pull/1604
  
@romain-dartigues  do we know why the `arping` is failing? Ideally, if the 
vif is configured correctly, the arping should go through properly. If the 
`arping` fails, do we still know that the connectivity is correct/establised? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1604: improvement: systemvm slow operations

[syed]

Github user syed commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1604#discussion_r69583623
  
--- Diff: systemvm/patches/debian/config/opt/cloud/bin/merge.py ---
@@ -228,7 +228,7 @@ def processCLItem(self, num, nw_type):
 if('localgw' in self.qFile.data['cmd_line']):
 dp['gateway'] = self.qFile.data['cmd_line']['localgw']
 else:
-dp['gateway'] = 'None'
+dp['gateway'] = self.qFile.data['cmd_line'].get('gateway', 
'None')
--- End diff --

there could be a possibility of having an ipv6 gateway, in that case, we 
would be looking at `ip6gateway`. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1603: Vrouter fixes pylint

[romain-dartigues]

Github user romain-dartigues commented on the issue:

https://github.com/apache/cloudstack/pull/1603
  
Hi @The-Loeki, and thank you for your prompt comment!

For your side notes:

* this is mostly a preference, I don't care what you use but here are my 
argument for the `%s`:

  + more portable than the `.format()` (Python 2.4 → Python 3)
  + coherent with `logging` methods; ie: `logging.debug('foo {}', 'bar')` = 
`TypeError`, the correct syntax is `logging.debug('foo %s', 'bar')`

* I also prefer single quotes, especially because double quotes are never 
"nessary" (in Python single and double are equivalent)

I didn't want to change "too much" (well, I think my patch is already too 
much; we should focus on documentation after that); if I did replace all double 
quote with single, for example, I would break `git blame` way too much for my 
taste.

But I'm totally willing to put some more work for conformity, if there is a 
will for it upstream ;^)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1603: Vrouter fixes pylint

[The-Loeki]

Github user The-Loeki commented on the issue:

https://github.com/apache/cloudstack/pull/1603
  
kudos for all the hard work! Most of this should've been done a long time 
ago as `pylint` is one of the first firewalls of code quality ensurance. 

Side notes:
* no fan of `%s` syntax; recommend further refactoring to employ 
`.format()` which should make the code a lot more elegant.
* Strings oughta be single-quoted unless double quotes are necessary

e.g.:
```
-self.fw.append(
-["nat", "",
- "-I PREROUTING -i ppp+ -m tcp --dport 53 -j DNAT --to-destination 
%s" %
- local_ip
- ]
-)
+self.fw.append([
+'nat', '',
+'-I PREROUTING -i ppp+ -m tcp --dport 53 -j DNAT --to-destination 
{}'.format(
+local_ip
+)])
```



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1604: improvement: systemvm slow operations

[romain-dartigues]

GitHub user romain-dartigues opened a pull request:

https://github.com/apache/cloudstack/pull/1604

improvement: systemvm slow operations

On our setup we had issues with vrouters when a large number of VM were 
created at once.

The vrouters try to arping the gateway (to refresh the ARP cache I 
suppose), but in some case it take too much time and the vrouters got killed 
for being unresponsive.

The following patches attempt to solve this by:

* a96ab18f, 8908371b defensive fix, maybe a bug?
* fce10c55 do not block on arping command

This patch has been running for a month on our test environment and seems 
to solve the initial problem.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/romain-dartigues/cloudstack 
systemvm-none-gateway

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1604.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1604


commit a96ab18fbed6f3411c0cd6062a6fbd799e3f4198
Author: Romain Dartigues 
Date:   2016-05-31T08:21:11Z

systemvm: ips.json without gateway

The file "/etc/cloudstack/ips.json" is generated with a gateway at
"None" even when the gateway is present in the input "cmd_line.json".

commit 8908371b213cb50f628b64637fedfda97201d33f
Author: Romain Dartigues 
Date:   2016-05-31T08:34:35Z

systemvm: do not arping None

If the gateway is None, don't bother arping it.

commit fce10c55a0d482751e4736663d32aae00733b913
Author: Romain Dartigues 
Date:   2016-05-31T09:02:09Z

systemvm: arping in background

Do not wait for :man:`arping(8)` to complete to continue.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1599: Independent marvin

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1599
  
@wido yes, with this PR we can split marvin to its own repository. As an 
example to show it's doable, here is a repository I extracted from cloudstack 
master -- https://github.com/rhtyd/marvin


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1599: Independent marvin

[wido]

Github user wido commented on the issue:

https://github.com/apache/cloudstack/pull/1599
  
This is just there to get the ground work done for Marvin to it's own repo?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [PROPOSAL] 2016-2017 Release Cycle and Calendar

[Wido den Hollander]

> Op 4 juli 2016 om 18:17 schreef Rohit Yadav :
> 
> 
> I think 4.9 brings some new features, enhancements and improvements that 
> people would appreciate in a LTS release. Many of them cannot be simply 
> backported and 2 weeks will be far shorter to identify, backport and 
> regression-test all those changes.
> 

But that way we keep finding reasons to delay 4.9 and make that the LTS. 4.10 
afterwards will have other cool features, etc. That will stay.

> 
> From my personal experience, I've done a lot of backporting in 4.3, then did 
> maintenance work in 4.5. I think it's still worth it to wait for few weeks to 
> get 4.9.0 out of the door and have LTS based on 4.9.0 release.
> 
> 
> If Will is not able to find time to work on 4.9 release, maybe someone else 
> can takeover and help with the 4.9 release by 15th July (at least get some 
> RCs up for testing and voting)? LTS can be 4.9.1 etc.
> 
> 
> Regards.
> 
> 
> From: Wido den Hollander 
> Sent: 04 July 2016 15:32:45
> To: dev@cloudstack.apache.org; John Burwell
> Subject: Re: [PROPOSAL] 2016-2017 Release Cycle and Calendar
> 
> 
> > Op 4 juli 2016 om 6:57 schreef John Burwell :
> >
> >
> > All,
> >
> > Following up on a our discussions about LTS [1], post 4.9 releases [2], and 
> > 5.0.0/6.0.0 [3], I have assembled a release plan for the next 18-20 months 
> > in the wiki [4].  My original proposal assumed by that 4.9 would be 
> > released by 1 July.  I have adjusted the timelines assuming that we will 
> > complete the release by 15 July.  My thinking is that the number of defect 
> > fixes present in 4.9 and 4.8.1 would require more than 2 weeks of work to 
> > backport to a 4.8-based LTS.  Therefore, I have shifted the LTS cutoff 
> > dates from 1 Jan and 1 July to 15 Jan and 15 July.
> >
> > Once we gain consensus on the release cycle and calendar, I will cleanup 
> > the wiki to move completed release plans to an archive area and remove 
> > obsolete release process documentation.
> >
> 
> Looks good to me. It's time that 4.9 is released and shifting to July 15th 
> looks good.
> 
> Overall your release schedule proposal looks good to me, so I'd say +1 on the 
> proposal.
> 
> Wido
> 
> > Thanks,
> > -John
> >
> > [1]: http://markmail.org/thread/zzrjldiryhy3ygfk
> > [2]: http://markmail.org/thread/7rv3jwz4hzf3yplf
> > [3]: http://markmail.org/thread/c24hvshz3z4babjb
> > [4]: 
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+Release+Cycle+and+Calendar
> >
> > john.burw...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> 
> rohit.ya...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>   
>  
>

[GitHub] cloudstack pull request #1575: systemvm - Python code cleanup

[romain-dartigues]

Github user romain-dartigues closed the pull request at:

https://github.com/apache/cloudstack/pull/1575


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1575: systemvm - Python code cleanup

[romain-dartigues]

Github user romain-dartigues commented on the issue:

https://github.com/apache/cloudstack/pull/1575
  
As suggested by @wido, rebase + new pull request in #1603.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1603: Vrouter fixes pylint

[romain-dartigues]

GitHub user romain-dartigues opened a pull request:

https://github.com/apache/cloudstack/pull/1603

Vrouter fixes pylint

In my dreams, all Python code pass through pylint and must validate before 
a merge.

This merge request is becoming quite big, maybe too much; I'd like to have 
the owners opinion.

It follow the previous pull requests:
* #1575
  + PCextreme/cloudstack/pull/2
- #1547


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/romain-dartigues/cloudstack 
vrouter-fixes-pylint

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1603.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1603


commit ffee6146e28013676b5eca200492324c1022682b
Author: Romain Dartigues 
Date:   2016-06-01T11:58:07Z

systemvm: make pylint happy

Mostly removing unused modules, but also defined some missing variables.

 40  'X' imported but unused
  9  'from X import *' used; unable to detect undefined names
  9  local variable 'X' is assigned to but never used
  1  redefinition of unused 'X' from line Y
  1  undefined name 'X'

commit 77a53d06107822fd69b940a0769d00cbbeef81b2
Author: Romain Dartigues 
Date:   2016-07-04T12:00:06Z

rebase in progress; onto 1f9bf93

You are currently rebasing branch 'vrouter-fixes-pylint' on '1f9bf93'

commit 8c6372c1c21062bfbd876dc751a94108e49c2e20
Author: Romain Dartigues 
Date:   2016-06-02T06:52:00Z

systemvm: pylint conformance work in progress

Few more code conformity (long lines and such).

Rebased after merge e4ba640.

Your code has been rated at 7.41/10 (previous run: 5.04/10, +2.37)

Raw metrics
---

+--+---+--+-+---+
|type  |number |% |previous |difference |
+==+===+==+=+===+
|code  |4179   |62.09 |4161 |+18.00 |
+--+---+--+-+---+
|docstring |385|5.72  |367  |+18.00 |
+--+---+--+-+---+
|comment   |1122   |16.67 |1087 |+35.00 |
+--+---+--+-+---+
|empty |1045   |15.53 |1042 |+3.00  |
+--+---+--+-+---+

Messages by category


+---+---+-+---+
|type   |number |previous |difference |
+===+===+=+===+
|convention |866|1095 |-229.00|
+---+---+-+---+
|refactor   |34 |55   |-21.00 |
+---+---+-+---+
|warning|76 |699  |-623.00|
+---+---+-+---+
|error  |3  |13   |-10.00 |
+---+---+-+---+

commit 38a36b5da0bcda58d670a135040739b52569a827
Author: Romain Dartigues 
Date:   2016-07-05T06:40:41Z

refactor monitorServices.py, lots of changes

Could be simplified by using the stdlib modules logging and json.

Messages by category


+++--++
| type   | number | previous | difference |
+++==++
| convention |  5 |  115 |-110.00 |
+++--++
| refactor   |  1 |4 |  -3.00 |
+++--++
| warning|  6 |   39 | -33.00 |
+++--++
| error  |  0 |0 |=   |
+++--++

Global evaluation
-
Your code has been rated at 9.40/10 (previous run: 3.01/10, +6.39)

commit c558c767e5b400a14fbdaca5c345abc48372adec
Author: Romain Dartigues 
Date:   2016-07-05T06:42:19Z

systemvm: pylint conformance work in progress

Raw metrics
---

+--+---+--+-+---+
|type  |number |% |previous |difference |
+==+===+==+=+===+
|code  |  4426 |62.50 |4179 |   +247.00 |
+--+---+--+-+---+
|docstring |   496 | 7.00 | 385 |   +111.00 |
+--+---+--+-+---+
|comment   |  1130 |15.96 |1122 | +8.00 |
+--+---+--+-+---+
|empty |  1030 |14.54 |1045 |-15.00 |