date:20160425

[GitHub] cloudstack pull request: 4.9 mvn version safeupgradeonly

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1510#issuecomment-214620580
  
@swill thanks, this can be merged first as we wait for bountycastle related 
changes to work with openjdk 1.7.0_95+


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: engine/schema: fix upgrade path to work w...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1517#issuecomment-214617500
  
@jburwell I've not tested, but if it is using/enforcing SQL99 it should 
fail too (or it could be mysql 5.7.4+ issue);

"As of MySQL 5.7.4, the IGNORE clause for ALTER TABLE is removed and its 
use produces an error."
from http://dev.mysql.com/doc/refman/5.7/en/alter-table.html




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: engine/schema: fix upgrade path to work w...

2016-04-25 Thread jburwell

Github user jburwell commented on the pull request:

https://github.com/apache/cloudstack/pull/1517#issuecomment-214574233
  
Do these instructions apply to MariaDB as well?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: CaaS in CloudStack with Kubernetes

2016-04-25 Thread Marco Sinhoreli

Congrats Sebastien! I am very excited to see it running!



Marco Sinhoreli
Consultant Manager
marco.sinhor...@shapeblue.com
mobile: +55 21 98276 3636
 
Praia de Botafogo 501, bloco 1 - sala 101 – Botafogo
Rio de Janeiro, RJ - Brazil - CEP 22250-040
office: + 55 21 2586 6390 | fax: +55 21 2586 6002
http://www.shapeblue.com/ | twitter: @shapeblue










Em [DATE], "[NAME]" <[ADDRESS]> escreveu:

>Hi folks,
>
>I have been quite silent this past few weeks because aside from voting in a 
>new VP (soon to be announced), I  have been hard at work for my new endeavors 
>with Skippbox.
>
>I am quite excited that today we could not wait and Skippbox together with 
>Shapeblue announced a preview of a Kubernetes/CloudStack integration:
>http://www.benzinga.com/pressreleases/16/04/g7872516/shapeblue-harness-the-power-of-kubernetes-and-apache-cloudstack-to-give
>
>This is currently still a by request preview, but once we are done with much 
>better testing and even tighter integration, the code will be open sourced as 
>a CloudStack plugin.
>
>I wouldn’t mind couple supportive tweets and if you want a demo before you can 
>run it yourself, ping me.
>
>Cheers,
>
>-Sebastien

[GitHub] cloudstack pull request: CLOUDSTACK-9323: Fix cancel host maintena...

2016-04-25 Thread jburwell

Github user jburwell commented on the pull request:

https://github.com/apache/cloudstack/pull/1454#issuecomment-214542430
  
LGTM for code


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-214538838
  
Running tests again in the same setup to see if any of the connection 
issues are resolved.  The error for sure should be looked into...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-214537887
  


### CI RESULTS

```
Tests Run: 29
  Skipped: 10
   Failed: 3
   Errors: 1
```

**Summary of the problem(s):**
```
ERROR: Test restart network with cleanup
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2848, in test_restart_network_with_cleanup
self.validateResult(result)
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2742, in validateResult
if m.group(0) == ip and re.search("secondary", line):
AttributeError: 'NoneType' object has no attribute 'group'
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_JX5MBT/results.txt
```

```
FAIL: Test network rules after starting a VpcVr that
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2465, in test_02_start_vm_network_gc
"Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_JX5MBT/results.txt
```

```
FAIL: Test Stop all the Vms that are part of the a Network
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2557, in test_03_restart_vpcvr
(self.public_ip_1.ipaddress.ipaddress, e))
AssertionError: Failed to SSH into VM - 192.168.23.54, SSH connection has 
Failed. Waited 600s. Error is SSH Connection Failed
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_JX5MBT/results.txt
```

```
FAIL: Test update Network that is part of a VPC to a network
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
1910, in test_01_network_services_upgrade
"Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_JX5MBT/results.txt
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_22_50_41_1M4DX6:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_22_50_41_1M4DX6/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_22_50_41_1M4DX6/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_22_50_41_1M4DX6/runinfo.txt)

**`/tmp/MarvinLogs/test_vpc_network_JX5MBT:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_JX5MBT/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_JX5MBT/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_JX5MBT/runinfo.txt)


Uploads will be available until `2016-06-25 02:00:00 +0200 CEST`

*Comment created by [`upr comment`](https://github.com/cloudops/upr).*



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Fix Sync of template.properties in Swift

2016-04-25 Thread rafaelweingartner

Github user rafaelweingartner commented on the pull request:

https://github.com/apache/cloudstack/pull/1331#issuecomment-214529726
  
I do not find them (final static) purely cosmetics (we are using too much 
this expression lately).  I understand its use, as I presented the references 
about the GC and static variables. Maybe we could use only the static keyword? 
That is what I am saying. The final keyword would work more like an assurance, 
so no one can change that reference during runtime. But, I believe we do not 
need such guarantees, right?

I could not find anything (specs or guides from Oracle or OpenJDK) saying 
that the "final" keyword would provide such improvements. Maybe you could help 
me find something to clarify my doubts?

About logging from static methods, I do not find that argument appealing. 
Even though static methods may be appealing, they are easier to use and code, 
not needing to integrate into a framework life cycle to wire all of the 
dependencies. I believe that if we have an âutilâ class with static 
methods, then it (the util class) will have a static variable âLoggerâ; 
now, mixing static methods into a singleton;  I see that as an anti-pattern. 
This is very personal, I like to delegate very specific tasks to each class. I 
believe that facilitates the design of the software and the writing of test 
cases (both unit and integration one)

I understand your feelings about Mockc. I had the same some time ago when I 
started working with TDD. At that time I used Easymock. But still, the point is 
that, once we start writing test cases (the unit ones), we get into a moment in 
which we have to write integration tests. That means a method that uses few 
methods that are self-contained and unit tested. Therefore, we do not need to 
test the whole method (we already assured that the units are working), but only 
check if the method is calling the methods (units) it should (checking the 
order), with the parameters that are expected and at the end returning what we 
expect it to return. To facilitate that job we would need to use Mocks, then we 
would not have to prepare complicated set ups to write our tests.

About the overhead problems to the GC, as I said, I am talking about 
singletons, they are created only once in the whole application life cycle. 
Therefore, they would not cause overhead problems with the GC.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

ACS PRs Status - 2016/04/25

2016-04-25 Thread Will Stevens

ACS PRs

   - 826 - master (ready, pending comments)
   - 1515 (1375) - 4.8 (ready, pending squash)
   - 1376 - master (ready, pending 1515 + rebase)
   - 1374 - master (ready, pending 1515 + 1376 + rebase)
   - 1054 - master (grep, pending CI)
   - 1464 - master (pending LGTM)
   - 1436 - master (CI posted, needs review)
   - 1417 - master (has CI errors)
   - 1493 - master (on cs2, pending ALL)
   - 1454 - master (on cs3)
   - 1304 - master (pending LGTM)
   - 1365 - 4.7 (rerun CI)
   - 1489 - master (rerun CI)
   - 1510 (1397) - master (being worked on)
   - 1475 - 4.7 (pending clarification)
   - 1409 - master (pending CI)
   - 1433 - master (pending CI)
   - 1230 - master (pending CI)
   - 1455 - master (*pending CI)
   - 1423 - master + svm (*pending CI)
   - 1502 - master (pending ALL)
   - 1428 - master (pending ALL)
   - 1450 - 4.7 (pending ALL)
   - 1453 - master (pending ALL)
   - 1403 - master (pending ALL)
   - 1331 - 4.7 (pending ALL)
   - 1475 - 4.7 (pending ALL)
   - 1297 - master (pending CI)
   - 1410 - 4.7 (pending ALL)
   - 1483 - 4.7 (pending ALL)
   - 1470 - 4.7 (pending ALL)
   - 1471 - 4.7 (pending ALL)
   - 1472 - 4.7 (pending ALL)
   - 1473 - 4.7 (pending LGTM)
   - 1474 - 4.7 (pending ALL)
   - 1486 - 4.7 (pending ALL)
   - 1483 - 4.7 (pending ALL)
   - 1488 - master (pending ALL)
   - 872 - master + svm (pending CI)
   - 1456 - 4.7 (pending ALL)
   - 1412 - 4.6 (pending ALL)
   - 1406 - 4.6 (pending LGTM)
   - 1378 - 4.6 (pending LGTM)
   - 1491 - 4.7 (pending ALL)
   - 1360 - master (pending LGTM)
   - 1490 - 4.7 (pending ALL)
   - 1499 - master (pending ALL)
   - 1371 - master + svm (pending ALL)
   - 1500 - master (pending ALL)
   - 1497 - master (pending ALL)
   - 883 - master (pending ALL)
   - 956 - master (needs work)
   - 880 - master (pending ALL)
   - 866 - master (pending ALL)
   - 846 - master (pending CI)
   - 1244 - master (needs work)
   - 1514 - 4.7 (pending ALL)
   - 1510 - master (pending ALL)
   - 1511 - master (pending ALL)
   - 1508 - master, 4.8? (pending ALL)
   - 1516 - master (pending ALL)

We are slowly making progress.  We merged 23 PRs in the last 7 days.  We
have a bunch in various stages of completion, so have a look and help
review anything that is pending 'ALL' or 'LGTM'.

I apologize if the status is not up to date on all of them.

More people will be coming online with CI this week, so I am working to try
to get the test suite in a good place so we can have more complete coverage
with our testing.

Thats it for now.

Will

[GitHub] cloudstack pull request: Fix Sync of template.properties in Swift

2016-04-25 Thread jburwell

Github user jburwell commented on the pull request:

https://github.com/apache/cloudstack/pull/1331#issuecomment-214518411
  
@rafaelweingartner yes, as I have said, ``final statics`` are not evaluated 
for GC.  A single case is not a noticeable problem.  However, if made all 
loggers were made instance variables, given the number we have, it would become 
a problem.  You stated that your preference was to make loggers instance 
variables, and that they being ``final static`` was purely cosmetic.  My point 
is that there are both performance and capability motivations to make them 
``final static`` (i.e. the ability to log from static methods and accumulated 
GC overhead).

Personally, I use Mockito (and its ilk) sparingly.  Namely, I only use it 
in circumstances where language constructs are unable to mock a mechanism.  
First, I find it more difficult to understand the intentions of a test using 
Mockito rather than pure Java as it layers another set of abstractions onto the 
test.  Second, Mockito approaches discourage reuse.  We could use Mockito to 
mock logging tests in this case.  However, we have done nothing to make testing 
of expected logging easier for future test cases.

It's most important to me that we do not start turning references to 
constant values into instance variables due to potential GC churn.  It's less a 
memory size issue than it is about the length of GC operations and CPU 
overhead.  The Mockito vs. ``TestAppender`` approach is up to @syed and what he 
feels most comfortable implementing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Fix Sync of template.properties in Swift

2016-04-25 Thread rafaelweingartner

Github user rafaelweingartner commented on the pull request:

https://github.com/apache/cloudstack/pull/1331#issuecomment-214509189
  
@jburwell, @syed, sorry the long post, I did a research on a few thing and 
I would like to share with you guys.

I was just looking at the state of this PR, and I noticed that the number 
of lines added jumped to 300+; most of those lines are needed in order to write 
a test case using the âTestAppenderâ approach. 
Please do not take me in a bad way; I find discussions like this very 
healthy for the future of the project.  I also know that the class 
âTestAppenderâ is there to be reused in some other tests that want to check 
the use of log; but still, it feels pretty complicated to me. Giving that, do 
you see why when I created the test as an example for @Syed, I used the Mocking 
approach? It feels simpler and more natural to write tests using that approach, 
at least to me.

Additionally, I was curious when you mentioned that the use of âstatic 
finalâ delimiters could optimize the GC, given that it (the GC) would not 
check if those attributes have or not to be garbage collected. I had never 
heard that before, so I tried to find something online. If you have some 
reliable reference about that, could you share?

I tried to find some specs or guideline documents from either Oracle or 
OpenJDK without much success.  However, I found something like you said to the 
android JVM [1], but that would not be our case. Then, I read some articles 
(not scientific ones) from IBM [2] and Oracle [3] about the tuning of java 
code. But still, they did not mention anything related to what you said. Then, 
I decided to revisit a forum that I did not visit for a long time, since the 
time of my first Java certification (I am definitely getting old :(), the 
coderanch [4]. There I found something that may be related to what you said 
[5]. There was a discussion there about the GC and static variables; at some 
point, someone highlights that âStatic variables are destroyed when the class 
is unloadedâ.  After that, I also found this [6] on Stack overflow, in which 
it is described that âStatic variables cannot be elected for garbage 
collection while the class is loaded. They can be collected when the respective
  class loader (that was responsible for loading this class) is itself 
collected for garbage.â. I believe that was the point you wanted to make, 
right? Static variables are not GCed by the GC, they are not even checked.
If that was the case, it would have nothing to do with the âfinalâ word 
per se, but rather with the use of the âstaticâ word.

After having said that, perhaps we could benefit from both words? I mean we 
could still use the âLoggerâ variable as static, but not final; then, we 
would be able to write a test case using Mockito (as the first example I 
presented to @Syed), which would add less code.
What do you think about that?

Additionally, I had taken a look at some spring framework code. Their 
framework is not only the base of ACS but also many other huge projects; so, I 
thought it would be interesting to see how they use the logger variables. They 
use their âloggerâ attributes as Object variables and not Classes. With 
that approach when you extend their code, you âget for freeâ a logger 
instance to be used. I believe that is why I am so used to loggers being object 
attributes. I might have been working too much with their components and 
frameworks.

When we do that, we can avoid the following example that happens in ACS:
Letâs take the example of âNfsSecondaryStorageResourceâ class. That 
class is an intermediate class to singletons (LocalNfsSecondaryStorageResource, 
MockLocalNfsSecondaryStorageResource, PremiumSecondaryStorageResource and 
SimulatorSecondaryStorageResource). All of them also have a Logger instance for 
their respective class. Also, the âNfsSecondaryStorageResourceâ extends the 
âServerResourceBaseâ that has a Logger instance too. In total, we have 5 
Logger instances. One for each class, giving that all of them are static 
attributes. If we used an approach similar to the one that is used by Spring-*, 
we would have one âLoggerâ instance for each singleton, which would 
represent 4 logger instances.

I did some tests, and the approximated size of a Logger instance is ~820 
bytes. So, if we save the instantiation of a few of this we can reduce a little 
bit of the use of ACS memory, giving that due to the way we create âLoggerâ 
objects today, we have an instance even for classes that are not object per se, 
but intermediated classes in a hierarchy of singletons.

[1] http://developer.android.com/training/articles/perf-tips.html
[2] http://www.ibm.com/developerworks/library/j-jtp01274/
[3] 
https://docs.oracle.com/cd/E26576_01/doc.312/e24936/tuning-apps.htm#GSPTG00161

[GitHub] cloudstack pull request: Remove unused images

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1475#issuecomment-214505566
  
@remibergsma bump.  :)  just a friendly reminder.  what should we do with 
this PR?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-214503490
  
Would you mind doing a force push to try to get everything green.  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9352: Test fails in Widows as ...

2016-04-25 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1498


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: BUG-ID:CLOUDSTACK-9331:added code in marv...

2016-04-25 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1458


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9322: Support for Internal LB ...

2016-04-25 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1452


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9164: Prevent firefox's quick ...

2016-04-25 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1271


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8745 : verify usage after root...

2016-04-25 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/713


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8302: Removing snapshots on RB...

2016-04-25 Thread dmytro-shevchenko

Github user dmytro-shevchenko commented on the pull request:

https://github.com/apache/cloudstack/pull/1230#issuecomment-214497874
  
Rebase with Master done, pom.xml file updated. 
Also I perform a small modification in code, during testing I found one 
issue: in 'snapshot_store_ref' table all snapshots from one volume was linked 
between each other as Parent->Child using field 'parent_snapshot_id'. If you 
removing one of previous snapshot and wait for 'storage.cleanup.interval' 
period,  it lead to NullPointerException when you creating new snapshot, 
because Cloudstack trying to build all this snapshot relations before. Before 
this patch this field was always set to '0' (no parent). From Cloudstack point 
of view all snapshots on Ceph not connected (Ceph care about this on his own 
level). 
So, in file 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java:
 
I moved this block:
`SnapshotDataStoreVO snapshotDataStoreVO = 
snapshotStoreDao.findByStoreSnapshot(primaryStore.getRole(), 
primaryStore.getId(), snapshot.getId());
if (snapshotDataStoreVO != null) {
snapshotDataStoreVO.setParentSnapshotId(0L);
snapshotStoreDao.update(snapshotDataStoreVO.getId(), 
snapshotDataStoreVO);
}`
from the condition: ...primaryStore).getPoolType() != StoragePoolType.RBD
and it will be executed in any way, as previously. Please review this part 
of code, if this is good solution.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9289:Automation for feature de...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1417#issuecomment-214495424
  


### CI RESULTS

```
Tests Run: 0
  Skipped: 0
   Failed: 0
   Errors: 0
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_21_28_18_JKPEMO:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_21_28_18_JKPEMO/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_21_28_18_JKPEMO/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/DeployDataCenter__Apr_25_2016_21_28_18_JKPEMO/runinfo.txt)

**`/tmp/MarvinLogs/Z09DRB:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/Z09DRB/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/Z09DRB/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1417/tmp/MarvinLogs/Z09DRB/runinfo.txt)


Uploads will be available until `2016-06-25 02:00:00 +0200 CEST`

*Comment created by [`upr comment`](https://github.com/cloudops/upr).*



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9 mvn version safeupgradeonly

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1510#issuecomment-214480409
  
go ahead.  I am trying to get to the bottom of marvin being strange right 
now.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9 mvn version safeupgradeonly

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1510#issuecomment-214480008
  
@swill busy on this one yet? I am looking to run it as well


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: No module named marvin

2016-04-25 Thread Daan Hoogland

I had this some bubbles back. the mysql connector python driver is not the
stablest of packages it seems. I started a new environment in my bubble
last this afternoon (your morning). I will restart one later tonight, just
to see.

On Mon, Apr 25, 2016 at 8:06 PM, Will Stevens 
wrote:

> Could this:
>
> Deploy data center..
> Traceback (most recent call last):
>   File "/data/git/cs3/cloudstack/tools/marvin/marvin/deployDataCenter.py",
> line 28, in 
> from marvin import configGenerator
> ImportError: No module named marvin
>
> Be caused by this:
>
> Installing Marvin
> DEPRECATION: --allow-external has been deprecated and will be removed in
> the future. Due to changes in the repository protocol, it no longer has any
> effect.
> Processing ./tools/marvin/dist/Marvin-4.9.0-SNAPSHOT.tar.gz
> Collecting mysql-connector-python>=1.1.6 (from Marvin===4.9.0-SNAPSHOT)
>   Could not find a version that satisfies the requirement
> mysql-connector-python>=1.1.6 (from Marvin===4.9.0-SNAPSHOT) (from
> versions: )
> No matching distribution found for mysql-connector-python>=1.1.6 (from
> Marvin===4.9.0-SNAPSHOT)
>
> This seems to be happening in all my CI environments this morning.
> Thoughts?
>
> Thanks,
>
> Will
>



-- 
Daan

No module named marvin

2016-04-25 Thread Will Stevens

Could this:

Deploy data center..
Traceback (most recent call last):
  File "/data/git/cs3/cloudstack/tools/marvin/marvin/deployDataCenter.py",
line 28, in 
from marvin import configGenerator
ImportError: No module named marvin

Be caused by this:

Installing Marvin
DEPRECATION: --allow-external has been deprecated and will be removed in
the future. Due to changes in the repository protocol, it no longer has any
effect.
Processing ./tools/marvin/dist/Marvin-4.9.0-SNAPSHOT.tar.gz
Collecting mysql-connector-python>=1.1.6 (from Marvin===4.9.0-SNAPSHOT)
  Could not find a version that satisfies the requirement
mysql-connector-python>=1.1.6 (from Marvin===4.9.0-SNAPSHOT) (from
versions: )
No matching distribution found for mysql-connector-python>=1.1.6 (from
Marvin===4.9.0-SNAPSHOT)

This seems to be happening in all my CI environments this morning.
Thoughts?

Thanks,

Will

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214463216
  
Just finished all reruns of failing tests in the bubble. They all succeeded.


[1511.results.network.txt](https://github.com/apache/cloudstack/files/235077/1511.results.network.txt)

[1511.results.vpc_routers.txt](https://github.com/apache/cloudstack/files/235078/1511.results.vpc_routers.txt)

[1511.results.loadbalance.txt](https://github.com/apache/cloudstack/files/235079/1511.results.loadbalance.txt)

[1511.results.ssvm.txt](https://github.com/apache/cloudstack/files/235080/1511.results.ssvm.txt)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214459138
  
Thank you sir.  :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214458410
  
I found one bug and added a commit to solve it. I will keep at tjis one for 
a while.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214455195
  
Lets get some CI results in here so we can start to figure out what is 
going on.   I believe I had a bunch of problems with this PR before it was 
broken out, so I want to make sure we understand what is going on.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214454225
  
@swill it seems the berfore script is failing sometimes. I don't think this 
is a problem with the PR itself.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: L10n update 4.8 20160422

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1515#issuecomment-214448604
  
Can you squash the commits for me?  I will get this merged and then forward 
merged to master and then we can rebase the other ones to to make sure we don't 
have merge conflicts.  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-21297
  
I will run those failed travis tests in the bubble. I am seeing some 
failures as well so this needs more work


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-214432357
  
@swill done, though as I mentioned in the comments above the test that 
failed in last travis run is a component test and fails on master too (i.e. not 
related to this PR). When Travis runs, the component tests are not run every 
time (it's random, if Travis allows to run additional jobs see tools/travis for 
details).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60944411
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,478 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214430311
  
> @rhtyd My comment regarding the test was more in the context of perf. 
test. In the DB for regular user I saw ~250 permissions got created. So this 
means iterating over all these entries twice (ALLOW and DENY) to find a match 
and then perform access check.

The two checkPermissions calls would not cause significant overhead as they 
are done in memory and cost a maximum (worst case) of O(n) computation (on same 
machine). While making two DB calls (once to get all ALLOW rules and once to 
get all DENY rules) is more expensive as we do two network calls to get the 
data and still hit worst case O(n) computation.

For a dynamic access checking system, this is a trade off and also a 
feature. In case of static checker we've now, rules are loaded at load-time 
only; any change requires restart and rules can be inconsistent across 
server(s).

> There will be a perf. overhead due to this and user should have an option 
to decide whether to use static or dynamic.

Users do have a choice, they can choose to not migrate to this feature. 
Typically, in production organizations would run multiple management servers; 
mgmt servers can be horizontally scaled to meet demanding usage. For example, 
they can can mgmt server on a separate machine (which they generally do) and 
tune their databases to accept up to 50k requests (or qps), optimize innodb 
settings (buffer pool size to 60% of memory etc.) and in db.properties increase 
num. of db connections from 250 to 1000.

> Also if the user finds some issues/bugs later during their testing there 
should be a fallback option.

There is a way to revert back to using static checker as I explained 
earlier, (1) switch off the global settings and (2) put back a 
commands.properties file in class-path usually at /etc/cloudstack/management; 
it's just that we don't want users to do it easily. Consider this an admin 
creates a read-only admin role and accounts with that (such users can only do 
list* calls), now if they go back to static-checker all such users now become 
default root admin (based on account type, translation) and can call all APIs 
defined in commands.properties -- this is a significant security risk. 
Therefore, I personally don't want to put users at risk and just discourage 
them using the static checker.

> Regarding upgrade implications, I went through the docs/FS but some 
things are still confusing. If existing user can continue using 
commands.properties then what happens to the new APIs that gets added.

In case you're not aware, with the current system each time a user upgrade 
they have to edit/add new rules to commands.properties by hand. Upgrading using 
packages does not update commands.properties file; it would create a 
.rpmnew/rpmsave file for example. In case of multiple mgmt server, you have to 
do this on all server(s) and restart all of them. While in case of dynamic 
roles based checker, you don't need to restart mgmt server at all (even during 
migration). I'm not sure why you say the static checker way is flexible, on the 
contrary I think it is not and a pain point of a lot of people.

> If the argument is that the permission can be put in as an annotation in 
code for new APIs then that removes the flexibility of the earlier mechanism 
(there is no way to modify the default in code).

This has existed for so long, but  not popular among API writers. Even 
before this feature, there have been few APIs using the annotation; the static 
checker also uses annotation as a fallback (i.e. not something I've introduced).

There is a way to modify default behavior by adding authorized field in 
@APIParam. See the new APIs implementation for example. I've added a section in 
the FS on how new APIs should be written if they need to be enabled by default 
for a role type; alternatively the release notes should properly document new 
APIs and leave the choice of allowing/denying those APIs to (custom) roles.

> We don't know how people are customising commands.properties and removing 
the flexibility may not be a good idea.

On the contrary, we've giving flexibility to people. It might make sense to 
enable certain features for all role types or a subset of them. We have deny 
rules (with API and wildcards supported) in case they want to override the 
default. Consider this, you wrote an API and enabled for users; the system 
admin can explicitly add allow rules and add a \* deny rule that is to say deny 
all (if not allowed) and the dynamic roles system would not consider default 
rules in annotations at all.

> The question is not about advantage of static checker, but more about 
choice and stability of the new mechanism.

There is both choice and stability. We've tried our best to make this

[GitHub] cloudstack pull request: CLOUDSTACK-9323: Fix cancel host maintena...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1454#issuecomment-214417865
  
@jburwell can I get your LGTM?  I will run CI on this again today because 
some code has changed since the last run.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8302: Removing snapshots on RB...

2016-04-25 Thread kiwiflyer

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1230#issuecomment-214414936
  
@dmytro-shevchenko We have 2 LGTMs on this and it just needs a CI run. 
Could you make the change @DaanHoogland requested above regarding the pom.xml 
and also rebase as there is a conflict?

Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214414511
  
Thanks @DaanHoogland.  Travis seems to be upset with a couple errors.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60938011
  
--- Diff: 
plugins/acl/dynamic-role-based/src/org/apache/cloudstack/acl/DynamicRoleBasedAPIAccessChecker.java
 ---
@@ -0,0 +1,167 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.acl;
+
+import com.cloud.exception.InvalidParameterValueException;
+import com.cloud.exception.PermissionDeniedException;
+import com.cloud.user.Account;
+import com.cloud.user.AccountService;
+import com.cloud.user.User;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.PluggableService;
+import com.google.common.base.Strings;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.log4j.Logger;
+
+import javax.ejb.Local;
+import javax.inject.Inject;
+import javax.naming.ConfigurationException;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Local(value = APIChecker.class)
+public class DynamicRoleBasedAPIAccessChecker extends AdapterBase 
implements APIChecker {
+
+protected static final Logger LOGGER = 
Logger.getLogger(DynamicRoleBasedAPIAccessChecker.class);
+
+@Inject
+private AccountService accountService;
+@Inject
+private RoleService roleService;
+
+private List services;
+private Map annotationRoleBasedApisMap = new 
HashMap<>();
+
+protected DynamicRoleBasedAPIAccessChecker() {
+super();
+for (RoleType roleType : RoleType.values()) {
+annotationRoleBasedApisMap.put(roleType, new 
HashSet());
+}
+}
+
+private void denyApiAccess(final String commandName) throws 
PermissionDeniedException {
+throw new PermissionDeniedException("The API does not exist or is 
blacklisted for the account's role. " +
+"The account with is not allowed to request the api: " + 
commandName);
+}
+
+private boolean checkPermission(final List  
permissions, final RolePermission.Permission permissionToCheck, final String 
commandName) {
+if (permissions == null || permissions.isEmpty() || 
Strings.isNullOrEmpty(commandName)) {
+return false;
+}
+for (final RolePermission permission : permissions) {
+if (permission.getPermission() != permissionToCheck) {
+continue;
+}
+try {
+if (permission.getRule().matches(commandName)) {
+return true;
+}
+} catch (InvalidParameterValueException e) {
+LOGGER.warn("Invalid rule permission, please fix id=" + 
permission.getId() + " rule=" + permission.getRule());
+}
+}
+return false;
+}
+
+public boolean isDisabled() {
+return !roleService.isEnabled();
+}
+
+@Override
+public boolean checkAccess(User user, String commandName) throws 
PermissionDeniedException {
+if (isDisabled()) {
+return true;
+}
+Account account = accountService.getAccount(user.getAccountId());
+if (account == null) {
+throw new PermissionDeniedException("The account id=" + 
user.getAccountId() + "for user id=" + user.getId() + "is null");
+}
+
+final Role accountRole = roleService.findRole(account.getRoleId());
+if (accountRole == null || accountRole.getId() < 1L) {
+denyApiAccess(commandName);
+}
+
+// Allow all APIs for root admins
+if (accountRole.getRoleType() == RoleType.Admin && 
accountRole.getId() == RoleType.Admin.getId()) {
+return true;
+}
+
+final List rolePermissions =

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60937993
  
--- Diff: 
plugins/acl/dynamic-role-based/src/org/apache/cloudstack/acl/DynamicRoleBasedAPIAccessChecker.java
 ---
@@ -0,0 +1,167 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.acl;
+
+import com.cloud.exception.InvalidParameterValueException;
+import com.cloud.exception.PermissionDeniedException;
+import com.cloud.user.Account;
+import com.cloud.user.AccountService;
+import com.cloud.user.User;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.PluggableService;
+import com.google.common.base.Strings;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.log4j.Logger;
+
+import javax.ejb.Local;
+import javax.inject.Inject;
+import javax.naming.ConfigurationException;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Local(value = APIChecker.class)
+public class DynamicRoleBasedAPIAccessChecker extends AdapterBase 
implements APIChecker {
+
+protected static final Logger LOGGER = 
Logger.getLogger(DynamicRoleBasedAPIAccessChecker.class);
+
+@Inject
+private AccountService accountService;
+@Inject
+private RoleService roleService;
+
+private List services;
+private Map annotationRoleBasedApisMap = new 
HashMap<>();
+
+protected DynamicRoleBasedAPIAccessChecker() {
+super();
+for (RoleType roleType : RoleType.values()) {
+annotationRoleBasedApisMap.put(roleType, new 
HashSet());
+}
+}
+
+private void denyApiAccess(final String commandName) throws 
PermissionDeniedException {
+throw new PermissionDeniedException("The API does not exist or is 
blacklisted for the account's role. " +
+"The account with is not allowed to request the api: " + 
commandName);
+}
+
+private boolean checkPermission(final List  
permissions, final RolePermission.Permission permissionToCheck, final String 
commandName) {
+if (permissions == null || permissions.isEmpty() || 
Strings.isNullOrEmpty(commandName)) {
+return false;
+}
+for (final RolePermission permission : permissions) {
+if (permission.getPermission() != permissionToCheck) {
+continue;
+}
+try {
+if (permission.getRule().matches(commandName)) {
+return true;
+}
+} catch (InvalidParameterValueException e) {
+LOGGER.warn("Invalid rule permission, please fix id=" + 
permission.getId() + " rule=" + permission.getRule());
+}
+}
+return false;
+}
+
+public boolean isDisabled() {
+return !roleService.isEnabled();
+}
+
+@Override
+public boolean checkAccess(User user, String commandName) throws 
PermissionDeniedException {
+if (isDisabled()) {
+return true;
+}
+Account account = accountService.getAccount(user.getAccountId());
+if (account == null) {
+throw new PermissionDeniedException("The account id=" + 
user.getAccountId() + "for user id=" + user.getId() + "is null");
+}
+
+final Role accountRole = roleService.findRole(account.getRoleId());
+if (accountRole == null || accountRole.getId() < 1L) {
+denyApiAccess(commandName);
+}
+
+// Allow all APIs for root admins
+if (accountRole.getRoleType() == RoleType.Admin && 
accountRole.getId() == RoleType.Admin.getId()) {
+return true;
+}
+
+final List rolePermissions =

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-214412952
  
I will run CI on this today.  @rhtyd can you force push again to see if we 
can get all green lights.  @jburwell I see you have been active on this PR, 
does it have your LGTM?  I need another LGTM as well.  Thx...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9164: Prevent firefox's quick ...

2016-04-25 Thread swill

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1271#issuecomment-214406895
  
Great thanks...  :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Jenkins broken?

2016-04-25 Thread Will Stevens

I have had to ask a lot of people to do this recently, so you are not
alone. The builds usually do work on a second attempt.
On Apr 25, 2016 9:56 AM, "David Mabry"  wrote:

> Sure, I give it a shot and let you know the results.
>
> Thanks,
> David Mabry
>
>
>
>
>
>
> On 4/25/16, 8:54 AM, "Will Stevens"  wrote:
>
> >Jenkins has been acting up a bit recently. Try doing a force push of your
> >PR to kick off the run again to see if it still fails.
> >On Apr 25, 2016 9:14 AM, "David Mabry"  wrote:
> >
> >> Hello everyone,
> >>
> >> Can someone check on Jenkins?  It looks like it not able to check out
> 4.7
> >> branch and it’s failing on my pull request.  See the logs below:
> >>
> >>
> >> FATAL: Could not checkout 4.7 with start point origin/4.7
> >> hudson.plugins.git.GitException<
> >>
> http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException
> >:
> >> Could not checkout 4.7 with start point origin/4.7
> >> at
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1962)<
> >>
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute=method
> >> >
> >> at
> >>
> org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)<
> >>
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch=method
> >> >
> >> at
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)<
> >>
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch=method
> >> >
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> at
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
> >> at
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
> >> at
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
> >> at hudson.remoting.UserRequest.perform(UserRequest.java:120)
> >> at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> >> at hudson.remoting.Request$2.run(Request.java:326)
> >> at
> >>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> at ..remote call to ubuntu-us1(Native Method)
> >> at
> >> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
> >> at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
> >> at hudson.remoting.Channel.call(Channel.java:781)
> >> at
> >>
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
> >> at com.sun.proxy.$Proxy115.checkoutBranch(Unknown Source)
> >> at
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.checkoutBranch(RemoteGitImpl.java:327)
> >> at
> >>
> com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:78)
> >> at
> >> hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:951)
> >> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1054)
> >> at hudson.scm.SCM.checkout(SCM.java:485)
> >> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
> >> at
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607)
> >> at
> >> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> at
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
> >> at hudson.model.Run.execute(Run.java:1738)
> >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> at
> >> hudson.model.ResourceController.execute(ResourceController.java:98)
> >> at hudson.model.Executor.run(Executor.java:410)
> >> Caused by: hudson.plugins.git.GitException: Command "git checkout -b 4.7
> >> origin/4.7" returned status code 1:
> >> stdout:
> >>
> engine/storage/image/src/org/apache/cloudstack/storage/image/TemplateServiceImpl.java:
> >> needs merge
> >>
>

Re: Jenkins broken?

2016-04-25 Thread David Mabry

Sure, I give it a shot and let you know the results.

Thanks,
David Mabry






On 4/25/16, 8:54 AM, "Will Stevens"  wrote:

>Jenkins has been acting up a bit recently. Try doing a force push of your
>PR to kick off the run again to see if it still fails.
>On Apr 25, 2016 9:14 AM, "David Mabry"  wrote:
>
>> Hello everyone,
>>
>> Can someone check on Jenkins?  It looks like it not able to check out 4.7
>> branch and it’s failing on my pull request.  See the logs below:
>>
>>
>> FATAL: Could not checkout 4.7 with start point origin/4.7
>> hudson.plugins.git.GitException<
>> http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>:
>> Could not checkout 4.7 with start point origin/4.7
>> at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1962)<
>> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute=method
>> >
>> at
>> org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)<
>> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch=method
>> >
>> at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)<
>> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch=method
>> >
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
>> at
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
>> at
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:120)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:48)
>> at hudson.remoting.Request$2.run(Request.java:326)
>> at
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> at ..remote call to ubuntu-us1(Native Method)
>> at
>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
>> at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
>> at hudson.remoting.Channel.call(Channel.java:781)
>> at
>> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
>> at com.sun.proxy.$Proxy115.checkoutBranch(Unknown Source)
>> at
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl.checkoutBranch(RemoteGitImpl.java:327)
>> at
>> com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:78)
>> at
>> hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:951)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1054)
>> at hudson.scm.SCM.checkout(SCM.java:485)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
>> at
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607)
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
>> at hudson.model.Run.execute(Run.java:1738)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:98)
>> at hudson.model.Executor.run(Executor.java:410)
>> Caused by: hudson.plugins.git.GitException: Command "git checkout -b 4.7
>> origin/4.7" returned status code 1:
>> stdout:
>> engine/storage/image/src/org/apache/cloudstack/storage/image/TemplateServiceImpl.java:
>> needs merge
>> services/secondary-storage/server/src/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java:
>> needs merge
>>
>> stderr: error: you need to resolve your current index first
>>
>> at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
>> at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:62)
>> at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1956)
>> at
>>

Re: Jenkins broken?

2016-04-25 Thread Will Stevens

Jenkins has been acting up a bit recently. Try doing a force push of your
PR to kick off the run again to see if it still fails.
On Apr 25, 2016 9:14 AM, "David Mabry"  wrote:

> Hello everyone,
>
> Can someone check on Jenkins?  It looks like it not able to check out 4.7
> branch and it’s failing on my pull request.  See the logs below:
>
>
> FATAL: Could not checkout 4.7 with start point origin/4.7
> hudson.plugins.git.GitException<
> http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>:
> Could not checkout 4.7 with start point origin/4.7
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1962)<
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute=method
> >
> at
> org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)<
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch=method
> >
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)<
> http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch=method
> >
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
> at hudson.remoting.UserRequest.perform(UserRequest.java:120)
> at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> at hudson.remoting.Request$2.run(Request.java:326)
> at
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> at ..remote call to ubuntu-us1(Native Method)
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
> at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
> at hudson.remoting.Channel.call(Channel.java:781)
> at
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
> at com.sun.proxy.$Proxy115.checkoutBranch(Unknown Source)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.checkoutBranch(RemoteGitImpl.java:327)
> at
> com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:78)
> at
> hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:951)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1054)
> at hudson.scm.SCM.checkout(SCM.java:485)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
> at hudson.model.Run.execute(Run.java:1738)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:98)
> at hudson.model.Executor.run(Executor.java:410)
> Caused by: hudson.plugins.git.GitException: Command "git checkout -b 4.7
> origin/4.7" returned status code 1:
> stdout:
> engine/storage/image/src/org/apache/cloudstack/storage/image/TemplateServiceImpl.java:
> needs merge
> services/secondary-storage/server/src/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java:
> needs merge
>
> stderr: error: you need to resolve your current index first
>
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:62)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1956)
> at
> org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

[GitHub] cloudstack pull request: CLOUDSTACK-9289:Automation for feature de...

2016-04-25 Thread GabrielBrascher

Github user GabrielBrascher commented on the pull request:

https://github.com/apache/cloudstack/pull/1417#issuecomment-214337105
  
Based on the code review, LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins broken?

2016-04-25 Thread David Mabry

Hello everyone,

Can someone check on Jenkins?  It looks like it not able to check out 4.7 
branch and it’s failing on my pull request.  See the logs below:


FATAL: Could not checkout 4.7 with start point origin/4.7
hudson.plugins.git.GitException:
 Could not checkout 4.7 with start point origin/4.7
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1962)
at 
org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
at hudson.remoting.UserRequest.perform(UserRequest.java:120)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at ..remote call to ubuntu-us1(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
at hudson.remoting.Channel.call(Channel.java:781)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
at com.sun.proxy.$Proxy115.checkoutBranch(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.checkoutBranch(RemoteGitImpl.java:327)
at 
com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:78)
at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:951)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1054)
at hudson.scm.SCM.checkout(SCM.java:485)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1738)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: hudson.plugins.git.GitException: Command "git checkout -b 4.7 
origin/4.7" returned status code 1:
stdout: 
engine/storage/image/src/org/apache/cloudstack/storage/image/TemplateServiceImpl.java:
 needs merge
services/secondary-storage/server/src/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java:
 needs merge

stderr: error: you need to resolve your current index first

at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:62)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:1956)
at 
org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.checkoutBranch(AbstractGitAPIImpl.java:82)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkoutBranch(CliGitAPIImpl.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)

[GitHub] cloudstack pull request: CLOUDSTACK-9228: Network update with mist...

2016-04-25 Thread nitin-maharana

Github user nitin-maharana commented on the pull request:

https://github.com/apache/cloudstack/pull/1333#issuecomment-214306315
  
@koushik-das : I will check it. Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9368: Fix for Support configur...

2016-04-25 Thread nvazquez

GitHub user nvazquez opened a pull request:

https://github.com/apache/cloudstack/pull/1518

CLOUDSTACK-9368: Fix for Support configurable NFS version for Secondary 
Storage mounts

## Description
JIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9368
This pull request address a problem introduced in #1361 in which NFS 
version couldn't be changed after hosts resources were configured on startup 
(for hosts using `VmwareResource`), and as host parameters didn't include 
`nfs.version` key, it was set `null`.

## Proposed solution
In this proposed solution `nfsVersion` would be passed in `NfsTO` through 
`CopyCommand` to `VmwareResource`, who will check if NFS version is still 
configured or not. If not, it will use the one sent in the command and will set 
it to its storage processor and storage handler. After those setups, it will 
proceed executing command.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nvazquez/cloudstack testnfs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1518


commit d0fa3245484b4e74a9f7be364deb6bb47da12de6
Author: nvazquez 
Date:   2016-04-22T19:47:05Z

CLOUDSTACK-9368: Fix NFS version set on VmwareResource




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214297629
  
@swill I started the smoke tests for this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread koushik-das

Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214296336
  
@rhtyd My comment regarding the test was more in the context of perf. test. 
In the DB for regular user I saw ~250 permissions got created. So this means 
iterating over all these entries twice (ALLOW and DENY) to find a match and 
then perform access check. There will be a perf. overhead due to this and user 
should have an option to decide whether to use static or dynamic. Also if the 
user finds some issues/bugs later during their testing there should be a 
fallback option.

Regarding upgrade implications, I went through the docs/FS but some things 
are still confusing. If existing user can continue using commands.properties 
then what happens to the new APIs that gets added. If the argument is that the 
permission can be put in as an annotation in code for new APIs then that 
removes the flexibility of the earlier mechanism (there is no way to modify the 
default in code). We don't know how people are customising commands.properties 
and removing the flexibility may not be a good idea.

The question is not about advantage of static checker, but more about 
choice and stability of the new mechanism.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214281121
  
@koushik-das can you share what you think are the advantages of static 
role-base api checker?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Cannot create issues on JIRA

2016-04-25 Thread Nicolás Vázquez

Hi guys,

I wanted to create an issue on JIRA but I can't see Cloudstack project
listed in Project field. I was able to see it before lockdown. Can I have
permission to create one?

Thanks,
Nicolas

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r60899950
  
--- Diff: 
tools/appliance/definitions/systemvmtemplate/install_systemvm_packages.sh ---
@@ -71,7 +71,7 @@ function install_packages() {
 iptables-persistent \
 libtcnative-1 libssl-dev libapr1-dev \
 python-flask \
-haproxy \
+haproxy quagga \
--- End diff --

Thanks @abhinandanprateek I'll see what I can do about building the 
systemvmtemplates


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214276787
  
@koushik-das Yes, all tests run as a regular user too. See the integration 
test, we're using user api clients (search self.getUserApiClient) to perform 
tests -- i.e. tests are not run all as root admin only. What you're asking is 
already covered, they are also run by Travis.

I'm sorry if the discussion confused you, please re-read FS again but let 
me try to explain below as well;

By default, we don't want to encourage new users to use static checker 
which is why dynamic-checker is enabled for developers/new-users. For this 
reason the commands.properties.in file in codebase has been deprecated. In 
packaging too, we're not including commands.properties file.

For existing deployments, we are *NOT* forcing users to migrate to the 
dynamic roles feature and their existing commands.properties file won't be 
renamed or removed during upgrade. Though, the upgrade path will add 
dynamic-role specific tables/schema and default roles. There is an 
upgrade/migrate script for such users who can migrate in future at their wish, 
the script will read rules from commands.properties file and put them in DB.

Please read the admin docs too if they help you understand the process:
https://github.com/apache/cloudstack-docs-admin/pull/37


Now, once a users is already using dynamic checker (fresh or migrated at a 
later stage), we don't want them to be easily able to migrate back to static 
checker as allowing admin to do that with a global setting switch is a security 
issue (sorry being pessimistic here). Therefore, we do two checks to evaluate 
if dynamic roles is allowed:
- check if the global setting says that dynamic roles is enabled
- check that commands.properties does not exist
The reverse is true for static checker, see the isEnabled()/isDisabled 
method in the checker implementation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

2016-04-25 Thread abhinandanprateek

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r60899216
  
--- Diff: 
tools/appliance/definitions/systemvmtemplate/install_systemvm_packages.sh ---
@@ -71,7 +71,7 @@ function install_packages() {
 iptables-persistent \
 libtcnative-1 libssl-dev libapr1-dev \
 python-flask \
-haproxy \
+haproxy quagga \
--- End diff --

yes we will need a new system vm template that has quagga preinstalled for 
ospf functionality to work. Other than that there are no changes and standard 
template upgrade procedure should apply.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r60898417
  
--- Diff: 
tools/appliance/definitions/systemvmtemplate/install_systemvm_packages.sh ---
@@ -71,7 +71,7 @@ function install_packages() {
 iptables-persistent \
 libtcnative-1 libssl-dev libapr1-dev \
 python-flask \
-haproxy \
+haproxy quagga \
--- End diff --

@agneya2001 do we only need this additional package? we'll need to build 
new systemvm template; in that case please change the min. required and we'll 
need an upgrade path that does that (moves to a 4.9 systemvmtemplate). We'll 
also need to build/host 4.9 systemvms somewhere.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack-docs-admin pull request: CLOUDSTACK-9299: doc for out-o...

2016-04-25 Thread rhtyd

GitHub user rhtyd opened a pull request:

https://github.com/apache/cloudstack-docs-admin/pull/38

CLOUDSTACK-9299: doc for out-of-band management

Admin documentation for out-of-band management feature

cc @pdion891 @swill 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack-docs-admin 
outofband-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack-docs-admin/pull/38.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #38


commit 758035259a5dd48a34a94c39ca9699bb2af57029
Author: Rohit Yadav 
Date:   2016-04-25T11:17:55Z

CLOUDSTACK-9299: doc for out-of-band management

Admin documentation for out-of-band management feature

Signed-off-by: Rohit Yadav 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread koushik-das

Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214269005
  
@rhtyd Have you ran the tests using a regular user? As per dynamic checker 
code, for root admin all checks are bypassed. I think a good comparison will be 
to run the tests on master with and without dynamic role checker using a 
regular user so that the DB code path gets exercised.

So are you saying static role checker will no longer be used and the 
migration of data from commands.properties to DB is forced. Rather than 
completely removing commands.properties, it will be good to keep both - static 
checker will only use commands.properties and dynamic checker only goes to DB. 
The choice should be left with users when they want to migrate.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IPv6 progress in Basic Networking

2016-04-25 Thread Wido den Hollander


> Op 25 april 2016 om 12:45 schreef Abhinandan Prateek 
> :
> 
> 
> 
> At shapeblue we are also working towards enabling ipv6 specially for advanced 
> networking/ospf.
> Our approach is more of bottom up where we plan to streamline basic 
> networking entities and extend them so that they are ipv6 aware. 

Great! Advanced and Basic will indeed overlap much.

Have you read my Wiki page? 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+Basic+Networking

My idea is **NOT** to use DHCPv6 (IA_NA), but simply use SLAAC. I found out 
that DHCPv6 is still badly supported in most Linux distributions and SLAAC 
works fine.

It is just important that Privacy Extensions are disabled in the Instance. 
Otherwise it will generate temporary IPv6 addresses.

Since we know the MAC address and the /64 prefix we can *calculate* the address 
a Instance will take. With that we do not have to store the address in a 
database.

Security Grouping can be configured accordingly.

No need to configure a DHCPv6 server on deployment of an Instance. It really 
simplifies life. Less code is better :)

> This way we ensure that these entities can then be used by both advanced and 
> basic networks.
> Some of that code has already made its way in this PR 
> https://github.com/apache/cloudstack/pull/1371/commits/2156a039cef38a397d15f6186eb515bef59bd45a
>  .
> 
> This work will have some good overlap, and will be eager to learn and suggest 
> as the PR progresses.

Looks good. I never work with VPC/Advanced Network, so I wouldn't know.

But inside a VR you would run radvd which will send out Router Advertisements. 
There is no need for DHCPv6 either since addresses obtain their IPv6 address 
using SLAAC.

Wido

>  
> 
> 
> 
> On 20/04/16, 8:54 PM, "Wido den Hollander"  wrote:
> 
> >Hi,
> >
> >Lately I've been working on IPv6 integration on CloudStack. Not much code has
> >been written (non actually), my work has mainly been getting things straight 
> >in
> >my head.
> >
> >Honestly, the work has been focused on the use-case at PCextreme Aurora 
> >Compute:
> >- Basic Networking
> >- Single IPv6 address per Instance
> >- Security Grouping on IPv6
> >- IPv6 Prefix Delegation
> >
> >Most of the things are still described here:
> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+Basic+Networking
> >
> >What I have working at PCextreme currently is:
> >- IPv6 using SLAAC
> >- DHCPv6 Prefix Delegation using ISC Kea
> >
> >The Kea configuration I'm using for my test is available on Github:
> >https://gist.github.com/wido/202b09dd574e016f6e99798036cd
> >
> >On my Instance I'm able to get an address and a prefix:
> >
> >$ ip -6 addr show dev eth0
> >
> >2: eth0:  mtu 1500 qlen 1000
> >inet6 2a00:f10:305:0:432:b2ff:fe00:479/64 scope global dynamic 
> >   valid_lft 2591904sec preferred_lft 604704sec
> >inet6 fe80::432:b2ff:fe00:479/64 scope link 
> >   valid_lft forever preferred_lft forever
> >
> >$ dhclient -6 -P -d -v eth0
> >
> >RCV: Reply message on eth0 from fe80::8618:8802:c5f6:6029.
> >RCV:  X-- IA_PD b2:00:04:79
> >RCV:  | X-- starts 1461164082
> >RCV:  | X-- t1 - renew  +1000
> >RCV:  | X-- t2 - rebind +2000
> >RCV:  | X-- [Options]
> >RCV:  | | X-- IAPREFIX 2a00:f10:500::/60
> >RCV:  | | | X-- Preferred lifetime 86400.
> >RCV:  | | | X-- Max lifetime 172800.
> >
> >This is know *ALL* happening outside the scope of CloudStack, but my next 
> >steps
> >are to start writing code.
> >
> >For IPv6 there is *NO* need to store addresses in a database. Using the /64
> >subnet + the MAC address you can calculate the address the Instance will 
> >obtain
> >using SLAAC (Stateless address autoconfiguration).
> >
> >Looking from the database perspective, using the IPv6 information in the 
> >'vlan'
> >and the 'ip6_cidr' field.
> >
> >ip6_cidr should *always* be a /64 and from there you can calculate the
> >Instance's address.
> >
> >The 'listNics' API call can calculate the 'ip6address' for the response. 
> >Using
> >the macaddress and the ip6cidr this can easily be calculated.
> >
> >The security grouping part is second, the same goes for the Prefix Delegation
> >support. Those will require more code. But just the IPv6 address in the 
> >response
> >is easy to do.
> >
> >This means that DHCPv6 is *ONLY* needed when Prefix Delegation (IA_PD) is 
> >used,
> >but otherwise it is just SLAAC. Easy and simple to deploy.
> >
> >Wido
> 
> Regards,
> 
> Abhinandan Prateek
> 
> abhinandan.prat...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue

Re: mysql-bin log files eating more space and DB server root fs filling up now at 98%

2016-04-25 Thread Wido den Hollander


> Op 25 april 2016 om 12:28 schreef anil lakineni 
> :
> 
> 
> Hi All,
> 
> In Cloud DB server, the root file system reached to 98% and found that
> */var/lib/mysql/* is consumed with more space.
> 
> Inside that specified directory i found that " *mysql bin logs* " are
> eating more space and files are there since one year.
> 
> My environment has enabled with DB replication.
> 
> Is it safe to purge the older mysql bin logs ? if yes, Could you please
> paste the working steps here which wouldn't affect replication as some
> blogs are saying that replication will be affected if we purge.
> 

Yes, you can remove them. Just use 'rm' to remove the old logs.

> *Please recommend best solution that was already worked on production
> environment*
> 
> So please suggest me the process to free up some space and clean up mysql
> bin logs.
> 

I recommend you search a bit on this topic. It is not CloudStack-specific, the 
MySQL documentation will tell you how to do this.

> P.S. I have verified other directories and logs, they are consuming very
> little space except this bin logs directory (not *ibdata1* file).
> 
> Cloud version is 4.5 and MySQL version is " 5.1.73-log "
> 
> My MySQL configuration file is,
> 
> #cat /etc/my.cnf
> *[mysqld]*
> *datadir=/var/lib/mysql*
> *socket=/var/lib/mysql/mysql.sock*
> *user=mysql*
> *# Disabling symbolic-links is recommended to prevent assorted security
> risks*
> *symbolic-links=0*
> 
> *innodb_rollback_on_timeout=1*
> *innodb_lock_wait_timeout=600*
> *max_connections=1400*
> *log-bin=mysql-bin*
> *binlog-format = 'ROW'*
> *innodb_buffer_pool_size=5500m*
> 
> *default-character-set=utf8*
> *default-collation=utf8_unicode_ci*
> *character-set-server=utf8*
> *collation-server=utf8_unicode_ci*
> *default-time-zone='+03:00'*
> 
> *# for Master / Slave*
> *server-id = 1*
> 
> *[mysqld_safe]*
> *log-error=/var/log/mysqld.log*
> *pid-file=/var/run/mysqld/mysqld.pid*
> 
> 
> Please let me know if any other information needed and please suggest the
> process that would cleanup old logs automatically by mysql.
> 
> Hope will get some help here..
> 
> Regards,
> Anil.

Re: IPv6 progress in Basic Networking

2016-04-25 Thread Abhinandan Prateek


At shapeblue we are also working towards enabling ipv6 specially for advanced 
networking/ospf.
Our approach is more of bottom up where we plan to streamline basic networking 
entities and extend them so that they are ipv6 aware. 
This way we ensure that these entities can then be used by both advanced and 
basic networks.
Some of that code has already made its way in this PR 
https://github.com/apache/cloudstack/pull/1371/commits/2156a039cef38a397d15f6186eb515bef59bd45a
 .

This work will have some good overlap, and will be eager to learn and suggest 
as the PR progresses.
 



On 20/04/16, 8:54 PM, "Wido den Hollander"  wrote:

>Hi,
>
>Lately I've been working on IPv6 integration on CloudStack. Not much code has
>been written (non actually), my work has mainly been getting things straight in
>my head.
>
>Honestly, the work has been focused on the use-case at PCextreme Aurora 
>Compute:
>- Basic Networking
>- Single IPv6 address per Instance
>- Security Grouping on IPv6
>- IPv6 Prefix Delegation
>
>Most of the things are still described here:
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+Basic+Networking
>
>What I have working at PCextreme currently is:
>- IPv6 using SLAAC
>- DHCPv6 Prefix Delegation using ISC Kea
>
>The Kea configuration I'm using for my test is available on Github:
>https://gist.github.com/wido/202b09dd574e016f6e99798036cd
>
>On my Instance I'm able to get an address and a prefix:
>
>$ ip -6 addr show dev eth0
>
>2: eth0:  mtu 1500 qlen 1000
>inet6 2a00:f10:305:0:432:b2ff:fe00:479/64 scope global dynamic 
>   valid_lft 2591904sec preferred_lft 604704sec
>inet6 fe80::432:b2ff:fe00:479/64 scope link 
>   valid_lft forever preferred_lft forever
>
>$ dhclient -6 -P -d -v eth0
>
>RCV: Reply message on eth0 from fe80::8618:8802:c5f6:6029.
>RCV:  X-- IA_PD b2:00:04:79
>RCV:  | X-- starts 1461164082
>RCV:  | X-- t1 - renew  +1000
>RCV:  | X-- t2 - rebind +2000
>RCV:  | X-- [Options]
>RCV:  | | X-- IAPREFIX 2a00:f10:500::/60
>RCV:  | | | X-- Preferred lifetime 86400.
>RCV:  | | | X-- Max lifetime 172800.
>
>This is know *ALL* happening outside the scope of CloudStack, but my next steps
>are to start writing code.
>
>For IPv6 there is *NO* need to store addresses in a database. Using the /64
>subnet + the MAC address you can calculate the address the Instance will obtain
>using SLAAC (Stateless address autoconfiguration).
>
>Looking from the database perspective, using the IPv6 information in the 'vlan'
>and the 'ip6_cidr' field.
>
>ip6_cidr should *always* be a /64 and from there you can calculate the
>Instance's address.
>
>The 'listNics' API call can calculate the 'ip6address' for the response. Using
>the macaddress and the ip6cidr this can easily be calculated.
>
>The security grouping part is second, the same goes for the Prefix Delegation
>support. Those will require more code. But just the IPv6 address in the 
>response
>is easy to do.
>
>This means that DHCPv6 is *ONLY* needed when Prefix Delegation (IA_PD) is used,
>but otherwise it is just SLAAC. Easy and simple to deploy.
>
>Wido

Regards,

Abhinandan Prateek

abhinandan.prat...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-214257913
  
 Marvin Init Successful 
=== TestName: test_oobm_background_powerstate_sync | Status : SUCCESS ===

=== TestName: test_oobm_change_password | Status : SUCCESS ===

=== TestName: test_oobm_configure_default_driver | Status : SUCCESS ===

=== TestName: test_oobm_configure_invalid_driver | Status : SUCCESS ===

=== TestName: test_oobm_disable_feature_invalid | Status : SUCCESS ===

=== TestName: test_oobm_disable_feature_valid | Status : SUCCESS ===

=== TestName: test_oobm_enable_feature_invalid | Status : SUCCESS ===

=== TestName: test_oobm_enable_feature_valid | Status : SUCCESS ===

=== TestName: test_oobm_enabledisable_across_clusterzones | Status : 
SUCCESS ===

=== TestName: test_oobm_issue_power_cycle | Status : SUCCESS ===

=== TestName: test_oobm_issue_power_off | Status : SUCCESS ===

=== TestName: test_oobm_issue_power_on | Status : SUCCESS ===

=== TestName: test_oobm_issue_power_reset | Status : SUCCESS ===

=== TestName: test_oobm_issue_power_soft | Status : SUCCESS ===

=== TestName: test_oobm_issue_power_status | Status : SUCCESS ===

=== TestName: test_oobm_multiple_mgmt_server_ownership | Status : SUCCESS 
===


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Introduction

2016-04-25 Thread Abhinandan Prateek

Welcome Rashmi !




On 08/04/16, 10:28 AM, "Rashmi Dixit"  wrote:

>Hello!
>
>I am Rashmi Dixit and have recently joined the CloudPlatform team in 
>Accelerite. I have worked on a hybrid cloud management solution supporting 
>hypervisors such as KVM, Xen, VMware, HyperV and public clouds such as EC2. My 
>areas of interest are User Interface, networking.
>
>I am really looking forward to contributing on CloudStack.
>
>See you around!
>Rashmi
>
>Rashmi Dixit
>Principal Product Engineer | CloudPlatform | www.accelerite.com
>
>
>
>
>DISCLAIMER
>==
>This e-mail may contain privileged and confidential information which is the 
>property of Accelerite, a Persistent Systems business. It is intended only for 
>the use of the individual or entity to which it is addressed. If you are not 
>the intended recipient, you are not authorized to read, retain, copy, print, 
>distribute or use this message. If you have received this communication in 
>error, please notify the sender and delete all copies of this message. 
>Accelerite, a Persistent Systems business does not accept any liability for 
>virus infected mails.

Regards,

Abhinandan Prateek

abhinandan.prat...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-214252922
  
All tests passes, except for a test that runs only on simulator that 
sometimes passes, sometimes fails. This issue is due to master and not specific 
to this PR:
=== TestName: 
test_listVolume_by_id_as_user_volumefromsamedomaindifferentaccount | Status : 
EXCEPTION ===

cc @swill 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

mysql-bin log files eating more space and DB server root fs filling up now at 98%

2016-04-25 Thread anil lakineni

Hi All,

In Cloud DB server, the root file system reached to 98% and found that
*/var/lib/mysql/* is consumed with more space.

Inside that specified directory i found that " *mysql bin logs* " are
eating more space and files are there since one year.

My environment has enabled with DB replication.

Is it safe to purge the older mysql bin logs ? if yes, Could you please
paste the working steps here which wouldn't affect replication as some
blogs are saying that replication will be affected if we purge.

*Please recommend best solution that was already worked on production
environment*

So please suggest me the process to free up some space and clean up mysql
bin logs.

P.S. I have verified other directories and logs, they are consuming very
little space except this bin logs directory (not *ibdata1* file).

Cloud version is 4.5 and MySQL version is " 5.1.73-log "

My MySQL configuration file is,

#cat /etc/my.cnf
*[mysqld]*
*datadir=/var/lib/mysql*
*socket=/var/lib/mysql/mysql.sock*
*user=mysql*
*# Disabling symbolic-links is recommended to prevent assorted security
risks*
*symbolic-links=0*

*innodb_rollback_on_timeout=1*
*innodb_lock_wait_timeout=600*
*max_connections=1400*
*log-bin=mysql-bin*
*binlog-format = 'ROW'*
*innodb_buffer_pool_size=5500m*

*default-character-set=utf8*
*default-collation=utf8_unicode_ci*
*character-set-server=utf8*
*collation-server=utf8_unicode_ci*
*default-time-zone='+03:00'*

*# for Master / Slave*
*server-id = 1*

*[mysqld_safe]*
*log-error=/var/log/mysqld.log*
*pid-file=/var/run/mysqld/mysqld.pid*


Please let me know if any other information needed and please suggest the
process that would cleanup old logs automatically by mysql.

Hope will get some help here..

Regards,
Anil.

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214249307
  
@DaanHoogland thanks, if we can migrate to Java8 I won't mind not fixing 
the issue for 1.7 at all


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1502#discussion_r60890292
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/outofbandmanagement/ConfigureOutOfBandManagementCmd.java
 ---
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.outofbandmanagement;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.NetworkRuleConflictException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.host.Host;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableMap;
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.BaseCmd;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.response.HostResponse;
+import org.apache.cloudstack.api.response.OutOfBandManagementResponse;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.outofbandmanagement.OutOfBandManagement;
+import 
org.apache.cloudstack.outofbandmanagement.OutOfBandManagementService;
+
+import javax.inject.Inject;
+
+@APICommand(name = "configureOutOfBandManagement", description = 
"Configures a host's out-of-band management interface",
+responseObject = OutOfBandManagementResponse.class, 
requestHasSensitiveInfo = true, responseHasSensitiveInfo = false, authorized = 
{RoleType.Admin})
+public class ConfigureOutOfBandManagementCmd extends BaseCmd {
+@Inject
+private OutOfBandManagementService outOfBandManagementService;
+
+/
+ API parameters /
+/
+
+@Parameter(name = ApiConstants.HOST_ID, type = CommandType.UUID, 
entityType = HostResponse.class, required = true, description = "the ID of the 
host")
+private Long hostId;
+
+@Parameter(name = ApiConstants.DRIVER, type = CommandType.STRING, 
required = true, description = "the host management interface driver, for 
example: ipmitool")
+private String driver;
+
+@Parameter(name = ApiConstants.IP_ADDRESS, type = CommandType.STRING, 
required = true, description = "the host management interface IP address")
--- End diff --

Fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-6975: Prevent dnsmasq from sta...

2016-04-25 Thread dsclose

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1514#issuecomment-214246821
  
@jburwell - sure, I should be able to look at that this week. I'll 
investigate why the CI build failed as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1502#discussion_r60890271
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/outofbandmanagement/ConfigureOutOfBandManagementCmd.java
 ---
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.outofbandmanagement;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.NetworkRuleConflictException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.host.Host;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableMap;
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.BaseCmd;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.response.HostResponse;
+import org.apache.cloudstack.api.response.OutOfBandManagementResponse;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.outofbandmanagement.OutOfBandManagement;
+import 
org.apache.cloudstack.outofbandmanagement.OutOfBandManagementService;
+
+import javax.inject.Inject;
+
+@APICommand(name = "configureOutOfBandManagement", description = 
"Configures a host's out-of-band management interface",
+responseObject = OutOfBandManagementResponse.class, 
requestHasSensitiveInfo = true, responseHasSensitiveInfo = false, authorized = 
{RoleType.Admin})
+public class ConfigureOutOfBandManagementCmd extends BaseCmd {
+@Inject
+private OutOfBandManagementService outOfBandManagementService;
+
+/
+ API parameters /
+/
+
+@Parameter(name = ApiConstants.HOST_ID, type = CommandType.UUID, 
entityType = HostResponse.class, required = true, description = "the ID of the 
host")
+private Long hostId;
+
+@Parameter(name = ApiConstants.DRIVER, type = CommandType.STRING, 
required = true, description = "the host management interface driver, for 
example: ipmitool")
+private String driver;
+
+@Parameter(name = ApiConstants.IP_ADDRESS, type = CommandType.STRING, 
required = true, description = "the host management interface IP address")
+private String address;
+
+@Parameter(name = ApiConstants.PORT, type = CommandType.STRING, 
required = true, description = "the host management interface port")
--- End diff --

For reasons described below, for now I'm keeping it a string.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-214243282
  
@rhtyd I am using 1.8 exclusively atm. I will see if I find any problems in 
that. may look at 1.7 someday.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214240215
  
I've fixed all outstanding issues, please comment if you see any 
outstanding issue.
LGTMs welcome, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-214233279
  
@wido thanks for the comments. It seemed easier to pass the arguments to 
the oobm subsystem, there is a immutable map of option,string being returned 
this is why all configuration arguments are string. Also, the driver would 
convert the port to a string to execute this in a forked process (that calls 
`ipmitool`). For this reason, all configs are passed as strings and left to how 
the driver would like to use them or interpret them.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1502#discussion_r60885329
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/outofbandmanagement/ConfigureOutOfBandManagementCmd.java
 ---
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.outofbandmanagement;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.NetworkRuleConflictException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.host.Host;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableMap;
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.BaseCmd;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.response.HostResponse;
+import org.apache.cloudstack.api.response.OutOfBandManagementResponse;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.outofbandmanagement.OutOfBandManagement;
+import 
org.apache.cloudstack.outofbandmanagement.OutOfBandManagementService;
+
+import javax.inject.Inject;
+
+@APICommand(name = "configureOutOfBandManagement", description = 
"Configures a host's out-of-band management interface",
+responseObject = OutOfBandManagementResponse.class, 
requestHasSensitiveInfo = true, responseHasSensitiveInfo = false, authorized = 
{RoleType.Admin})
+public class ConfigureOutOfBandManagementCmd extends BaseCmd {
+@Inject
+private OutOfBandManagementService outOfBandManagementService;
+
+/
+ API parameters /
+/
+
+@Parameter(name = ApiConstants.HOST_ID, type = CommandType.UUID, 
entityType = HostResponse.class, required = true, description = "the ID of the 
host")
+private Long hostId;
+
+@Parameter(name = ApiConstants.DRIVER, type = CommandType.STRING, 
required = true, description = "the host management interface driver, for 
example: ipmitool")
+private String driver;
+
+@Parameter(name = ApiConstants.IP_ADDRESS, type = CommandType.STRING, 
required = true, description = "the host management interface IP address")
--- End diff --

@wido thanks, I'll update the arg name to 'address' instead of 'ipaddress' 
as we also may want to use domain names here. It's left to interpretation of 
how a out-of-band management plugin would execute requests with these configs; 
currently the default ipmitool plugin forks out a process with arguments, so 
ipv6/ipv4/domain names are supported as long as ipmitool process is able to use 
them and execute a request.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1502#discussion_r60885367
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/outofbandmanagement/ConfigureOutOfBandManagementCmd.java
 ---
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.outofbandmanagement;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.NetworkRuleConflictException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.host.Host;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableMap;
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.BaseCmd;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.response.HostResponse;
+import org.apache.cloudstack.api.response.OutOfBandManagementResponse;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.outofbandmanagement.OutOfBandManagement;
+import 
org.apache.cloudstack.outofbandmanagement.OutOfBandManagementService;
+
+import javax.inject.Inject;
+
+@APICommand(name = "configureOutOfBandManagement", description = 
"Configures a host's out-of-band management interface",
+responseObject = OutOfBandManagementResponse.class, 
requestHasSensitiveInfo = true, responseHasSensitiveInfo = false, authorized = 
{RoleType.Admin})
+public class ConfigureOutOfBandManagementCmd extends BaseCmd {
+@Inject
+private OutOfBandManagementService outOfBandManagementService;
+
+/
+ API parameters /
+/
+
+@Parameter(name = ApiConstants.HOST_ID, type = CommandType.UUID, 
entityType = HostResponse.class, required = true, description = "the ID of the 
host")
+private Long hostId;
+
+@Parameter(name = ApiConstants.DRIVER, type = CommandType.STRING, 
required = true, description = "the host management interface driver, for 
example: ipmitool")
+private String driver;
+
+@Parameter(name = ApiConstants.IP_ADDRESS, type = CommandType.STRING, 
required = true, description = "the host management interface IP address")
+private String address;
+
+@Parameter(name = ApiConstants.PORT, type = CommandType.STRING, 
required = true, description = "the host management interface port")
--- End diff --

Thanks, will fix this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: engine/schema: fix upgrade path to work w...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1517#issuecomment-214229683
  
I found this article by Oracle MySQL team useful: 
https://www.digitalocean.com/community/tutorials/how-to-prepare-for-your-mysql-5-7-upgrade


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60882026
  
--- Diff: 
plugins/user-authenticators/ldap/src/org/apache/cloudstack/api/command/LdapCreateAccountCmd.java
 ---
@@ -119,6 +131,9 @@ private Long getDomainId() {
 
 @Override
 public void execute() throws ServerApiException {
+if (getAccountType() == null && getRoleId() == null) {
+throw new ServerApiException(ApiErrorCode.PARAM_ERROR, "Both 
account type and role ID are not provided");
--- End diff --

@jburwell before this changes `accountType` was required, now it's made 
optional with introduction of roleId, this check is to ensure at least one of 
the two are provided.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60881906
  
--- Diff: 
plugins/user-authenticators/ldap/src/org/apache/cloudstack/api/command/LdapImportUsersCmd.java
 ---
@@ -70,10 +72,12 @@
 
 @Parameter(name = ApiConstants.ACCOUNT_TYPE,
type = CommandType.SHORT,
-   required = true,
--- End diff --

@jburwell see here ^^, accountype was a required arg but with our changes 
we've made it non-required also added the roleId arg. Therefore, at least one 
of the two must be provided, if both are provided we consider roleId (and 
account type of the role).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60881499
  
--- Diff: engine/schema/src/com/cloud/upgrade/dao/Upgrade481to490.java ---
@@ -53,6 +62,139 @@ public boolean supportsRollingUpgrade() {
 
 @Override
 public void performDataMigration(Connection conn) {
+setupRolesAndPermissionsForDynamicRBAC(conn);
+}
+
+private void createDefaultRole(final Connection conn, final Long id, 
final String name, final RoleType roleType) {
+final String insertSql = String.format("INSERT INTO 
`cloud`.`roles` (`id`, `uuid`, `name`, `role_type`, `description`) values (%d, 
UUID(), '%s', '%s', 'Default %s role');",
+id, name, roleType.name(), roleType.name().toLowerCase());
+try ( PreparedStatement updatePstmt = 
conn.prepareStatement(insertSql) ) {
+updatePstmt.executeUpdate();
+} catch (SQLException e) {
+throw new CloudRuntimeException("Unable to create default role 
with id: " + id + " name: " + name, e);
+}
+}
+
+private void createRoleMapping(final Connection conn, final Long 
roleId, final String apiName) {
+final String insertSql = String.format("INSERT INTO 
`cloud`.`role_permissions` (`uuid`, `role_id`, `rule`, `permission`) values 
(UUID(), %d, '%s', 'ALLOW') ON DUPLICATE KEY UPDATE rule=rule;",
+roleId, apiName);
+try ( PreparedStatement updatePstmt = 
conn.prepareStatement(insertSql)) {
+updatePstmt.executeUpdate();
+} catch (SQLException ignored) {
+s_logger.debug("Unable to insert mapping for role id:" + 
roleId + " apiName: " + apiName);
+}
+}
+
+private void addRoleColumnAndMigrateAccountTable(final Connection 
conn, final RoleType[] roleTypes) {
+final String alterTableSql = "ALTER TABLE `cloud`.`account` ADD 
COLUMN `role_id` bigint(20) unsigned COMMENT 'role id for this account' AFTER 
`type`, " +
+"ADD KEY `fk_account__role_id` (`role_id`), " +
+"ADD CONSTRAINT `fk_account__role_id` FOREIGN KEY 
(`role_id`) REFERENCES `roles` (`id`);";
+try (PreparedStatement pstmt = 
conn.prepareStatement(alterTableSql)) {
+pstmt.executeUpdate();
+s_logger.info("Altered cloud.account table and added column 
role_id");
+} catch (SQLException e) {
+if (e.getMessage().contains("role_id")) {
+s_logger.warn("cloud.account table already has the role_id 
column, skipping altering table and migration of accounts");
+return;
+} else {
+throw new CloudRuntimeException("Unable to create column 
quota_calculated in table cloud_usage.cloud_usage", e);
+}
+}
+migrateAccountsToDefaultRoles(conn, roleTypes);
+}
+
+private void migrateAccountsToDefaultRoles(final Connection conn, 
final RoleType[] roleTypes) {
+try (PreparedStatement selectStatement = 
conn.prepareStatement("SELECT `id`, `type` FROM `cloud`.`account`;");
+ ResultSet selectResultSet = selectStatement.executeQuery()) {
+while (selectResultSet.next()) {
+Long accountId = selectResultSet.getLong(1);
+Short accountType = selectResultSet.getShort(2);
+Long roleId = null;
+for (RoleType roleType : roleTypes) {
+if (roleType.getAccountType() == accountType) {
+roleId = roleType.getId();
+break;
+}
+}
+if (roleId == null) {
+continue;
+}
+try (PreparedStatement updateStatement = 
conn.prepareStatement("UPDATE `cloud`.`account` SET role_id = ? WHERE id = 
?;")) {
+updateStatement.setLong(1, roleId);
+updateStatement.setLong(2, accountId);
+updateStatement.executeUpdate();
+updateStatement.close();
+
+} catch (SQLException e) {
+s_logger.error("Failed to update cloud.account role_id 
for account id:" + accountId + " with exception: " + e.getMessage());
+throw new CloudRuntimeException("Exception while 
updating cloud.account role_id", e);
+}
+}
+} catch (SQLException e) {
+throw new CloudRuntimeException("Exception while migrating 
existing account table's role_id column to a role based on account type", e);
+}
+s_logger.debug("Done migrating existing accounts to use one of

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60881018
  
--- Diff: server/src/org/apache/cloudstack/acl/RoleManagerImpl.java ---
@@ -0,0 +1,273 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.acl;
+
+import com.cloud.event.ActionEvent;
+import com.cloud.event.EventTypes;
+import com.cloud.exception.PermissionDeniedException;
+import com.cloud.user.Account;
+import com.cloud.user.dao.AccountDao;
+import com.cloud.utils.PropertiesUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.component.PluggableService;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallback;
+import com.cloud.utils.db.TransactionStatus;
+import com.google.common.base.Strings;
+import org.apache.cloudstack.acl.dao.RoleDao;
+import org.apache.cloudstack.acl.dao.RolePermissionsDao;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.command.admin.acl.CreateRoleCmd;
+import org.apache.cloudstack.api.command.admin.acl.CreateRolePermissionCmd;
+import org.apache.cloudstack.api.command.admin.acl.DeleteRoleCmd;
+import org.apache.cloudstack.api.command.admin.acl.DeleteRolePermissionCmd;
+import org.apache.cloudstack.api.command.admin.acl.ListRolePermissionsCmd;
+import org.apache.cloudstack.api.command.admin.acl.ListRolesCmd;
+import org.apache.cloudstack.api.command.admin.acl.UpdateRoleCmd;
+import org.apache.cloudstack.api.command.admin.acl.UpdateRolePermissionCmd;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+
+import javax.ejb.Local;
+import javax.inject.Inject;
+import java.io.File;
+import java.util.ArrayList;
+import java.util.List;
+
+@Local(value = {RoleService.class})
+public class RoleManagerImpl extends ManagerBase implements RoleService, 
Configurable, PluggableService {
+@Inject
+private AccountDao accountDao;
+@Inject
+private RoleDao roleDao;
+@Inject
+private RolePermissionsDao rolePermissionsDao;
+
+private void checkCallerAccess() {
+if (!isEnabled()) {
+throw new PermissionDeniedException("Dynamic api checker is 
not enabled, aborting role operation");
+}
+Account caller = CallContext.current().getCallingAccount();
+if (caller == null || caller.getRoleId() == null) {
+throw new PermissionDeniedException("Restricted API called by 
an invalid user account");
+}
+Role callerRole = findRole(caller.getRoleId());
+if (callerRole == null || callerRole.getRoleType() != 
RoleType.Admin) {
+throw new PermissionDeniedException("Restricted API called by 
an user account of non-Admin role type");
+}
+}
+
+@Override
+public boolean isEnabled() {
+File apiCmdFile = 
PropertiesUtil.findConfigFile(PropertiesUtil.getDefaultApiCommandsFileName());
+return RoleService.EnableDynamicApiChecker.value() && (apiCmdFile 
== null || !apiCmdFile.exists());
+}
+
+@Override
+public Role findRole(final Long id) {
+if (id == null || id < 1L) {
+return null;
+}
+return roleDao.findById(id);
+}
+
+@Override
+public RolePermission findRolePermission(final Long id) {
+if (id == null) {
+return null;
+}
+return rolePermissionsDao.findById(id);
+}
+
+@Override
+@ActionEvent(eventType = EventTypes.EVENT_ROLE_CREATE, 
eventDescription = "creating Role")
+public Role createRole(final String name, final RoleType roleType, 
final String description) {
+

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880846
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880204
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880231
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880213
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880195
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60880160
  
--- Diff: test/integration/smoke/test_dynamicroles.py ---
@@ -0,0 +1,474 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from marvin.cloudstackAPI import *
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.cloudstackException import CloudstackAPIException
+from marvin.lib.base import Account, Role, RolePermission
+from marvin.lib.utils import cleanup_resources
+from nose.plugins.attrib import attr
+
+import random
+import re
+
+
+class TestData(object):
+"""Test data object that is required to create resources
+"""
+def __init__(self):
+self.testdata = {
+"account": {
+"email": "mtu@test.cloud",
+"firstname": "Marvin",
+"lastname": "TestUser",
+"username": "roletest",
+"password": "password",
+},
+"role": {
+"name": "MarvinFake Role ",
+"type": "User",
+"description": "Fake Role created by Marvin test"
+},
+"roleadmin": {
+"name": "MarvinFake Admin Role ",
+"type": "Admin",
+"description": "Fake Admin Role created by Marvin test"
+},
+"roledomainadmin": {
+"name": "MarvinFake DomainAdmin Role ",
+"type": "DomainAdmin",
+"description": "Fake Domain-Admin Role created by Marvin 
test"
+},
+"rolepermission": {
+"roleid": 1,
+"rule": "listVirtualMachines",
+"permission": "allow",
+"description": "Fake role permission created by Marvin 
test"
+},
+"apiConfig": {
+"listApis": "allow",
+"listAccounts": "allow",
+"listClusters": "deny",
+"*VM*": "allow",
+"*Host*": "deny"
+}
+}
+
+
+class TestDynamicRoles(cloudstackTestCase):
+"""Tests dynamic role and role permission management in CloudStack
+"""
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.testdata = TestData().testdata
+
+feature_enabled = 
self.apiclient.listCapabilities(listCapabilities.listCapabilitiesCmd()).dynamicrolesenabled
+if not feature_enabled:
+self.skipTest("Dynamic Role-Based API checker not enabled, 
skipping test")
+
+self.testdata["role"]["name"] += self.getRandomString()
+self.role = Role.create(
+self.apiclient,
+self.testdata["role"]
+)
+
+self.testdata["rolepermission"]["roleid"] = self.role.id
+self.rolepermission = RolePermission.create(
+self.apiclient,
+self.testdata["rolepermission"]
+)
+
+self.account = Account.create(
+self.apiclient,
+self.testdata["account"],
+roleid=self.role.id
+)
+self.cleanup = [
+self.account,
+self.rolepermission,
+self.role
+]
+
+
+def tearDown(self):
+try:
+   cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+self.debug("Warning! Exception in tearDown: %s" % e)
+
+
+def translateRoleToAccountType(self, role_type):
+if role_type == "User":
+return 0
+elif role_type == "Admin":
+return 1
+elif role_type == "DomainAdmin":
+return 2

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1489#discussion_r60879938
  
--- Diff: utils/src/main/java/com/cloud/utils/PropertiesUtil.java ---
@@ -34,6 +34,10 @@
 public class PropertiesUtil {
 private static final Logger s_logger = 
Logger.getLogger(PropertiesUtil.class);
 
+public static String getDefaultApiCommandsFileName() {
+return "commands.properties";
+}
--- End diff --

commands.properties is consumed by server, plugins/acl/static-role-based 
and engine/schema packages. I'm keeping it this way so it can be consumed by 
all these packages, as utils is used by all and PropertiesUtils is a utility 
class to manage .properties files


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8562: Dynamic Role-Based API C...

2016-04-25 Thread rhtyd

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1489#issuecomment-214212648
  
@koushik-das this is part of the feature to be able to check access based 
on rules in DB and be consistent across all mgmt servers. In my local 
environment with stock (un-optimized) mysql server, I can do a max of 12.8k 
req/s  benchmarked against wrk

```
$ wrk -t16 -c1000 -d30s  
"http://localhost:8080/client/api?command=listUsers;
   
[14:08:08]
Running 30s test @ http://localhost:8080/client/api?command=listUsers
  16 threads and 1000 connections
  Thread Stats   Avg  Stdev Max   +/- Stdev
Latency78.35ms   64.44ms   1.52s93.98%
Req/Sec   810.93171.75 1.98k77.53%
  387964 requests in 30.09s, 147.26MB read
  Socket errors: connect 0, read 0, write 0, timeout 2
  Non-2xx or 3xx responses: 387964
Requests/sec:  12893.98
Transfer/sec:  4.89MB
```

And with another query, where dynamic checker is forced to fail doing all 
sorts of db queries, it resulted about 700 req/s.
```
$ wrk -t16 -c1000 -d30s 
"http://localhost:8096/client/api?signatureversion=3==2016-04-25T08%3A50%3A19%2B=listUsers=fmgUHUhRdCYf%2BoPHgcTVqzx0am4%3D=json=true;
Running 30s test @ 
http://localhost:8096/client/api?signatureversion=3==2016-04-25T08%3A50%3A19%2B=listUsers=fmgUHUhRdCYf%2BoPHgcTVqzx0am4%3D=json=true
  16 threads and 1000 connections
  Thread Stats   Avg  Stdev Max   +/- Stdev
Latency 1.32s   197.24ms   1.79s90.25%
Req/Sec72.78 91.71   570.00 89.25%
  21252 requests in 30.09s, 31.43MB read
  Socket errors: connect 0, read 0, write 0, timeout 18
Requests/sec:706.17
Transfer/sec:  1.04MB
```

@koushik-das we've db schema for consistency, we read data from 
commands.properties and write them to a db table. We've a test_staticroles.py 
too, that can do pre-upgrade integration testing and post-upgrade we've 
test_dynamicroles.py. Lastly, it is intended to make reverse-migration 
difficult to avoid inconsistent and unknown security behavior, read FS for 
details. If you simply turn off the restricted global setting (from true to 
false), it will disable both dynamic and static checker. One constraint for 
this to enable is that a flag in db is enabled and commands.properties file 
does not exist or readable from its classpath. Also, since commands.properties 
is removed even if you switch the flags you'll need to create this file, put in 
client/tomcatconf (as developer) and restart mgmt server as unlike 
dynamic-checker, the static checker initializes only at boot time and not 
runtime.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9322: Support for Internal LB ...

2016-04-25 Thread DaanHoogland

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1452#issuecomment-214211132
  
@swill I think this one has enough LGTM and testing and the only question I 
see remaining has been answerred. @pedro-martins is not responded anymore so 
let's merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9322: Support for Internal LB ...

2016-04-25 Thread prashanthvarma

Github user prashanthvarma commented on the pull request:

https://github.com/apache/cloudstack/pull/1452#issuecomment-214209793
  
@DaanHoogland, We haven't heard from @pedro-martins in a while. Let's hope, 
he responds soon (or) someone form the community reviews our code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8841: Storage XenMotion from X...

2016-04-25 Thread koushik-das

Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/815#issuecomment-214208033
  
@priyankparihar Are you saying that you have tested the fix in all 
scenarios with different HVs (XS, KVM, Vmware)? Also if HV doesn't allow 
certain operations it is best to prevent them with appropriate checks rather 
than some failure in the HV. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9228: Network update with mist...

2016-04-25 Thread koushik-das

Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/1333#issuecomment-214204041
  
@nitin-maharana The second dialog is not going away after clicking "yes" 
even though the API call is issued with forced=true. I don't think this is 
expected. It only goes away on clicking "no" but results in re-issue of same 
API call. Tried it in both FF and Safari.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9164: Prevent firefox's quick ...

2016-04-25 Thread anshul1886

Github user anshul1886 commented on the pull request:

https://github.com/apache/cloudstack/pull/1271#issuecomment-214154019
  
@swill done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

92 matches

Mail list logo