Re: MultiPart Upload

[Zack Shoylev]

You can control the number of threads, but I think the minimum and maximum part 
size and the maximum number of parts is controlled by the specific jclouds 
provider.

-Zack



From: Archana C 
Sent: Wednesday, February 8, 2017 6:18 AM
To: user@jclouds.apache.org
Subject: MultiPart Upload

Hi

Is there a way to substitute attributes in MultipartUploadSlicingAlgorithm  
(min, max and number of parts) in jClouds-2.0 ?


Regards
Archana

Re: jClouds 2.0 MultiPart Upload

[Zack Shoylev]

It will use an internal executorservice, if i remember correctly. I don't 
remember how it was configured, though, I'll have to look. Probably slow if not 
even sequential.

-Zack



From: Archana C 
Sent: Saturday, February 4, 2017 12:37 AM
To: user@jclouds.apache.org
Subject: Re: jClouds 2.0 MultiPart Upload

Does it mean if we are not specifying  executorservice, the upload happens in 
sequential ?

Regards
Archana



On Saturday, 4 February 2017, 10:34, Andrew Gaul  wrote:


PutOptions *takes* an ExecutorService which allows multiple threads to
concurrently upload multiple parts.

On Sat, Feb 04, 2017 at 03:34:30AM +, Archana C wrote:
> Hi
>
> I think the question was not clear. Parallel upload of multiple file is fine 
> and that can be achieved by using executorservice.
> The question here is, does multipartUpload i.e uploading of each part is 
> happening in parallel ?
> Does sequential upload of part deprecated ?
> RegardsArchana
>
>
>
>On Saturday, 4 February 2017, 1:30, Andrew Gaul 
> > wrote:
>
>
>  We rewrote multi-part uploads in jclouds 2.0.  You should pass an
> ExecutorService and via PutOptions in your call to BlobStore.putBlob.
>
> On Fri, Feb 03, 2017 at 01:11:15PM +, Archana C wrote:
> > Hi
> >
> > Is SequentialMultiPartUpload deprecated in jClouds2.0. Is all the multipart 
> > uploads are parallel now ?
> > RegardsArchana
> >
> >On Friday, 3 February 2017, 18:39, Archana C 
> > > wrote:
> >
> >
> >  Thanks it helped
> > RegardsArchana
> >
> >On Friday, 3 February 2017, 12:06, Ignasi Barrera 
> > > wrote:
> >
> >
> >  It looks like the OOM exception is thrown when writing the wire logs. When 
> > using the blob store apis you might see binary data in the logs, as the 
> > "jclouds.wire" logger logs the response/request payloads which might be 
> > huge for some blobs and can cause this kind of exceptions.
> > Could you try disabling the wire logs? (I recommend doing this for 
> > production environments).
> > Perhaps for your use case the "jclouds.headers" are enough; that will log 
> > all request/reponse path and headers but skip the bodies.
> > More on this here:https://issues.apache.org/jira/browse/JCLOUDS-1187
> > https://issues.apache.org/jira/browse/JCLOUDS-932
> >
> >
> > HTH!
> > I.
> > On Feb 3, 2017 06:22, "Archana C" 
> > > wrote:
> >
> > Hi
> >
> > I have written a sample code for multipart upload using jClouds-2.0
> > Properties overrides = new Properties();
> > BlobStoreContext context = ContextBuilder.newBuilder(" 
> > openstack-swift")
> > .endpoint("http://x.xxx.xx.xx:  
> > 5000/v2.0")
> > .credentials("xx:xx", "xx")
> > .overrides(overrides)
> > .modules(modules)
> > .buildView(BlobStoreContext. class);
> > BlobStore blobStore = context.getBlobStore();
> > blobStore. createContainerInLocation( null, CONTAINER_NAME);
> > Path path = Paths.get("test2");
> > File f = new File("test2");
> > byte []byteArray =  Files.readAllBytes(path);
> > Payload payload = newByteSourcePayload(wrap( byteArray));
> > PutOptions opt = new PutOptions();
> > opt.multipart();
> > Blob blob = blobStore.blobBuilder(OBJECT_ NAME)
> > .payload(payload). contentLength(f.length())
> > .build();
> > String etag =  blobStore.putBlob(CONTAINER_ NAME, blob, opt);
> > test2 is the file I am trying to upload which is of size 36MB and I am 
> > getting the following exception
> > 10:21:52.355 [main] DEBUG o.j.h.i. JavaUrlHttpCommandExecutorServ ice - 
> > Sending request 1344471693: PUT http://x.x.x.x:8091/v1/AUTH_ 
> > 0909ac10e7024847b1a9fe9787c7de 8f/arctestMP HTTP/1.1
> > 10:21:52.356 [main] DEBUG jclouds.headers - >> PUT 
> > http://x.x.x.x:8091/v1/AUTH_ 0909ac10e7024847b1a9fe9787c7de 8f/arctestMP 
> > HTTP/1.1
> > 10:21:52.356 [main] DEBUG jclouds.headers - >> Accept: application/json
> > 10:21:52.357 [main] DEBUG jclouds.headers - >> X-Auth-Token: 
> > fd72b74db90c46cabcca3f317d5a09 d4
> > 10:21:53.129 [main] DEBUG o.j.h.i. JavaUrlHttpCommandExecutorServ ice - 
> > Receiving response 1344471693: HTTP/1.1 201 Created
> > 10:21:53.129 [main] DEBUG jclouds.headers - << HTTP/1.1 201 Created
> > 10:21:53.129 [main] DEBUG jclouds.headers - << Date: Fri, 03 Feb 2017 
> > 04:51:53 GMT
> > 10:21:53.129 [main] DEBUG jclouds.headers - << X-Trans-Id: 
> > tx83ba6249347c43c99bb41- 0058940c68
> > 10:21:53.129 [main] DEBUG jclouds.headers - << Connection: keep-alive
> > 10:21:53.129 [main] DEBUG jclouds.headers - << Content-Type: text/html; 
> > charset=UTF-8
> > 10:21:53.129 [main] DEBUG jclouds.headers - << 

Re: put operation is not repeatable

[Zack Shoylev]

There were some bugfixes in 2.0.0 that can potentially fix your problem based 
on exactly what is failing. Have you tried that?

From: Aleksandra Nowak 
Sent: Wednesday, November 23, 2016 5:08 AM
To: user@jclouds.apache.org
Subject: put operation is not repeatable

Hi,

I use jclouds to communicate with Openstack Swift (authentication is
provided by Keystone).

Every once in a while I get this exception:

Cannot retry after server error, command is not replayable:
[method=org.jclouds.openstack.swift.SwiftKeystoneClient.public abstract
java.lang.String
org.jclouds.openstack.swift.CommonSwiftClient.putObject(java.lang.String,org.jclouds.openstack.swift.domain.SwiftObject)...

Server rejected operation connecting to PUT http://swift... (full
exception here: http://pastebin.com/WVtc0U3f)

Here is the code which I run to upload objects: http://pastebin.com/GKr1CaSQ


After the error the application can be "fixed" by a restart or by
executing a GET request to an existing object. I figured that this
happens because the key from Keystone expires before the PUT command is
executed. And while GET method is requesting a new key from Keystone,
PUT is not. And I confirmed it using ngrep on Keystone port. During PUT
there is no communication to Keystone, but during GET I can see that it
requested key renewal:

http://pastebin.com/Wi2xzWZ9

I turned on jclouds logging on TRACE level. Here is the log when I try
PUT method: http://pastebin.com/RxRBTXDH and here is log for GET first:
http://pastebin.com/WhvG4Wy9

So, it looks like PUT method is not repeatable/replayable for
InputStream. Can somebody tell me why? Or point to a point in
documentation where it is described?

Is there a why to upload InputStream to Swift in a repeatable way? My
could is wrapped by a REST call which uses Multipart, so I really would
like to use InputStream. Or is it any other way to renew the key?

Thanks in advance!
Kind regards,
Aleksandra

Re: Stream file to the OpenStack Swift

[Zack Shoylev]

You should be able to write (for example) an Apache Tomcat web application that 
accepts a file and then uploads it to Swift using jclouds.


I am not completely sure about the Tomcat part (receiving the file), but we 
have lots of examples here: https://github.com/jclouds/jclouds-examples


From: Alexandr Porunov 
Sent: Tuesday, September 13, 2016 8:57 AM
To: user@jclouds.apache.org
Subject: Stream file to the OpenStack Swift

Hello,

I need to store very large files in the OpenStack Swift through the Java server.
The problem is that I don't need to store big files locally on the Java server. 
I don't need them there. So, Java server has to act like a pipe. It has to 
accept an input stream from the client and create an output stream to the 
OpenStack Swift Proxy after that it has to read small parts of data from the 
input stream and send them to the output stream.

Can jclouds perform such acts?

Sincerely,
Alexandr

Re: Unable to access OpenStack Neutron's FloatingIPApi

[Zack Shoylev]

I added some more info to help the extension API here:

https://github.com/jclouds/jclouds-labs-openstack/pull/202

We already use name to resolve automatically, so as long as it matches the 
openstack default, it should work for detection (with this PR).

-Zack


From: Gozman, Oleksandr 
Sent: Monday, February 8, 2016 7:48 AM
To: user@jclouds.apache.org
Cc: wolle5...@gmx.de
Subject: RE: Unable to access OpenStack Neutron's FloatingIPApi

Hallo,
this is a known problem with the Openstack Liberty. Jclouds API were not yet 
updated to work with Openstack Liberty. Because of the changes made to 
different parameters, like deleting parameter namespace, API does not work 
properly with Liberty version yet.

Cheers,
Alex

From: Jens Offenbach [wolle5...@gmx.de]
Sent: Monday, February 08, 2016 2:33 PM
To: user@jclouds.apache.org
Subject: Unable to access OpenStack Neutron's FloatingIPApi

Hi,
we have an OpenStack HA Liberty deployment and cannot access the FloatingIPApi 
of Neutron. We always get a NullPointer indicating that the namespace is null. 
I think there is a mismatch between our extension list and the JClouds API. I 
have attached our extension list below. I cannot find any namespace declaration 
in the JSON descpription.
Has anybody an idea, how to get JClouds Neutron API working with this extension 
list?

{
"extensions": [{
"alias": "qos",
"updated": "2015-06-08T10:00:00-00:00",
"name": "qos",
"links": [],
"description": "The Quality of Service extension."
},
{
"alias": "dns-integration",
"updated": "2015-08-15T18:00:00-00:00",
"name": "DNS Integration",
"links": [],
"description": "Provides integration with internal DNS."
},
{
"alias": "ext-gw-mode",
"updated": "2013-03-28T10:00:00-00:00",
"name": "Neutron L3 Configurable external gateway mode",
"links": [],
"description": "Extension of the router abstraction for specifying 
whether SNAT should occur on the external gateway"
},
{
"alias": "binding",
"updated": "2014-02-03T10:00:00-00:00",
"name": "Port Binding",
"links": [],
"description": "Expose port bindings of a virtual port to external 
application"
},
{
"alias": "metering",
"updated": "2013-06-12T10:00:00-00:00",
"name": "Neutron Metering",
"links": [],
"description": "Neutron Metering extension."
},
{
"alias": "agent",
"updated": "2013-02-03T10:00:00-00:00",
"name": "agent",
"links": [],
"description": "The agent management extension."
},
{
"alias": "subnet_allocation",
"updated": "2015-03-30T10:00:00-00:00",
"name": "Subnet Allocation",
"links": [],
"description": "Enables allocation of subnets from a subnet pool"
},
{
"alias": "l3_agent_scheduler",
"updated": "2013-02-07T10:00:00-00:00",
"name": "L3 Agent Scheduler",
"links": [],
"description": "Schedule routers among l3 agents"
},
{
"alias": "external-net",
"updated": "2013-01-14T10:00:00-00:00",
"name": "Neutron external network",
"links": [],
"description": "Adds external network attribute to network resource."
},
{
"alias": "flavors",
"updated": "2014-07-06T10:00:00-00:00",
"name": "Neutron Service Flavors",
"links": [],
"description": "Service specification for advanced services"
},
{
"alias": "fwaasrouterinsertion",
"updated": "2015-01-27T10:00:00-00:00",
"name": "Firewall Router insertion",
"links": [],
"description": "Firewall Router insertion on specified set of routers"
},
{
"alias": "net-mtu",
"updated": "2015-03-25T10:00:00-00:00",
"name": "Network MTU",
"links": [],
"description": "Provides MTU attribute for a network resource."
},
{
"alias": "quotas",
"updated": "2012-07-29T10:00:00-00:00",
"name": "Quota management support",
"links": [],
"description": "Expose functions for quotas management per tenant"
},
{
"alias": "l3-ha",
"updated": "2014-04-26T00:00:00-00:00",
"name": "HA Router extension",
"links": [],
"description": "Add HA capability to routers."
},
{
"alias": "provider",
"updated": "2012-09-07T10:00:00-00:00",
"name": "Provider Network",
"links": [],
"description": "Expose mapping of virtual networks to physical networks"
},
{
"alias": "multi-provider",
"updated": "2013-06-27T10:00:00-00:00",
"name": "Multi Provider Network",
"links": [],

Re: UserAdminApi.create() method does not pass the tenantId to Keystone of OpenStack

[Zack Shoylev]

​I can try to add it this week.


From: Guy Dumouchel 
Sent: Wednesday, November 18, 2015 7:26 AM
To: Ignasi Barrera; user@jclouds.apache.org
Subject: Re: UserAdminApi.create() method does not pass the tenantId to 
Keystone of OpenStack

Ignasi,

I'm not too familiar with the Git environment and creating a pull request, 
therefore I would like to pass on creating a pull request for the fix; can 
someone else do this based on my problem description/solution described below?

Thanks
Guy

On 11/17/2015 02:45 PM, Ignasi Barrera wrote:

Thanks for the details Guy, and great job!

Wanna send a pull request with the fix? Further comments can be made there and 
that will also be the fastest way to get the fix in.

Thanks!

El 17/11/2015 20:11, "Andrew Phillips" 
> escribió:
Hi Guy

Thanks for the detailed report! Definitely sounds like a fix that we'd want to 
have in the code.

Hopefully, one of the devs that work more on the OpenStack providers can add 
some thoughts on how best to get this included, or whether there are some other 
aspects that need to be considered.

Regards

ap

On 2015-11-17 14:04, Guy Dumouchel wrote:
Hi,

jclouds version: v1.9.1
Cloud and API version: OpenStack Kilo 2015.1.0, Keystone API v2.0
Operating System version: Red Hat Enterprise Linux Server release 6.5
(Santiago)
Java version: v1.8.0.45

Problem description:

When I use jclouds client code "UserAdminApi.create()" method with the
"CreateUserOptions" object and supply a valid tenantId, the new user
gets created but the project ID is blank in OpenStack as seen in
Horizon:

USER OVERVIEW

INFORMATION

-
 User Name testU1 ID d4a18274e9c94ba0b30a14e56d9f2ca0 Email
new.em...@example.com Enabled True Project ID None

Code details:

1) setup a
"org.jclouds.openstack.keystone.v2_0.options.CreateUserOptions" object
and call the UserAdminApi.create() method as follows:

import org.jclouds.openstack.keystone.v2_0.extensions.UserAdminApi;
import org.jclouds.openstack.keystone.v2_0.options.CreateUserOptions;

...

private void createUser(Tenant aInTenant, String aInUserName) {

CreateUserOptions userOptions = CreateUserOptions.Builder
.tenant(aInTenant.getId())
.email("new.em...@example.com")
.enabled(true);

User user = userAdminApi.create(aInUserName, "abc123",
userOptions);

CreateUserOptions userOptions = CreateUserOptions.Builder
.tenant(aInTenant.getId())
   <--- aInTenant is a "
.email("new.em...@example.com")
.enabled(true);

User user = userAdminApi.create(aInUserName, "abc123",
userOptions);

...

2) I see the following in the console:

[main] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking
user:create
13:12:48.092 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Sending request 907780534: POST http:///v2.0/users
HTTP/1.1
13:12:48.092 [main] DEBUG jclouds.wire - >>
"{"user":{"name":"testU1","tenant":"2062a44f6ef64cbd89688a8a0feb2c18","password":"abc123","email":"new.em...@example.com","enabled":true}}"
13:12:48.093 [main] DEBUG jclouds.headers - >> POST
http:///v2.0/users HTTP/1.1
13:12:48.093 [main] DEBUG jclouds.headers - >> Accept:
application/json
13:12:48.093 [main] DEBUG jclouds.headers - >> X-Auth-Token: 
13:12:48.093 [main] DEBUG jclouds.headers - >> Content-Type:
application/json
13:12:48.093 [main] DEBUG jclouds.headers - >> Content-Length: 137
Nov 17, 2015 1:12:48 PM com.SomeClassManager main
INFO: SomeClassManager User Management terminated successfully
13:12:48.180 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Receiving response 907780534: HTTP/1.1 200 OK
13:12:48.181 [main] DEBUG jclouds.headers - << HTTP/1.1 200 OK
13:12:48.181 [main] DEBUG jclouds.headers - << Connection: keep-alive
13:12:48.181 [main] DEBUG jclouds.headers - << X-Openstack-Request-Id:
req-0162d808-632a-408e-a5f6-5d073ee9d551
13:12:48.181 [main] DEBUG jclouds.headers - << Vary: X-Auth-Token
13:12:48.181 [main] DEBUG jclouds.headers - << Date: Tue, 17 Nov 2015
18:12:48 GMT
13:12:48.181 [main] DEBUG jclouds.headers - << Content-Type:
application/json
13:12:48.181 [main] DEBUG jclouds.headers - << Content-Length: 189
13:12:48.181 [main] DEBUG jclouds.wire - << "{"user": {"username":
"testU1", "name": "testU1", "enabled": true, "email":
"new.em...@example.com", "tenant": 
"2062a44f6ef64cbd89688a8a0feb2c18",
"id": "d4a18274e9c94ba0b30a14e56d9f2ca0"}}"

In the above you see that the JSON data passed on the wire has
provided "tenant":"2062a44f6ef64cbd89688a8a0feb2c18" for the tenantId
portion but according to the OpenStack Identity Admin API v2 API

Re: [COMMERCIAL] Re: UserAdminApi.create() method does not pass the tenantId toKeystone of OpenStack

[Zack Shoylev]

I will after it's merged to master.


https://github.com/jclouds/jclouds/pull/880


From: Guy Dumouchel <guy.dumouc...@alcatel-lucent.com>
Sent: Wednesday, November 18, 2015 8:05 AM
To: user@jclouds.apache.org
Subject: Re: [COMMERCIAL] Re: UserAdminApi.create() method does not pass the 
tenantId toKeystone of OpenStack

Thanks Zack.

Can you also backport the commit to the 1.9.x release branches.

Thanks
Guy

On 11/18/2015 08:55 AM, Zack Shoylev wrote:

​I can try to add it this week.


From: Guy Dumouchel 
<guy.dumouc...@alcatel-lucent.com><mailto:guy.dumouc...@alcatel-lucent.com>
Sent: Wednesday, November 18, 2015 7:26 AM
To: Ignasi Barrera; user@jclouds.apache.org<mailto:user@jclouds.apache.org>
Subject: Re: UserAdminApi.create() method does not pass the tenantId to 
Keystone of OpenStack

Ignasi,

I'm not too familiar with the Git environment and creating a pull request, 
therefore I would like to pass on creating a pull request for the fix; can 
someone else do this based on my problem description/solution described below?

Thanks
Guy

On 11/17/2015 02:45 PM, Ignasi Barrera wrote:

Thanks for the details Guy, and great job!

Wanna send a pull request with the fix? Further comments can be made there and 
that will also be the fastest way to get the fix in.

Thanks!

El 17/11/2015 20:11, "Andrew Phillips" 
<andr...@apache.org<mailto:andr...@apache.org>> escribió:
Hi Guy

Thanks for the detailed report! Definitely sounds like a fix that we'd want to 
have in the code.

Hopefully, one of the devs that work more on the OpenStack providers can add 
some thoughts on how best to get this included, or whether there are some other 
aspects that need to be considered.

Regards

ap

On 2015-11-17 14:04, Guy Dumouchel wrote:
Hi,

jclouds version: v1.9.1
Cloud and API version: OpenStack Kilo 2015.1.0, Keystone API v2.0
Operating System version: Red Hat Enterprise Linux Server release 6.5
(Santiago)
Java version: v1.8.0.45

Problem description:

When I use jclouds client code "UserAdminApi.create()" method with the
"CreateUserOptions" object and supply a valid tenantId, the new user
gets created but the project ID is blank in OpenStack as seen in
Horizon:

USER OVERVIEW

INFORMATION

-
 User Name testU1 ID d4a18274e9c94ba0b30a14e56d9f2ca0 Email
new.em...@example.com<mailto:new.em...@example.com> Enabled True Project ID None

Code details:

1) setup a
"org.jclouds.openstack.keystone.v2_0.options.CreateUserOptions" object
and call the UserAdminApi.create() method as follows:

import org.jclouds.openstack.keystone.v2_0.extensions.UserAdminApi;
import org.jclouds.openstack.keystone.v2_0.options.CreateUserOptions;

...

private void createUser(Tenant aInTenant, String aInUserName) {

CreateUserOptions userOptions = CreateUserOptions.Builder
.tenant(aInTenant.getId())
.email("new.em...@example.com<mailto:new.em...@example.com>")
.enabled(true);

User user = userAdminApi.create(aInUserName, "abc123",
userOptions);

CreateUserOptions userOptions = CreateUserOptions.Builder
.tenant(aInTenant.getId())
   <--- aInTenant is a "
.email("new.em...@example.com<mailto:new.em...@example.com>")
.enabled(true);

User user = userAdminApi.create(aInUserName, "abc123",
userOptions);

...

2) I see the following in the console:

[main] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking
user:create
13:12:48.092 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Sending request 907780534: POST <http://> http:///v2.0/users
HTTP/1.1
13:12:48.092 [main] DEBUG jclouds.wire - >>
"{"user":{"name":"testU1","tenant":"2062a44f6ef64cbd89688a8a0feb2c18","password":"abc123","email":"<mailto:new.em...@example.com>new.em...@example.com<mailto:new.em...@example.com>","enabled":true}}"
13:12:48.093 [main] DEBUG jclouds.headers - >> POST
http:///v2.0/users HTTP/1.1
13:12:48.093 [main] DEBUG jclouds.headers - >> Accept:
application/json
13:12:48.093 [main] DEBUG jclouds.headers - >> X-Auth-Token: 
13:12:48.093 [main] DEBUG jclouds.headers - >> Content-Type:
application/json
13:12:48.093 [main] DEBUG jclouds.headers - >> Content-Length: 137
Nov 17, 2015 1:12:48 PM com.SomeClassManager main
INFO: SomeClassManager User Management terminated successfully
13:12:48.180 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Receiving response 907780534: HTTP/1.1 200 OK
13:12:48.181 [main] DEBUG jclouds.headers - << HTTP/1.1 200 OK
13:12:48.181 [main] DEBUG jclouds.headers - << C

Re: JCloud how to with non keystone Swift setup

[Zack Shoylev]

Can you explain a bit more? Are you trying to connect to a vanilla openstack 
setup?




From: Trupti Mali 
Sent: Wednesday, October 28, 2015 5:06 AM
To: user@jclouds.apache.org
Subject: JCloud how to with non keystone Swift setup

Hi,
I am failing at Authentication since I think it is trying to go through 
Keystone auth APIs and my setup doesn't have a keystone installed. Can you 
please let us know how that works?

swiftApi = 
ContextBuilder.newBuilder("openstack-swift").endpoint(auth_url).credentials(username,
 password).buildApi(SwiftApi.class);

We are trying it this way -
Auth_url is URL of swift setup.

Re: openstack affinity / antiaffinity in jcouds

[Zack Shoylev]

If I remember correctly, the compute template options will be converted to nova 
template options, but you might have to update the code that does that.
If you are using compute, you will have to add support for the new option in 
both.
If you are just using openstack-nova, you will only have to change 
openstack-nova.

I would also recommend you try this against 1.9.x

From: Martin Eppel (meppel) mep...@cisco.com
Sent: Monday, August 3, 2015 6:54 PM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Another question, to support the server-groups in the interim I am also looking 
into the temporary solution of  creating the server-groups through the nova 
client (outside jclouds) and pass the server-group uuid as parameter to the 
ComputeService API as part of the nova boot:

1. create the server-groups using nova client (or client outside jclouds)  ... 
nova server-group-create ...

2. pass the affinity filter as part of the create API (group=uuid) - jcloud
ComputeService.createNodesInGroup ...


E.g:

{
server: {
name: new-server-test,
imageRef: 
http://glance.openstack.example.com/openstack/images/70a599e0-31e7-49b7-b260-868f441e862b;,
flavorRef: http://openstack.example.com/openstack/flavors/1;
},
os:scheduler_hints : {
group: group_id
}
}


To add the extra parameter to the jcloud compute api is there an example I can 
compare with to implement it - at the moment it looks to me that it is just a 
matter of extending  the TemplateOptions ?

Thanks

Martin

Btw, I am basing my (temporary) changes on jclouds 1.8.1



-Original Message-
From: Zack Shoylev [mailto:zack.shoy...@rackspace.com]
Sent: Monday, August 03, 2015 3:50 AM
To: user@jclouds.apache.org
Subject: Re: openstack affinity / antiaffinity in jcouds

We already have quite a few nova extensions implemented. The Instance Group Api 
Extension will be implemented as just another extension.
I have added a JIRA here:

https://issues.apache.org/jira/browse/JCLOUDS-980

The other nova extensions, as linked before, are here:

https://github.com/jclouds/jclouds/tree/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions

It makes sense to implement the extension to be consistent with other 
extensions. The only different thing I would recommend would be using autovalue 
with builders for the domain classes and mock tests for the tests. Nova will 
have to be refactored to use these eventually anyways.

-Zack

From: Martin Eppel (meppel) mep...@cisco.com
Sent: Sunday, August 2, 2015 8:06 PM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Hi,

After digging somewhat deeper I realize, in order to support affinity / 
anti-affinity in openstack  I would have to implement (as you are already 
pointed out) the Compute API v2 extensions API (or at least portions of it). 
Do you have recommendations on how do this, how would this  implementation 
coexist with Compute API v2 or would it simply replace it or does it require 
just to extend the current implementation (I don't have yet a complete picture 
on how jclouds works) ? Would I have to worry about backward compatibility 
between  Compute API v2 extensions API and Compute API v2  API (the API's 
seem to overlap) ?

Thanks

Martin

(my apologies for any redundant or novice questions)

-Original Message-
From: Martin Eppel (meppel)
Sent: Friday, July 31, 2015 10:21 PM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Hi,

I was looking at the references below but I am still wondering, if I just want 
to support extra parameter(s) (e.g server-groups) in the nova boot command 
(corresponding RestAPI) how would I go about it - is it a matter of having to 
extend and implement new classes (any examples of similar  options I could 
reference ?) or would it be as simple as to add / pass a new property to the  
ComputeService API (which invokes the boot) ?

Thanks

Martin

-Original Message-
From: Martin Eppel (meppel)
Sent: Friday, July 31, 2015 10:18 AM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Thanks for the pointers, I'll take a look

Regards

Martin

-Original Message-
From: Zack Shoylev [mailto:zack.shoy...@rackspace.com]
Sent: Thursday, July 30, 2015 7:51 AM
To: user@jclouds.apache.org
Subject: Re: openstack affinity / antiaffinity in jcouds

I think you are referring to the instance group api extension.
https://wiki.openstack.org/wiki/InstanceGroupApiExtension

As far as I know, this extension is not supported in jclouds right now.

If it helps,  the extensions supported for nova in master are here:
https://github.com/jclouds/jclouds/tree/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions

From

Re: openstack affinity / antiaffinity in jcouds

[Zack Shoylev]

We already have quite a few nova extensions implemented. The Instance Group Api 
Extension will be implemented as just another extension.
I have added a JIRA here:

https://issues.apache.org/jira/browse/JCLOUDS-980

The other nova extensions, as linked before, are here:

https://github.com/jclouds/jclouds/tree/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions

It makes sense to implement the extension to be consistent with other 
extensions. The only different thing I would recommend would be using autovalue 
with builders for the domain classes and mock tests for the tests. Nova will 
have to be refactored to use these eventually anyways.

-Zack

From: Martin Eppel (meppel) mep...@cisco.com
Sent: Sunday, August 2, 2015 8:06 PM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Hi,

After digging somewhat deeper I realize, in order to support affinity / 
anti-affinity in openstack  I would have to implement (as you are already 
pointed out) the Compute API v2 extensions API (or at least portions of it). 
Do you have recommendations on how do this, how would this  implementation 
coexist with Compute API v2 or would it simply replace it or does it require 
just to extend the current implementation (I don't have yet a complete picture 
on how jclouds works) ? Would I have to worry about backward compatibility 
between  Compute API v2 extensions API and Compute API v2  API (the API's 
seem to overlap) ?

Thanks

Martin

(my apologies for any redundant or novice questions)

-Original Message-
From: Martin Eppel (meppel)
Sent: Friday, July 31, 2015 10:21 PM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Hi,

I was looking at the references below but I am still wondering, if I just want 
to support extra parameter(s) (e.g server-groups) in the nova boot command 
(corresponding RestAPI) how would I go about it - is it a matter of having to 
extend and implement new classes (any examples of similar  options I could 
reference ?) or would it be as simple as to add / pass a new property to the  
ComputeService API (which invokes the boot) ?

Thanks

Martin

-Original Message-
From: Martin Eppel (meppel)
Sent: Friday, July 31, 2015 10:18 AM
To: user@jclouds.apache.org
Subject: RE: openstack affinity / antiaffinity in jcouds

Thanks for the pointers, I'll take a look

Regards

Martin

-Original Message-
From: Zack Shoylev [mailto:zack.shoy...@rackspace.com]
Sent: Thursday, July 30, 2015 7:51 AM
To: user@jclouds.apache.org
Subject: Re: openstack affinity / antiaffinity in jcouds

I think you are referring to the instance group api extension.
https://wiki.openstack.org/wiki/InstanceGroupApiExtension

As far as I know, this extension is not supported in jclouds right now.

If it helps,  the extensions supported for nova in master are here:
https://github.com/jclouds/jclouds/tree/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions

From: Martin Eppel (meppel) mep...@cisco.com
Sent: Wednesday, July 29, 2015 7:26 PM
To: user@jclouds.apache.org
Subject: openstack affinity / antiaffinity in jcouds

Hi,

I had a question if jclouds currently supports the affinity / anti-affinity 
concept (and more specifically for openstack)  where VMs get deployed either 
co-located on the same compute host or not.

In openstack, affinity / anti-affinity is supported through server-groups, does 
jcloud  support any of the following (openstack specific) nova commands :

server-group-create Create a new server group with the specified
server-group-delete Delete specific server group(s).
server-group-getGet a specific server group.
server-group-list   Print a list of all server groups.

Secondly, does it support the option in the nova boot command to apply the 
affinity / anti-affinity configuration, for example:

   nova boot --image image1 --hint group=group-UUID ..

If there is a documentation which spells out the support please point me to it

Thanks

Martin Eppel

Re: openstack affinity / antiaffinity in jcouds

[Zack Shoylev]

I think you are referring to the instance group api extension.
https://wiki.openstack.org/wiki/InstanceGroupApiExtension

As far as I know, this extension is not supported in jclouds right now. 

If it helps,  the extensions supported for nova in master are here:
https://github.com/jclouds/jclouds/tree/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions

From: Martin Eppel (meppel) mep...@cisco.com
Sent: Wednesday, July 29, 2015 7:26 PM
To: user@jclouds.apache.org
Subject: openstack affinity / antiaffinity in jcouds

Hi,

I had a question if jclouds currently supports the affinity / anti-affinity 
concept (and more specifically for openstack)  where VMs get deployed either 
co-located on the same compute host or not.

In openstack, affinity / anti-affinity is supported through server-groups, does 
jcloud  support any of the following (openstack specific) nova commands :

server-group-create Create a new server group with the specified
server-group-delete Delete specific server group(s).
server-group-getGet a specific server group.
server-group-list   Print a list of all server groups.

Secondly, does it support the option in the nova boot command to apply the 
affinity / anti-affinity configuration, for example:

   nova boot --image image1 --hint group=group-UUID ..

If there is a documentation which spells out the support please point me to it

Thanks

Martin Eppel

Re: jclouds 1.9.0 with example code returns errors

[Zack Shoylev]

Are you using any of the examples here:

https://github.com/jclouds/jclouds-examples


- Zack


From: Vijay Sukthankar vijay.sukthan...@gmail.com
Sent: Friday, July 3, 2015 8:44 AM
To: user@jclouds.apache.org
Subject: Re: jclouds 1.9.0 with example code returns errors

I have set as in docs..using pom.xml the dependencies are setup and libraries 
in lib, class path has all the libs as in ./lib. I hv not deviated from docs. 
I am installing on Ubuntu 14.04 LTS.

On Friday, July 3, 2015, Andrew Phillips 
andr...@apache.orgmailto:andr...@apache.org wrote:
I installed jclouds1.9.0 and was trying out the
JCloudsNovajava example from docs and I get an error
while compiling the code. I have copied the pom.xml from
the doc and tried to compile it.. The error is

How are you setting up this project? What libraries are on your classpath?

Regards

ap


--
Regards
Vijay Sukthankar

Re: [OpenStack][Trove]datastore

[Zack Shoylev]

I don't think that is supported in the trove API

http://developer.openstack.org/api-ref-databases-v1.html?


Some providers might support additional configuration extensions, I think.


From: Mario Culjak mcul...@fesb.hr
Sent: Thursday, July 2, 2015 3:39 AM
To: user@jclouds.apache.org
Subject: [OpenStack][Trove]datastore


Hi,


I have question regarding database service. Is it possible to select datastore 
type and version when building database instance via jclouds?

I am using 1.9.0 version and create instance command only have 4 parameters: 
region, name, flavorId and size.


Thanks,

Mario.

Re: [Openstack] Routes from virtual router

[Zack Shoylev]

Hi!

I think you are referring to
http://developer.openstack.org/api-ref-networking-v2-ext.html#extraroute-ext

We don't support that feature extension right now, but we can add it. 
I have added a JIRA here:
https://issues.apache.org/jira/browse/JCLOUDS-940

Thanks!
-Zack

From: Aravindh Puthiyaparambil aravi...@mosaixsoft.com
Sent: Wednesday, June 17, 2015 9:41 PM
To: user@jclouds.apache.org
Subject: [Openstack] Routes from virtual router

Hi,

I am using jClouds 1.9.0 to work with a Openstack Juno installation.
When I query for virtual router information using the RouterApi, I see
that id, status, name, tenantId, adminStateUp and externalGatewayInfo
are returned. However I know that the equivalent Openstack REST API
also returns the routing info in a field called routes. Is there a way
to get this information using the jCloud API?

Thanks,
Aravindh

Re: COMMERCIAL:Re: Openstack keystone swift token expiration

[Zack Shoylev]

?As far as I know this fix should have been in 1.9.0, so I'm surprised it's not.



From: Forrest Townsend forrest.tow...@gmail.com
Sent: Wednesday, May 6, 2015 6:32 PM
To: user@jclouds.apache.org
Subject: COMMERCIAL:Re: Openstack keystone swift token expiration

I have already voted for this bug report haha :)

Thank you for your comments! Unfortunately I am not in a situation right not to 
run with 2.0.0, but if I have some down time I will try and comment back on 
https://issues.apache.org/jira/browse/JCLOUDS-615.

Thanks,
Forrest T.



On Wed, May 6, 2015 at 3:32 PM Ignasi Barrera 
n...@apache.orgmailto:n...@apache.org wrote:
Hi Forrest,

If I'm not wrong you're hitting
https://issues.apache.org/jira/browse/JCLOUDS-615.
If that is the case, there is a comment indicating that is fixed in
2.0.0-SNAPSHOT. Could you confirm that? Let's help us test this and
add you feedback (and vote) to the issue.

HTH!

I.

On 6 May 2015 at 23:41, Forrest Townsend 
forrest.tow...@gmail.commailto:forrest.tow...@gmail.com wrote:
 Hi

 I am running with jclouds version 1.8 and was curious if any other people
 have been hitting the default 60 minute token expiration. If someone has
 experienced this, what is a known work around? I know this isn't an
 openstack email list but but if you [1] issue a new connection to openstack
 through jclouds, will this recreate a new token? Or [2] would it be better
 to close the current connection and then connect again?

 [1] CloudWrite( ... )
 {
 ..
 ..
 putBlob( ... );

 rc = handle an exception from put;

 if rc equals any kind of fail
connect( ... )
retry putBlob( ... )

 } //end CloudWrite

 or

 [2] CloudWrite( ... )
 {
 ..
 ..
 putBlob( ... );

 rc = handle an exception from put;

 if rc equals any kind of fail
close cloud connection
connect
retry putBlob( ... )

 } //end CloudWrite



 Thanks,
 Forrest T.

RE: Does JCloud support OpenStack Juno Release ?

[Zack Shoylev]

Unfortunately, the latest release is not fully backwards-compatible. However, 
there aren't many breaking changes, so it is likely that your code will be fine 
as is - or will require only few changes.


From: Mohit Gupta [mohit2.gu...@aricent.com]
Sent: Wednesday, February 04, 2015 10:58 PM
To: Zack Shoylev; user@jclouds.apache.org
Subject: RE: Does JCloud support OpenStack Juno Release ?


Thanks for your response Zack. I am not able to find any tangible proof on the 
internet to ascertain if the Juno release is backward compatible with the 
Icehouse but most likely Juno should be backward compatible.

One more query. Is the latest Jcloud release backward compatible with the 
JCloud 1.7.3 API Library that we are currently using ?

Best Regards,
Mohit

From: Zack Shoylev [mailto:zack.shoy...@rackspace.com]
Sent: Monday, February 02, 2015 10:35 PM
To: user@jclouds.apache.org
Cc: Mohit Gupta
Subject: RE: Does JCloud support OpenStack Juno Release ?

It should be backwards compatible, but we haven't tested with Juno directly 
yet. I don't expect problems on the jclouds side though. Side note: you can 
also update jclouds, you are a few versions behind.
-Zack

From: Mohit Gupta [mohit2.gu...@aricent.com]
Sent: Monday, February 02, 2015 8:50 AM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Cc: Mohit Gupta
Subject: Does JCloud support OpenStack Juno Release ?

Hi,

I want to know if JCloud support OpenStack Juno release. If yes, which Jcloud 
release has its support ?

I am currently using JCloud 1.7.3 API Library and want to migrate our OpenStack 
setup from Icehouse to Juno. So, I just want to understand the changes required 
in my code if I go for this OpenStack setup upgrade

Best Regards,
Mohit
DISCLAIMER: This message is proprietary to Aricent and is intended solely for 
the use of the individual to whom it is addressed. It may contain privileged or 
confidential information and should not be circulated or used for any purpose 
other than for what it is intended. If you have received this message in error, 
please notify the originator immediately. If you are not the intended 
recipient, you are notified that you are strictly prohibited from using, 
copying, altering, or disclosing the contents of this message. Aricent accepts 
no responsibility for loss or damage arising from the use of the information 
transmitted by this email including damage from virus.
DISCLAIMER: This message is proprietary to Aricent and is intended solely for 
the use of the individual to whom it is addressed. It may contain privileged or 
confidential information and should not be circulated or used for any purpose 
other than for what it is intended. If you have received this message in error, 
please notify the originator immediately. If you are not the intended 
recipient, you are notified that you are strictly prohibited from using, 
copying, altering, or disclosing the contents of this message. Aricent accepts 
no responsibility for loss or damage arising from the use of the information 
transmitted by this email including damage from virus.

RE: Does JCloud support OpenStack Juno Release ?

[Zack Shoylev]

It should be backwards compatible, but we haven't tested with Juno directly 
yet. I don't expect problems on the jclouds side though. Side note: you can 
also update jclouds, you are a few versions behind.
-Zack

From: Mohit Gupta [mohit2.gu...@aricent.com]
Sent: Monday, February 02, 2015 8:50 AM
To: user@jclouds.apache.org
Cc: Mohit Gupta
Subject: Does JCloud support OpenStack Juno Release ?


Hi,

I want to know if JCloud support OpenStack Juno release. If yes, which Jcloud 
release has its support ?

I am currently using JCloud 1.7.3 API Library and want to migrate our OpenStack 
setup from Icehouse to Juno. So, I just want to understand the changes required 
in my code if I go for this OpenStack setup upgrade

Best Regards,
Mohit
DISCLAIMER: This message is proprietary to Aricent and is intended solely for 
the use of the individual to whom it is addressed. It may contain privileged or 
confidential information and should not be circulated or used for any purpose 
other than for what it is intended. If you have received this message in error, 
please notify the originator immediately. If you are not the intended 
recipient, you are notified that you are strictly prohibited from using, 
copying, altering, or disclosing the contents of this message. Aricent accepts 
no responsibility for loss or damage arising from the use of the information 
transmitted by this email including damage from virus.

RE: assign a port on vm create?

[Zack Shoylev]

(co);
  } catch (Exception e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
  }
.
.

  TemplateOptions to = templateOptions.as(NovaTemplateOptions.class)
   .generateKeyPair(true)
   .userData(userData.getBytes())
   .autoAssignFloatingIp(true)
   .networks(virgaNetID) //specified the virga net here 
to get ip 190... range
   .floatingIpPoolNames(public)
   .securityGroupNames(default) //force inherit of 
default http/icmpRULES
   //TODO
   //HOW to add http/icmp on the 
fly??
   .runAsRoot(runAsRoot
   ;
  Template template = compute.templateBuilder().imageId(
   RegionOne/ + id)
   .osFamily(OsFamily.UNRECOGNIZED).minRam(2000)//2000
   .options(to)
   .build();
.
.
.
  try {
 nodeStackSets[sdx] = compute.createNodesInGroup(basegroup, 
maxNumberOfInstances, template);
  } catch (RunNodesException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
  }

From: Zack Shoylev [mailto:zack.shoy...@rackspace.com]
Sent: Wednesday, December 03, 2014 5:14 PM
To: user@jclouds.apache.org
Subject: RE: assign a port on vm create?

Here is my suggestion.
Create a neutron network.
Create a subnet (you will be allocating fixed IPs from that subnet's CIDR 
range).
Create 8 ports on that subnet.
These ports should specify fixed ips.
These fixed ips should provide the subnet id and a fixed ip that is within the 
subnet range.

Then:
For each VM that you boot, you can provide a port uuid (when you boot with the 
nova API)
http://jclouds.apache.org/reference/javadoc/1.8.x/org/jclouds/openstack/nova/v2_0/domain/Network.Builder.html#portUuid(java.lang.String)
(It seems this is the part that you might be having trouble with).

Let me know if this works or if you are having issues getting it to work.

-Zack

From: Andrea Turli [andrea.tu...@gmail.com]
Sent: Sunday, November 30, 2014 12:08 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: RE: assign a port on vm create?

Thx Dan for the details.

I think it is a specific question for Neutron support and I can't help you 
immediately on it. Could you please file an issue on jira reporting pretty much 
the same description so that we as community don't forget it again?

thanks,
Andrea
Il 30/nov/2014 00:57 dan rusak 
merlinl...@rseincorp.commailto:merlinl...@rseincorp.com ha scritto:
Oh .. of course ..
thought that might be  a generic question.

I use JClouds 1.8.0 (I notice that neutron is broken in 1.8.1 or would have 
upgraded).
I am using Openstack icehouse on a  local endpoint collocated with my
VM installations with Chef-solo installed on my centos 6.5 image.
I need to instantiate and provision 8 VMs which need to perform nfs mounts
against each other during their provisioning and issue commands over ssh 
between them
when all are up and operational.
I have looked into everything I can find scraps of in the ether to provide 
static
ips  so this can happen but they all seem to dead end for one reason or 
another, usually
resulting in silence in fact when someone else has the same general question.
 Anyway wrapping an ip in a port using the neutron api looked promising but 
need to get the
VMs to use the port to see if that would even work.

thx
-dan



-Original Message-
From: Andrea Turli 
[mailto:andrea.tu...@gmail.commailto:andrea.tu...@gmail.com]
Sent: Saturday, November 29, 2014 9:31 AM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: assign a port on vm create?

Hi Dan,

could you please clarify which jclouds version are you using?
which provider/api are you consuming?
Is it a public openstack (I guess) endpoint or an on-premise installation?
If the latter what services can you use?

I think those are useful preliminary questions to get the context of your use 
case.

Thanks,
Andrea

On Sat, Nov 29, 2014 at 4:53 PM, dan rusak 
merlinl...@rseincorp.commailto:merlinl...@rseincorp.com wrote:
 I need to assign static ips on VM creates. It appears that the only
 way MAY be to use neutron to

 create a port and assign an ip there and assign the port to the VM.

 However when it came to assigning the port I could not find an
 interface to accomplish that.

 Hopefully I am just missing seeing it.

 Doesn’t Jclouds support this? If it can be done from command line it
 should be available

 programmatically?



 -d

RE: Can we build a template using a set of nova networks and use neutron API to assign floating IPs?

[Zack Shoylev]

Here is my suggestion

1. Create the ports that you need using neutron. You will get back port ids. 
You can probably use fixed IPs or have neutron allocate some IPs for you when 
creating the ports.

2. Boot the instances using the port ids from step 1), I think you can use 
http://jclouds.apache.org/reference/javadoc/1.8.x/org/jclouds/openstack/nova/v2_0/domain/Network.html
and
http://jclouds.apache.org/reference/javadoc/1.8.x/org/jclouds/openstack/nova/v2_0/compute/options/NovaTemplateOptions.Builder.html#novaNetworks(java.util.Set)

Let me know if are having issues. I am not sure if you can make it work going 
the other way, though.

-Zack

From: Raj [r...@apache.org]
Sent: Thursday, November 13, 2014 8:23 AM
To: user@jclouds.apache.org
Subject: Can we build a template using a set of nova networks and use neutron 
API to assign floating IPs?

Hi,

I am completely new to Jclouds. Please bare with me :)

We can build a template with a set of nova networks. Then when use that 
template to create an instance, it will get one private IP per network. For 
example, If I built my template with two nova networks, my instance will get 
two private IP.

Then, I can somehow use the PortApi to get the port uuids of the above created 
instance. If I have port uuid, then I can use 1.8.1 neutron API to assign a 
floating IP to the port.

Is this a proper way? Or is there any better way to achieve this?

Looking forward to your comment on this.

Regards.

--
Rajkumar Rajaratnam
Committer  PMC Member, Apache Stratos
Software Engineer, WSO2

RE: Neutron API versions v2 and v2_0

[Zack Shoylev]

v2_0 is deprecated.

If you are just starting with Neutron, use v2.

Thanks!
-Zack

From: Raj [r...@apache.org]
Sent: Wednesday, November 12, 2014 10:54 AM
To: user@jclouds.apache.org
Subject: Neutron API versions v2 and v2_0

Hi,

Why there two versions(v2 and v2_0) in 1.8.1 release of neutron API [1]

Could you please let me know the difference between these two and which one to 
use in a production environment?

1. 
https://github.com/jclouds/jclouds-labs-openstack/tree/jclouds-labs-openstack-1.8.1/openstack-neutron/src/main/java/org/jclouds/openstack/neutron

Regards.

--
Rajkumar Rajaratnam
Committer  PMC Member, Apache Stratos
Software Engineer, WSO2

RE: Error using clouds on Rackspace API

[Zack Shoylev]

Hi Luca,

I have been able to reproduce this. I suspect it's a combination of the java 
problem you mentioned on top of a recent service-side change, but I am still 
looking into it.

Thanks,
Zack


From: Luca Marturana [l...@draios.com]
Sent: Monday, November 03, 2014 12:45 PM
To: user@jclouds.apache.org
Subject: Re: Error using clouds on Rackspace API

Yes, I have the same error with other regions.

-
Luca Marturana
Software Engineer / Draios

 Il giorno 03/nov/2014, alle ore 19:05, Zack Shoylev 
 zack.shoy...@rackspace.com ha scritto:

 Interesting. Does your code work against other regions (such as DFW or ORD)?
 
 From: Luca Marturana [l...@draios.com]
 Sent: Monday, November 03, 2014 11:50 AM
 To: user@jclouds.apache.org
 Subject: Re: Error using clouds on Rackspace API

 I'm using version 1.8.1

 -
 Luca Marturana
 Software Engineer / Draios

 Il giorno 03/nov/2014, alle ore 18:44, Zack Shoylev 
 zack.shoy...@rackspace.com ha scritto:

 Which jclouds version is this?
 
 From: Luca Marturana [l...@draios.com]
 Sent: Monday, November 03, 2014 11:22 AM
 To: user@jclouds.apache.org
 Subject: Error using clouds on Rackspace API

 Hi to all,

 I'm using jclouds on Rackspace APIs and I get this error:

 
 org.jclouds.http.HttpResponseException: java.lang.RuntimeException: Could 
 not generate DH keypair connecting to GET 
 https://syd.databases.api.rackspacecloud.com/v1.0/794470/instances HTTP/1.1
   at 
 org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:110)
   at 
 org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
   at 
 org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
   at 
 org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
   at 
 org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
   at 
 com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
 [...]
 Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Could not 
 generate DH keypair
 


 Seems a problem on lowlevel SSL library: 
 http://stackoverflow.com/questions/10687200/java-7-and-could-not-generate-dh-keypair

 It should be solved on latest 1.7 Java JREs but something can still fail on 
 same client/server combinations.

 Anybody had this issue?

 Regards,
 Luca

RE: Error using clouds on Rackspace API

[Zack Shoylev]

Which jclouds version is this?

From: Luca Marturana [l...@draios.com]
Sent: Monday, November 03, 2014 11:22 AM
To: user@jclouds.apache.org
Subject: Error using clouds on Rackspace API

Hi to all,

I'm using jclouds on Rackspace APIs and I get this error:


org.jclouds.http.HttpResponseException: java.lang.RuntimeException: Could not 
generate DH keypair connecting to GET 
https://syd.databases.api.rackspacecloud.com/v1.0/794470/instances HTTP/1.1
at 
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:110)
at 
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at 
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at 
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at 
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at 
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
[...]
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Could not 
generate DH keypair



Seems a problem on lowlevel SSL library: 
http://stackoverflow.com/questions/10687200/java-7-and-could-not-generate-dh-keypair

It should be solved on latest 1.7 Java JREs but something can still fail on 
same client/server combinations.

Anybody had this issue?

Regards,
Luca

RE: How to get reference to FloatingIpAPI in Neutron

[Zack Shoylev]

I am not sure associating a floating IP to a specific port/interface (as 
opposed to an instance) is possible with nova. I am pretty sure it is possible 
with neutron, but all the nova examples I have seen associate a floating ip and 
an instance.
In fact, if you check what gets passed to the service with addToServer, you 
will see that an IP address gets associated to an instance.

On the other hand, if you use neutron instead, I think you might be able to use 
RouterApi.addInterfaceForPort where you can associate the floatingIP ID of your 
floatingIP to the right port with the fixed ip of the instance.

From: Udara Liyanage [udaraliyan...@gmail.com]
Sent: Thursday, September 11, 2014 1:57 AM
To: user@jclouds.apache.org
Subject: Re: How to get reference to FloatingIpAPI in Neutron


Hi,

Thanks for the help.
In my Openstack setup there are two networks.

When creating the instance I specify both networks so instance get private ips 
from both networks.


  1.
SetString novaNetworksSet = new LinkedHashSetString(2);
  2.

  3.
novaNetworksSet.add(uuid_of_network1);
  4.
novaNetworksSet.add(uuid_of_network2);
  5.
template.getOptions().as(NovaTemplateOptions.class).networks(novaNetworksSet);

Creating floating Ips.

privateIp1 and privateIp2 refers to the two private IPs allocated for the 
instance

  1.
FloatingIP publicIP1 = FloatingIP.builder().instanceId(instanceId).
  2.
fixedIp(privateIp1).
  3.
id(927b976e-0666-4039-9c38-df5b8366cce4).
  4.
ip(192.168.16.66).
  5.
instanceId(231c1871-0b54-4af6-afe6-c868b44c2307).
  6.
build();
  7.

  8.
FloatingIP publicIP2 = FloatingIP.builder().instanceId(instanceId).
  9.
fixedIp(privateIp2).
  10.
id(2de964d3-1e6b-45fc-b870-55b82cc21ab1).ip(192.168.16.64).
  11.
instanceId(231c1871-0b54-4af6-afe6-c868b44c2307).
  12.
build();

The problem is both
floatingIpApi.addToServer(publicIP1.getIp(), node.getProviderId()) and
floatingIpApi.addToServer(publicIP2.getIp(), node.getProviderId())
associates the floating ip for the same interface.

Still I am unable to allocate the floatingIp to the specified network interface.



On Wed, Sep 10, 2014 at 1:07 PM, Ambadas Ramanna Adam 
ambadas_ada...@infosys.commailto:ambadas_ada...@infosys.com wrote:
Hi Udara,

As per [1], you were asking for REST APIs. If you want to use REST APIs, then 
Dave’s answer seems correct, though haven’t used it.

If you are using JClouds API and want to associate a floating IP to a server 
with multiple nics, then, you will have to use the fixed IP associated with 
that nic.
Ex.
FloatingIP _publicIP = 
FloatingIP.Builder().instanceId(_instanceID).fixedIp(_fixedPrivateIP).pool(_publicIPPoolName).build();

Regards,
Ambadas Ramanna Adam

From: Udara Liyanage 
[mailto:udaraliyan...@gmail.commailto:udaraliyan...@gmail.com]
Sent: Wednesday, September 10, 2014 12:39 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: How to get reference to FloatingIpAPI in Neutron

Hi Ambadas,
I think I am bit confused here.
Please refer to my requirement on [1]. There I was suggested to use Openstack 
Neutron API to associate floating IPs by specifying port information. Shouldn't 
I use Jclouds Neutron in this case to create floating ip s.

[1] 
http://stackoverflow.com/questions/25637605/openstack-neutron-specify-a-network-interface-when-associating-a-floating-ip

On Wed, Sep 10, 2014 at 12:21 PM, Ambadas Ramanna Adam 
ambadas_ada...@infosys.commailto:ambadas_ada...@infosys.com wrote:
Hi Udara,

JClouds has FloatingIPApi in package org.jclouds.openstack.nova.v2_0.extensions.

You can get hold of FloatingIPApi (not FloatingIpAPI, mind the p in IP!) like 
below.

com.google.common.base.Optional? extends FloatingIPApi  
NovaApi.getFloatingIPExtensionForZone(String zone)

Hope this helps.

regards,
Ambadas Ramanna Adam

From: Udara Liyanage 
[mailto:udaraliyan...@gmail.commailto:udaraliyan...@gmail.com]
Sent: Wednesday, September 10, 2014 12:07 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: How to get reference to FloatingIpAPI in Neutron

Hi,

I was able to get reference to PortAPI and NetworkAPi, but not to 
FloatingIpAPI. Any help is appriciated.

neutronApi = ContextBuilder.newBuilder(provider)
.credentials(identity, password)
.endpoint(endpoint)
.modules(modules)
.buildApi(NeutronApi.class);
I am using Jclouds 1.8.0 version.

--
Udara S.S Liyanage.
Software Engineer at WSO2.
Commiter and PPMC Member of Apache Stratos.
Blog - http://udaraliyanage.wordpress.comhttp://udaraliyanage.wordpress.com/
phone: +94 71 443 6897tel:%2B94%2071%20443%206897

 CAUTION - Disclaimer *

This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely

for the use of the addressee(s). If you are not the 

RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

[Zack Shoylev]

Hi Daniel,

You should try testing with these Properties:

PROPERTY_SO_TIMEOUT - This will time-out the HTTP connection socket after the 
specified time. This specifies how long the client should wait before jclouds 
closes the connection and attempts to retry the HTTP call. I would suggest a 
value of 5 seconds.

PROPERTY_MAX_RETRIES - This specifies the maximum number of retries for the 
call that will be attempted by jclouds. Note that setting this to 0 will result 
in a single call. Setting this to 1 will retry once before failing. 

PROPERTY_RETRY_DELAY_START - This specifies how long the exponential backoff 
waits between retries. The exponential backoff retry mechanism in jclouds will 
double this value after each retry.

I don't think you can set them differently for different operations, though.

-Zack

From: Daniel Hsueh [daniel.hs...@evault.com]
Sent: Wednesday, September 10, 2014 8:38 AM
To: user@jclouds.apache.org
Subject: use exponential backoff on retry (was RE: JClouds BlobStore 
re-authentication when auth token expires)

Hello everyone,

Is there a way to configure the lengths of the delay in between retry attempts 
on auth, read, or write operations?

Thank you.


Daniel Hsueh
mailto:daniel.hs...@evault.com tel:+1-905-287-2167

-Original Message-
From: Daniel Hsueh
Sent: September 2, 2014 11:38 AM
To: 'user@jclouds.apache.org'
Subject: RE: JClouds BlobStore re-authentication when auth token expires

[snip]

One thing I'd like to configure is some kind of exponential backoff when the 
auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either 
retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th 
attempts.  Is there a way to configure an increasing backoff?

Thanks!


Daniel Hsueh
mailto:daniel.hs...@evault.com tel:+1-905-287-2167

RE: Associate floating ip to a specified network interface

[Zack Shoylev]

Just as a heads-up, I am working on adding support for the floating IP 
extension in neutron.
Implementation-wise it would be similar to how the Router extension is 
implemented in neutron v2.
Thanks!

From: Ignasi Barrera [ignasi.barr...@gmail.com]
Sent: Thursday, August 21, 2014 7:17 AM
To: user@jclouds.apache.org
Cc: d...@jclouds.apache.org
Subject: Re: Associate floating ip to a specified network interface

That's right, but you'll have to also rename the @PayloadParam from
pool to address.

On 21 August 2014 14:07, Udara Liyanage udaraliyan...@gmail.com wrote:
 Hi Ignasi,

 I am trying to figure out how the response is generated by looking at method
 declaration and know requests.

 For instance, if I change the method as

 @Payload(%7B\addFloatingIp\:%7B\address\:\{address}\,\interface\:\{interface}\%7D%7D)
 FloatingIP allocateFromPool(@PayloadParam(pool) String pool,
 @PayloadParam(interface) String interface);

 Will the  HTTP request generated would be as below.

 {
 addFloatingIp: {
 address: 10.10.10.1,interface:eth1
 }
 }



 On Thu, Aug 21, 2014 at 3:45 PM, Ignasi Barrera ignasi.barr...@gmail.com
 wrote:

 Hi Udara,

 There is no implementation for the API interfaces. jclouds dynamically
 generates the HTTP requests on the fly, based on the annotations of
 the invoked method. Adding an operation to the FloatingIpApi should
 only require to add the method to the interface with the corresponding
 annotations.

 You can take a look at the AllocateFromPool method [1] to see an
 example of how a simple POST request is generated.

 Feel free to ask anything you need or to join the #jclouds IRC channel!


 HTH!

 Ignasi


 [1]
 https://github.com/jclouds/jclouds/blob/master/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/extensions/FloatingIPApi.java#L93-L108

 On 21 August 2014 11:53, Udara Liyanage udaraliyan...@gmail.com wrote:
  Hi Zack,
 
  Thanks for the confirmation. Could you please point me to the place
  where
  REST API call is made to the Openstack so I would give a try patching
  Jclouds. I had a look at the code, I only found FloatingIpAPI.java which
  is
  the interface. I could not find the place where REST API call is made.
 
  According to the Openstack Quantum API, it may be possible to specify a
  nic
  other than eth0
 
  associate a floating IP to a certain instance
 
  POST /v2.0/floating-ips/{ipid}/action
  {associate: {instance_id: xxx, interface: eth0}}
 
 
 
  [1] https://wiki.openstack.org/wiki/Neutron/Quantum-floating-ips
 
 
  On Thu, Aug 21, 2014 at 11:14 AM, Udara Liyanage ud...@wso2.com wrote:
 
  Hi,
 
 
  Created a jira for this [1]
 
  [1] https://issues.apache.org/jira/browse/JCLOUDS-682
 
 
  On Wed, Aug 20, 2014 at 10:53 PM, Udara Liyanage ud...@wso2.com
  wrote:
 
  Hi,
 
  I create instances with multiple network interfaces. However when I
  associate an floating ip to the instance as below, floating ip is
  always get
  allocated to the first interface of the instance.
 
  api.addToServer(ip, server-id);
 
  How do I get the floating ip assigned to the second or an interface
  other
  than the first interface. I could not find a way of specifying a nic
  when
  associating an floating ip.
 
  I tried a different approach using Openstack commandline and seems it
  is
  working.
 
  neutron floatingip-create ext
  neutron floatingip-associate floating_ip-id port-id
 
  However I could not find a way to associate a floating ip to a port
  also.
 
  Then starting the instance with the port-id.
 
 
  --
 
  Udara Liyanage
  Software Engineer
  WSO2, Inc.: http://wso2.com
  lean. enterprise. middleware
 
  web: http://udaraliyanage.wordpress.com
  phone: +94 71 443 6897
 
 
 
 
  --
 
  Udara Liyanage
  Software Engineer
  WSO2, Inc.: http://wso2.com
  lean. enterprise. middleware
 
  web: http://udaraliyanage.wordpress.com
  phone: +94 71 443 6897
 
 
 
 
  --
  Udara S.S Liyanage.
  Software Engineer at WSO2.
  Commiter and PPMC Member of Apache Stratos.
  Blog - http://udaraliyanage.wordpress.com
  phone: +94 71 443 6897




 --
 Udara S.S Liyanage.
 Software Engineer at WSO2.
 Commiter and PPMC Member of Apache Stratos.
 Blog - http://udaraliyanage.wordpress.com
 phone: +94 71 443 6897

RE: Associate floating ip to a specified network interface

[Zack Shoylev]

I don't think this particular neutron extension is supported right now. 
Hopefully we can add support for it soon!


From: Udara Liyanage [ud...@wso2.com]
Sent: Wednesday, August 20, 2014 12:23 PM
To: user@jclouds.apache.org
Subject: Associate floating ip to a specified network interface


Hi,

I create instances with multiple network interfaces. However when I associate 
an floating ip to the instance as below, floating ip is always get allocated to 
the first interface of the instance.

api.addToServer(ip, server-id);


How do I get the floating ip assigned to the second or an interface other than 
the first interface. I could not find a way of specifying a nic when 
associating an floating ip.

I tried a different approach using Openstack commandline and seems it is 
working.

neutron floatingip-create ext
neutron floatingip-associate floating_ip-id port-id


However I could not find a way to associate a floating ip to a port also.

Then starting the instance with the port-id.

--

Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.comhttp://wso2.com/
lean. enterprise. middleware

web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

No, because then you have a silent resource leak of some kind.

From: Andrew Phillips [andr...@apache.org]
Sent: Friday, August 15, 2014 9:22 AM
To: user@jclouds.apache.org
Subject: RE: IllegalArgumentException when deleting node on Rackspace

 Of course, the proper fix is to make sure the DNS validator is not
 used when cleaning up key pairs...

Should an error during a cleanup process like this lead to the
exception being caught and logged, rather than being propagated to the
user?

ap

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

I have tracked the issue here for now:

https://issues.apache.org/jira/browse/JCLOUDS-672

From: Zack Shoylev [zack.shoy...@rackspace.com]
Sent: Friday, August 15, 2014 10:31 AM
To: user@jclouds.apache.org
Subject: RE: IllegalArgumentException when deleting node on Rackspace

No, because then you have a silent resource leak of some kind.

From: Andrew Phillips [andr...@apache.org]
Sent: Friday, August 15, 2014 9:22 AM
To: user@jclouds.apache.org
Subject: RE: IllegalArgumentException when deleting node on Rackspace

 Of course, the proper fix is to make sure the DNS validator is not
 used when cleaning up key pairs...

Should an error during a cleanup process like this lead to the
exception being caught and logged, rather than being propagated to the
user?

ap

RE: How to get reference to NeutronApi from ComputeServiceContext

[Zack Shoylev]

Interesting. Usually I would not recommend using different versions of the 
dependencies. However, I don't recall any particular reasons they might 
conflict. Can you give it a try and let me know?
Thanks!
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.com]
Sent: Thursday, August 14, 2014 12:18 AM
To: user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext


Ok, I'll change my dependency to reference 1.8.0 instead of 1.7.3.   Can 
you confirm this neutron dependency will still work with jclouds 1.7.1?

Thanks,
-Jeffrey

From: Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 10:10 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: RE: How to get reference to NeutronApi from ComputeServiceContext

Hi Jeffrey,

As this new neutron code is... new, you will have to use 1.8.0 instead of 1.7.3 
in your dependency.
Your dependency looks good to me and with 1.8.0 it will include both v2 and 
v2_0: v2_0 is deprecated (making sure we don't break people's code).
Thanks!
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.commailto:jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 11:48 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext

Hi Zack,

I looked at the application code again.  It turns out that endpoint can be 
obtained at the application level as one of the properties of the underline 
Iaas provider.   So I'm good on this one.
Question on the version v2: which version of jclouds have this?   I'm using 
jclouds 1.7.1 and plan to introduce just the neutron dependency below and not 
upgrading jclouds.   Will that work or does neutron have other dependencies 
that require upgrading jclouds?   Also, is the version below correct for using 
v2 instead of v2_0?

dependency
groupIdorg.apache.jclouds.labs/groupId
artifactIdopenstack-neutron/artifactId
version1.7.3/version
/dependency

Thanks,

-Jeffrey

From: jeffrngu jeffr...@cisco.commailto:jeffr...@cisco.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 9:32 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext


Thanks Zack.  I'm familiar with the method to get NeutronApi you mentioned.   
The problem is I can get credentials  but I'm not sure how to get the value for 
endpoint from the existing ComputeServiceContext.   Any idea?

-Jeffrey

From: Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 6:41 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: RE: How to get reference to NeutronApi from ComputeServiceContext

Hi Jeffrey,

There is no NeutronAsyncApi, just a NeutronApi. Use neutron version v2 (instead 
of v2_0) from labs.

You should be able to do something like:
NeutronApi neutronApi = ContextBuilder.newBuilder(openstack-neutron)
.credentials(username, password).endpoint(endpoint).buildApi(NeutronApi.class);

The code you are referring to is unwrapping a provider-specific implementation 
of compute from the compute abstraction layer. At the moment, jclouds has no 
networking abstraction layer, so you get to use neutron directly.

Let me know if this helps or if you need more info.
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.commailto:jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 5:09 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org; 
d...@jclouds.apache.orgmailto:d...@jclouds.apache.org
Subject: How to get reference to NeutronApi from ComputeServiceContext


Hi,

I'm not sure whether this should be sent to user or dev aliases, so I'm 
including both.

I'm trying to access neutron API from an existing ComputeServiceContext.   This 
context was created with the openstack-nova provider.   For the neutron API, 
my understanding is the provider need to be set to openstack-neutron.   My 
question is how to get a reference to neutron API given an existing 
ComputeServiceContext that was created for openstack-nova.

Below is the code I see for getting reference to NovaApi from 
ComputeServiceContext.   Is there something similar for NeutronApi?   I checked 
jclouds lab

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

Hi Dan,

Thanks for following up and for your bug report!
I have a couple of ideas why this might be happening, but I am still looking.
I'll keep you posted.
-Zack


From: Daniel Widdis [wid...@gmail.com]
Sent: Thursday, August 14, 2014 1:17 AM
To: user@jclouds.apache.org
Subject: Re: IllegalArgumentException when deleting node on Rackspace

Zack,

I am able to reproduce this exception at will, using jclouds 1.8.0.

The exception appears associated with deleting the very last server on
the account, when it has a mixed-case name.

This was the ONLY server on my account. I could not get this to
reproduce with any other servers running, or even in a
creating/deleting/error state.

Here is the sequence:
  1. Create a server on Rackspace (via website) named Cloud-Server-01.
- If it's relevant, the server was subsequently connected to my
managed server network via RackConnect.
- If it's relevant, I created from a saved image.
  2. Shut down the server using the below code (the ipAddr string passed
to the below code was 10.176.1.165)
   nodes = computeService
   .destroyNodesMatching(new PredicateNodeMetadata() {
 @Override
 public boolean apply(final NodeMetadata input) {
   return input.getPrivateAddresses().contains(ipAddr);
 }
   });

Result: java.lang.IllegalArgumentException: Object 'Cloud-Server'
doesn't match dns naming constraints. Reason: Should be only lowercase.

Stack trace for the deletion-by-IP at: http://pastebin.com/gJ8ti7yX

Additionally I can reproduce the exception by destroying server by
groupname using
computeService.destroyNodesMatching(inGroup(Cloud-Server)).  While
there is clearly a mixed-case string involved with this test, by the
time the code gets to the cleanup phase it's only passing around
NodeMetaData and not the string.

Stack trace for the group deletion: http://pastebin.com/NW76Vms5

Dan


On 8/11/14, 10:29 PM, Zack Shoylev wrote:
 Hi Daniel,

 I am still working on reproducing this, but have not been able to yet.
 My (generic) advice for now would be to update to the latest jclouds (1.8.0) 
 and let us know if this happens again.
 I will let you know if we make some progress reproducing this.
 Also, if I remember correctly, the group is usually based on how the server 
 is named.

 -Zack

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

Hi Dan,

If you go to your myclouds.rackspace.com panel, then go to Servers, under the 
SSH keys tab, what SSH key names do you have?
I suspect jclouds expects the SSH key names to have DNS-valid names. Please let 
me know.

Thanks!

From: Zack Shoylev [zack.shoy...@rackspace.com]
Sent: Thursday, August 14, 2014 8:15 AM
To: user@jclouds.apache.org
Subject: RE: IllegalArgumentException when deleting node on Rackspace

Hi Dan,

Thanks for following up and for your bug report!
I have a couple of ideas why this might be happening, but I am still looking.
I'll keep you posted.
-Zack


From: Daniel Widdis [wid...@gmail.com]
Sent: Thursday, August 14, 2014 1:17 AM
To: user@jclouds.apache.org
Subject: Re: IllegalArgumentException when deleting node on Rackspace

Zack,

I am able to reproduce this exception at will, using jclouds 1.8.0.

The exception appears associated with deleting the very last server on
the account, when it has a mixed-case name.

This was the ONLY server on my account. I could not get this to
reproduce with any other servers running, or even in a
creating/deleting/error state.

Here is the sequence:
  1. Create a server on Rackspace (via website) named Cloud-Server-01.
- If it's relevant, the server was subsequently connected to my
managed server network via RackConnect.
- If it's relevant, I created from a saved image.
  2. Shut down the server using the below code (the ipAddr string passed
to the below code was 10.176.1.165)
   nodes = computeService
   .destroyNodesMatching(new PredicateNodeMetadata() {
 @Override
 public boolean apply(final NodeMetadata input) {
   return input.getPrivateAddresses().contains(ipAddr);
 }
   });

Result: java.lang.IllegalArgumentException: Object 'Cloud-Server'
doesn't match dns naming constraints. Reason: Should be only lowercase.

Stack trace for the deletion-by-IP at: http://pastebin.com/gJ8ti7yX

Additionally I can reproduce the exception by destroying server by
groupname using
computeService.destroyNodesMatching(inGroup(Cloud-Server)).  While
there is clearly a mixed-case string involved with this test, by the
time the code gets to the cleanup phase it's only passing around
NodeMetaData and not the string.

Stack trace for the group deletion: http://pastebin.com/NW76Vms5

Dan


On 8/11/14, 10:29 PM, Zack Shoylev wrote:
 Hi Daniel,

 I am still working on reproducing this, but have not been able to yet.
 My (generic) advice for now would be to update to the latest jclouds (1.8.0) 
 and let us know if this happens again.
 I will let you know if we make some progress reproducing this.
 Also, if I remember correctly, the group is usually based on how the server 
 is named.

 -Zack

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

The key data is stored using name-key pairs. My guess is that the problematic 
string is the Dan Widdis one, but I am still testing.
-Zack


From: Daniel Widdis [wid...@gmail.com]
Sent: Thursday, August 14, 2014 10:56 AM
To: user@jclouds.apache.org
Subject: Re: IllegalArgumentException when deleting node on Rackspace

Or, possibly the name I just gave was just a Rackspace-friendly name.
The one actually in the uploaded key is:

danielwiddis@Daniel-Widdiss-MacBook-Pro.local

On 8/14/14, 7:10 AM, Zack Shoylev wrote:
 Hi Dan,

 If you go to your myclouds.rackspace.com panel, then go to Servers, under the 
 SSH keys tab, what SSH key names do you have?
 I suspect jclouds expects the SSH key names to have DNS-valid names. Please 
 let me know.

 Thanks!

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

To be precise, when cleaning up resources, jclouds also cleans up any orphaned 
key pairs. However, when it cleans up the keypairs, it validates the key pair 
name against the DNS validator. The key pairs are controlled using a key pair 
extension, so jclouds validates these values before it executes the clean up 
call. Unfortunately this means once keypairs have specific names, the cleanup 
process breaks.

To get the actual key pair name, you can directly use the extension. 
Additionally, as a workaround, after deleting the server, you could delete the 
key pair yourself. The extension is part of the Nova Apis I believe.

Of course, the proper fix is to make sure the DNS validator is not used when 
cleaning up key pairs...

From: Daniel Widdis [wid...@gmail.com]
Sent: Thursday, August 14, 2014 3:43 PM
To: user@jclouds.apache.org
Subject: Re: IllegalArgumentException when deleting node on Rackspace

Chris,

I tried again with a dns-compliant key name... same exception.  I don't think 
the name given to keys matters; the error is somewhere where the 
(non-compliant) server name is associated with the key (and, it's cleaning up 
resources after the very last server deletion.)

Dan

On 8/14/14, 12:50 PM, Chris Custine wrote:
If that is still being evaluated against the DnsNameValidator, it is probably 
now failing because of the underscore.  Try “dan-widdis” and see if that works. 
 Either way, this is an interesting type of issue that has come up several 
times recently so we should probably noodle this a bit and figure out if we can 
selectively relax some of these validations.

Thanks,
Chris
--
Chris Custine



On August 14, 2014 at 11:16:15 AM, Daniel Widdis 
(wid...@gmail.commailto:wid...@gmail.com) wrote:

Zack,

I re-uploaded the key, giving it a lowercase name in Rackspace
(dan_widdis) and also all lowercase in my uploaded public key.

But the exception has returned. So I don't think it matters what the
key is named: what matters is the name of the server to which the key is
associated.

Dan

On 8/14/14, 9:37 AM, Zack Shoylev wrote:
 The key data is stored using name-key pairs. My guess is that the problematic 
 string is the Dan Widdis one, but I am still testing.
 -Zack

 
 From: Daniel Widdis [wid...@gmail.commailto:wid...@gmail.com]
 Sent: Thursday, August 14, 2014 10:56 AM
 To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
 Subject: Re: IllegalArgumentException when deleting node on Rackspace

 Or, possibly the name I just gave was just a Rackspace-friendly name.
 The one actually in the uploaded key is:

 danielwiddis@Daniel-Widdiss-MacBook-Pro.localmailto:danielwiddis@Daniel-Widdiss-MacBook-Pro.local

 On 8/14/14, 7:10 AM, Zack Shoylev wrote:
 Hi Dan,

 If you go to your myclouds.rackspace.com panel, then go to Servers, under 
 the SSH keys tab, what SSH key names do you have?
 I suspect jclouds expects the SSH key names to have DNS-valid names. Please 
 let me know.

 Thanks!

RE: How to get reference to NeutronApi from ComputeServiceContext

[Zack Shoylev]

Hi Jeffrey,

There is no NeutronAsyncApi, just a NeutronApi. Use neutron version v2 (instead 
of v2_0) from labs.

You should be able to do something like:
NeutronApi neutronApi = ContextBuilder.newBuilder(openstack-neutron)
.credentials(username, password).endpoint(endpoint).buildApi(NeutronApi.class);

The code you are referring to is unwrapping a provider-specific implementation 
of compute from the compute abstraction layer. At the moment, jclouds has no 
networking abstraction layer, so you get to use neutron directly.

Let me know if this helps or if you need more info.
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 5:09 PM
To: user@jclouds.apache.org; d...@jclouds.apache.org
Subject: How to get reference to NeutronApi from ComputeServiceContext


Hi,

I'm not sure whether this should be sent to user or dev aliases, so I'm 
including both.

I'm trying to access neutron API from an existing ComputeServiceContext.   This 
context was created with the openstack-nova provider.   For the neutron API, 
my understanding is the provider need to be set to openstack-neutron.   My 
question is how to get a reference to neutron API given an existing 
ComputeServiceContext that was created for openstack-nova.

Below is the code I see for getting reference to NovaApi from 
ComputeServiceContext.   Is there something similar for NeutronApi?   I checked 
jclouds lab code and there is NeutronApi but there isn't a NeutronAsyncApi.


ComputeServiceContext context;

RestContextNovaApi, NovaAsyncApi nova = context.unwrap();


Thanks,

-Jeffrey

RE: How to get reference to NeutronApi from ComputeServiceContext

[Zack Shoylev]

Hi Jeffrey,

The endpoint is most likely set in the provider you use. For example, in 
cloudservers-us (The US rackspace provider), we have:
https://github.com/jclouds/jclouds/blob/master/providers/rackspace-cloudservers-us/src/main/java/org/jclouds/rackspace/cloudservers/us/CloudServersUSProviderMetadata.java#L84

For neutron, the endpoint will likely be the same. For openstack deployments, 
the endpoints will also be in the service catalog.

-Zack


From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 11:32 PM
To: user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext


Thanks Zack.  I'm familiar with the method to get NeutronApi you mentioned.   
The problem is I can get credentials  but I'm not sure how to get the value for 
endpoint from the existing ComputeServiceContext.   Any idea?

-Jeffrey

From: Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 6:41 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: RE: How to get reference to NeutronApi from ComputeServiceContext

Hi Jeffrey,

There is no NeutronAsyncApi, just a NeutronApi. Use neutron version v2 (instead 
of v2_0) from labs.

You should be able to do something like:
NeutronApi neutronApi = ContextBuilder.newBuilder(openstack-neutron)
.credentials(username, password).endpoint(endpoint).buildApi(NeutronApi.class);

The code you are referring to is unwrapping a provider-specific implementation 
of compute from the compute abstraction layer. At the moment, jclouds has no 
networking abstraction layer, so you get to use neutron directly.

Let me know if this helps or if you need more info.
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.commailto:jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 5:09 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org; 
d...@jclouds.apache.orgmailto:d...@jclouds.apache.org
Subject: How to get reference to NeutronApi from ComputeServiceContext


Hi,

I'm not sure whether this should be sent to user or dev aliases, so I'm 
including both.

I'm trying to access neutron API from an existing ComputeServiceContext.   This 
context was created with the openstack-nova provider.   For the neutron API, 
my understanding is the provider need to be set to openstack-neutron.   My 
question is how to get a reference to neutron API given an existing 
ComputeServiceContext that was created for openstack-nova.

Below is the code I see for getting reference to NovaApi from 
ComputeServiceContext.   Is there something similar for NeutronApi?   I checked 
jclouds lab code and there is NeutronApi but there isn't a NeutronAsyncApi.


ComputeServiceContext context;

RestContextNovaApi, NovaAsyncApi nova = context.unwrap();


Thanks,

-Jeffrey

RE: How to get reference to NeutronApi from ComputeServiceContext

[Zack Shoylev]

Hi Jeffrey,

As this new neutron code is... new, you will have to use 1.8.0 instead of 1.7.3 
in your dependency.
Your dependency looks good to me and with 1.8.0 it will include both v2 and 
v2_0: v2_0 is deprecated (making sure we don't break people's code).
Thanks!
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 11:48 PM
To: user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext

Hi Zack,

I looked at the application code again.  It turns out that endpoint can be 
obtained at the application level as one of the properties of the underline 
Iaas provider.   So I'm good on this one.
Question on the version v2: which version of jclouds have this?   I'm using 
jclouds 1.7.1 and plan to introduce just the neutron dependency below and not 
upgrading jclouds.   Will that work or does neutron have other dependencies 
that require upgrading jclouds?   Also, is the version below correct for using 
v2 instead of v2_0?

dependency
groupIdorg.apache.jclouds.labs/groupId
artifactIdopenstack-neutron/artifactId
version1.7.3/version
/dependency

Thanks,

-Jeffrey

From: jeffrngu jeffr...@cisco.commailto:jeffr...@cisco.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 9:32 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: How to get reference to NeutronApi from ComputeServiceContext


Thanks Zack.  I'm familiar with the method to get NeutronApi you mentioned.   
The problem is I can get credentials  but I'm not sure how to get the value for 
endpoint from the existing ComputeServiceContext.   Any idea?

-Jeffrey

From: Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com
Reply-To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Date: Wednesday, August 13, 2014 6:41 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org 
user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: RE: How to get reference to NeutronApi from ComputeServiceContext

Hi Jeffrey,

There is no NeutronAsyncApi, just a NeutronApi. Use neutron version v2 (instead 
of v2_0) from labs.

You should be able to do something like:
NeutronApi neutronApi = ContextBuilder.newBuilder(openstack-neutron)
.credentials(username, password).endpoint(endpoint).buildApi(NeutronApi.class);

The code you are referring to is unwrapping a provider-specific implementation 
of compute from the compute abstraction layer. At the moment, jclouds has no 
networking abstraction layer, so you get to use neutron directly.

Let me know if this helps or if you need more info.
-Zack

From: Jeffrey Nguyen (jeffrngu) [jeffr...@cisco.commailto:jeffr...@cisco.com]
Sent: Wednesday, August 13, 2014 5:09 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org; 
d...@jclouds.apache.orgmailto:d...@jclouds.apache.org
Subject: How to get reference to NeutronApi from ComputeServiceContext


Hi,

I'm not sure whether this should be sent to user or dev aliases, so I'm 
including both.

I'm trying to access neutron API from an existing ComputeServiceContext.   This 
context was created with the openstack-nova provider.   For the neutron API, 
my understanding is the provider need to be set to openstack-neutron.   My 
question is how to get a reference to neutron API given an existing 
ComputeServiceContext that was created for openstack-nova.

Below is the code I see for getting reference to NovaApi from 
ComputeServiceContext.   Is there something similar for NeutronApi?   I checked 
jclouds lab code and there is NeutronApi but there isn't a NeutronAsyncApi.


ComputeServiceContext context;

RestContextNovaApi, NovaAsyncApi nova = context.unwrap();


Thanks,

-Jeffrey

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

Hi Daniel,

Can this issue be reproduced consistently? Does the number of servers matter?

From your stack traces it seems this code is called
https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/predicates/validators/DnsNameValidator.java#L56
 so it could be some kind of weird interaction.


From: Daniel Widdis [wid...@gmail.com]
Sent: Saturday, August 09, 2014 1:57 PM
To: user@jclouds.apache.org
Subject: IllegalArgumentException when deleting node on Rackspace

This issue may be related, or may be completely different than Sunil
Shah's recent thread on 2-letter DNS names when creating GCE nodes, but
ultimately boils down to the same exception at the same line of code.  I
debated whether to reply to that thread or start a new one, and elected
the latter.  I will take a look at JIRA where the issues may possibly be
combined.

My specific symptoms/situation:
- jclouds 1.7.3
- I had manually created (through website API) several Rackspace cloud
servers, with the names Cloud-Server-30, Cloud-Server-31, etc.  These
were the default names provided on the website and I chose (by virtue of
laziness) to leave them as they were.
- My code deletes these servers when it is done using them.  I do this
by passing the private IP address of the server to this code:
   nodes = computeService
   .destroyNodesMatching(new PredicateNodeMetadata() {
 @Override
 public boolean apply(final NodeMetadata input) {
   return input.getPrivateAddresses().contains(ipAddr);
 }
   });

- Note that my code never uses the server name, nor is even aware of
it.  I simply want to tell Rackspace, delete the server with this IP
address.
- This code worked fine for 34 out of 35 servers, 15 of which had
Cloud-Server-## names.  However, on the very last server (named
Cloud-Server-35), passing the String ipAddr = 10.208.232.65 , I got
this exception:
java.lang.IllegalArgumentException: Object 'Cloud-Server' doesn't
match dns naming constraints. Reason: Should be only lowercase.
- Full stack trace here: http://pastebin.com/PKuMqkqz (ServerDeletor:110
is the destroyNodesMatching() call listed above.)

RE: IllegalArgumentException when deleting node on Rackspace

[Zack Shoylev]

Hi Daniel,

I am still working on reproducing this, but have not been able to yet.
My (generic) advice for now would be to update to the latest jclouds (1.8.0) 
and let us know if this happens again.
I will let you know if we make some progress reproducing this.
Also, if I remember correctly, the group is usually based on how the server is 
named.

-Zack

From: Daniel Widdis [wid...@gmail.com]
Sent: Monday, August 11, 2014 1:40 PM
To: user@jclouds.apache.org
Subject: Re: IllegalArgumentException when deleting node on Rackspace

Zack,

I've tried to reproduce this multiple ways, but was unable to do so
(yet), but I have a theory about what happened (which is difficult to
intentionally replicate).

Reviewing the stack trace, I noticed that the exception is occurring
after the actual deletion of the node, in a subsequent call to
cleanUpIncidentalResourcesOfDeadNodes(), which apparently attempts to
delete orphaned groups.  I don't understand enough of the concept of a
group to know with certainty what's going on here, and my reading of
Rackspace documentation only implies security groups which I don't
have; I do, however, utilize their RackConnect system to connect my
cloud nodes to a managed server and it is possible that setup is
involved somehow.  After my nodes are created, they are subsequently
switched over (by Rackspace using their scripts) to RackConnect.

I don't know what resources were being cleaned up, but it may be a
possibility that the following factors contributed:
- I had previously created a server named Cloud-Server-35 which failed
to fully create (this happens about 1% of the time) and never got
switched over to the RackConnect group.
- I had issued the delete server command on the failed server, but the
node may have still been hanging around in the system; or at least, not
fully deleted by the time the second server got created.
- I had created another server named Cloud-Server-35 to replace the
failed one.  This one operated normally and was switched over to
RackConnect.
- When I went to delete the second server by IP, jclouds found leftover
resources associated with the original server, and attempted to clean
them up.
- Somehow in this cleanup, the code/API detected duplicate names/groups,
and attempted to rename one of the groups, encountering the exception
because the group name was mixed case.

Dan


On 8/11/14, 6:45 AM, Zack Shoylev wrote:
 Hi Daniel,

 Can this issue be reproduced consistently? Does the number of servers matter?

  From your stack traces it seems this code is called
 https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/predicates/validators/DnsNameValidator.java#L56
   so it could be some kind of weird interaction.

 
 From: Daniel Widdis [wid...@gmail.com]
 Sent: Saturday, August 09, 2014 1:57 PM
 To:user@jclouds.apache.org
 Subject: IllegalArgumentException when deleting node on Rackspace

 This issue may be related, or may be completely different than Sunil
 Shah's recent thread on 2-letter DNS names when creating GCE nodes, but
 ultimately boils down to the same exception at the same line of code.  I
 debated whether to reply to that thread or start a new one, and elected
 the latter.  I will take a look at JIRA where the issues may possibly be
 combined.

 My specific symptoms/situation:
 - jclouds 1.7.3
 - I had manually created (through website API) several Rackspace cloud
 servers, with the names Cloud-Server-30, Cloud-Server-31, etc.  These
 were the default names provided on the website and I chose (by virtue of
 laziness) to leave them as they were.
 - My code deletes these servers when it is done using them.  I do this
 by passing the private IP address of the server to this code:
 nodes = computeService
 .destroyNodesMatching(new PredicateNodeMetadata() {
   @Override
   public boolean apply(final NodeMetadata input) {
 return input.getPrivateAddresses().contains(ipAddr);
   }
 });

 - Note that my code never uses the server name, nor is even aware of
 it.  I simply want to tell Rackspace, delete the server with this IP
 address.
 - This code worked fine for 34 out of 35 servers, 15 of which had
 Cloud-Server-## names.  However, on the very last server (named
 Cloud-Server-35), passing the String ipAddr = 10.208.232.65 , I got
 this exception:
  java.lang.IllegalArgumentException: Object 'Cloud-Server' doesn't
 match dns naming constraints. Reason: Should be only lowercase.
 - Full stack trace here:http://pastebin.com/PKuMqkqz  (ServerDeletor:110
 is the destroyNodesMatching() call listed above.)

RE: How to write a Blob using an OutputStream?

[Zack Shoylev]

Your code seems fine. I have used 
http://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/output/ByteArrayOutputStream.html
 in the past to convert between stream types, but it seems like it doesn't 
match your case very well.

Note you might have to do writeBytesToBlob() before super.close(), but you can 
test that.

Let us know how it turns out!

From: Steve Kingsland [steve.kingsl...@opower.com]
Sent: Monday, August 04, 2014 9:22 PM
To: user@jclouds.apache.org
Subject: Re: How to write a Blob using an OutputStream?

OK, then it appears that my calling code (which would be difficult and risky to 
change) is incompatible with jclouds' BlobStore API: my caller wants to obtain 
an OutputStream for writing to the blob store, and jclouds wants to obtain an 
InputStream for reading the blob's content that should be written. Therefore, 
my only solution is to buffer the blob data, either in memory or on disk, 
before uploading it to the blob store.

Given that the documents I'm trying to write to the blob store will generally 
be small (1KB to 1MB), I'm going with a simple approach, for providing my 
caller with an OutputStream that they can use to write the blob's payload:

class BlobWritingByteArrayOutputStream extends java.io.ByteArrayOutputStream {

// these are all set in the constructor
private BlobStore blobStore;
private String containerName, blobName;

// the client will have to call this when he's finished writing, so this is 
our chance to upload the blob,
// now that we have the full payload in memory
@Override
public void close() throws IOException {
super.close();

writeBytesToBlob();
}

private void writeBytesToBlob() {
byte[] payload = toByteArray();

Blob blob = blobStore.blobBuilder(blobName)
 .payload(payload)
 .contentLength(payload.size)
 .build();
blobStore.putBlob(containerName, blob);
}
}

Aside from the weird inversion of control going on and the requirement that 
close() be called, I think something simple like this - to buffer the bytes 
being written before uploading them to the blob store - might work for me.

Thoughts?





Steve Kingsland

Senior Software Engineer
http://www.opower.com/

Opower


We’re hiring! See jobs herehttp://www.opower.com/careers


On Mon, Aug 4, 2014 at 9:05 PM, Andrew Gaul 
g...@apache.orgmailto:g...@apache.org wrote:
On Mon, Aug 04, 2014 at 08:46:37PM -0400, Steve Kingsland wrote:
 Here is Kevin's example using PipedInputStream and PipedOutputStream:
 https://groups.google.com/d/msg/jclouds/F2pCt9i7TSg/AUF4AqOO0TMJ

 I don't have the need to use different threads, though, so instead I'd do
 something like this?

This will not work; putBlob blocks until the operation completes.
Further you must use PipedInputStream/PipedOutputStream with separate
threads to avoid deadlock, as its Javadoc states:

http://docs.oracle.com/javase/7/docs/api/java/io/PipedInputStream.html

Unfortunately jclouds has poor support for asynchronous operations and
you can really only fake the desired behavior with various InputStream.
I strongly recommend trying to cast your solution into some kind of
ByteSource or InputStream.

 And then when close() or flush() is called on the returned OutputStream,
 the blob is uploaded like magic? Is it OK that I'm not setting the content
 length?

Some blobstores, specifically Amazon S3, require a content length, while
others such as OpenStack Swift do not.

--
Andrew Gaul
http://gaul.org/

RE: How to write a Blob using an OutputStream?

[Zack Shoylev]

With buffered streams, for example, close() causes buffers to be flushed (which 
is technically what you are doing).
So yes, you can get some serious exceptions when closing.


From: Steve Kingsland [steve.kingsl...@opower.com]
Sent: Tuesday, August 05, 2014 9:06 AM
To: user@jclouds.apache.org
Subject: Re: How to write a Blob using an OutputStream?

org.apache.commons.io.output.ByteArrayOutputStream sounds like a nice 
improvement over java.io.ByteArrayOutputStream (at least for my purposes), 
thanks Zack!

The problem I'm running into is actually with the caller's 
Closeables.closeQuietly(documentOutputStream); call. That catches any 
IOException that's thrown from close() and logs it, instead of throwing it. 
That won't work for me, since I won't know if there was an error writing to the 
blob store until close() is called on my OutputStream. I can of course change 
the caller to use different error-handling for closing the stream, but it makes 
me wonder if using the close() method to upload the blob is the right approach. 
If you're given an OutputStream to write to, you'd expect the real errors to 
come from the write() methods, and not the close() method, right?



Steve Kingsland

Senior Software Engineer
http://www.opower.com/

Opower


We’re hiring! See jobs herehttp://www.opower.com/careers


On Tue, Aug 5, 2014 at 7:21 AM, Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com wrote:
Your code seems fine. I have used 
http://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/output/ByteArrayOutputStream.html
 in the past to convert between stream types, but it seems like it doesn't 
match your case very well.

Note you might have to do writeBytesToBlob() before super.close(), but you can 
test that.

Let us know how it turns out!

From: Steve Kingsland 
[steve.kingsl...@opower.commailto:steve.kingsl...@opower.com]
Sent: Monday, August 04, 2014 9:22 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org
Subject: Re: How to write a Blob using an OutputStream?

OK, then it appears that my calling code (which would be difficult and risky to 
change) is incompatible with jclouds' BlobStore API: my caller wants to obtain 
an OutputStream for writing to the blob store, and jclouds wants to obtain an 
InputStream for reading the blob's content that should be written. Therefore, 
my only solution is to buffer the blob data, either in memory or on disk, 
before uploading it to the blob store.

Given that the documents I'm trying to write to the blob store will generally 
be small (1KB to 1MB), I'm going with a simple approach, for providing my 
caller with an OutputStream that they can use to write the blob's payload:

class BlobWritingByteArrayOutputStream extends java.io.ByteArrayOutputStream {

// these are all set in the constructor
private BlobStore blobStore;
private String containerName, blobName;

// the client will have to call this when he's finished writing, so this is 
our chance to upload the blob,
// now that we have the full payload in memory
@Override
public void close() throws IOException {
super.close();

writeBytesToBlob();
}

private void writeBytesToBlob() {
byte[] payload = toByteArray();

Blob blob = blobStore.blobBuilder(blobName)
 .payload(payload)
 .contentLength(payload.size)
 .build();
blobStore.putBlob(containerName, blob);
}
}

Aside from the weird inversion of control going on and the requirement that 
close() be called, I think something simple like this - to buffer the bytes 
being written before uploading them to the blob store - might work for me.

Thoughts?





Steve Kingsland

Senior Software Engineer
http://www.opower.com/

Opower


We’re hiring! See jobs herehttp://www.opower.com/careers


On Mon, Aug 4, 2014 at 9:05 PM, Andrew Gaul 
g...@apache.orgmailto:g...@apache.org wrote:
On Mon, Aug 04, 2014 at 08:46:37PM -0400, Steve Kingsland wrote:
 Here is Kevin's example using PipedInputStream and PipedOutputStream:
 https://groups.google.com/d/msg/jclouds/F2pCt9i7TSg/AUF4AqOO0TMJ

 I don't have the need to use different threads, though, so instead I'd do
 something like this?

This will not work; putBlob blocks until the operation completes.
Further you must use PipedInputStream/PipedOutputStream with separate
threads to avoid deadlock, as its Javadoc states:

http://docs.oracle.com/javase/7/docs/api/java/io/PipedInputStream.html

Unfortunately jclouds has poor support for asynchronous operations and
you can really only fake the desired behavior with various InputStream.
I strongly recommend trying to cast your solution into some kind of
ByteSource or InputStream.

 And then when close() or flush() is called on the returned OutputStream,
 the blob is uploaded like magic? Is it OK that I'm not setting

RE: How to create Winodows Compute with random or predefine password

[Zack Shoylev]

You might want to try something like

templateBuilder.options(overrideLoginUser(loginUser))

before you call templateBuilder.build();

As per this example:
https://github.com/jclouds/jclouds-examples/blob/master/compute-basics/src/main/java/org/jclouds/examples/compute/basics/MainApp.java

In any case I am thinking the idea is that you should not be able to modify the 
template (or its options) after you call build().

Let me know if this helps or not.
-Zack

From: Vineet Saini [switchcod...@gmail.com]

Sent: Wednesday, July 09, 2014 10:49 AM

To: user@jclouds.apache.org

Subject: Re: How to create Winodows Compute with random or predefine password













Andrew,



Based on information you provided and using existing 
example, I able to make some progress, but not able to resolved one thing, here 
is code snippet.





IterableModule modules = ImmutableSet. of( new SshjSshClientModule(), new 
SLF4JLoggingModule(), new EnterpriseConfigurationModule(), new 
BouncyCastleCryptoModule();



ComputeServiceContext context = ContextBuilder.newBuilder(aws-ec2) 
.credentials(identity, credential ) 
.modules(modules).buildView(ComputeServiceContext.class);





// use image id for windows Server 2003 R2 base =  ami-9050a9f8

TemplateBuilder templateBuilder = context.getComputeService().templateBuilder();


templateBuilder.imageId(us-east-1/ami-9050a9f8) .smallest() 
.locationId(us-east-1).os64Bit(true);





// Get Template

Template template = templateBuilder.build();







// Here try to override administrator password with static password,

// This does not work



template.getOptions().overrideLoginUser(Administrator);

template.getOptions().overrideLoginPassword(test123); 





// setup port for rdp



template.getOptions().inboundPorts(3389);





// Create Node: it wait for call to finish with admin user it setup. that take 
care of wait till ready.

Set?extends NodeMetadata nodes = computeService 
.createNodesInGroup(grouptest, 2, template);







NodeMetadata nodeMetadata = Iterables.getOnlyElement(nodes);




PasswordDataAndPrivateKey dataAndKey = new PasswordDataAndPrivateKey(   


context.unwrapApi(EC2Api.class).getWindowsApi().get().getPasswordDataInRegion(region,
 nodeMetadata.getId()), nodeMetadata.getCredentials().getPrivateKey());




WindowsLoginCredentialsFromEncryptedData f = 
context.utils().injector().getInstance(WindowsLoginCredentialsFromEncryptedData.class);





// In this case it need parameter type to be EncryptedPasswordDataAndPrivateKey 
instead of PasswordDataAndPrivateKey, not sure how to resolve that.


LoginCredentials credentials = f.apply(dataAndKey);






// get decrypted pass

logger.info(Login name: %s, credentials.getUser());

logger.info(Password: %s, credentials.getPassword());







Thanks for all the help.















On Tue, Jul 8, 2014 at 3:53 PM, Andrew Phillips 
andr...@apache.org wrote:






Also if can point me how to set default password for administrator user.

that would be helpful too.






I am not sure if that is possible. The EC2 Windows Security guide [1] says 
that, during initial install, it generates and sets a random password on the 
Administrator account, but with your own AMIs you may be able to do something 
different.



As regards retrieving the Administrator credentials: have you been able to try 
getting the WindowsApi [2] from the EC2Api [3] and using that? If you are 
working with a ComputeService view, you will need to unwrap the view to 
access the underlying API first
 [4].



Regards



ap



[1] 
https://aws.amazon.com/articles/1767

[2] 
http://javadocs.jclouds.cloudbees.net/org/jclouds/ec2/features/WindowsApi.html

[3] 
http://javadocs.jclouds.cloudbees.net/org/jclouds/ec2/EC2Api.html#getWindowsApi()

[4] 
http://jclouds.apache.org/start/concepts/#apis

RE: neutron refactoring - breaking changes

[Zack Shoylev]

Hi Jeremy,

That is exactly what I am suggesting, yes.

My initial reaction was that this would be very confusing, however, with the 
right deprecation messages it should be ok.


From: Jeremy Daggett [jeremy.dagg...@gmail.com]
Sent: Thursday, April 03, 2014 8:44 AM
To: d...@jclouds.apache.org
Cc: user@jclouds.apache.org
Subject: Re: neutron refactoring - breaking changes

Hi Zack,

Just to make this clear, we would have both packages:

org.jclouds.openstack.neutron.v2_0.*
org.jclouds.openstack.neutron.v2.*

... and then deprecate the v2_0 one in 1.8, to be removed in jclouds 2.0? WDYT?

/jd

On Wed, Apr 2, 2014 at 2:38 PM, Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com wrote:
I would hold off on separate renaming PRs and here is why:

One of the best ways to deprecate the existing neutron implementation is to 
deprecate it while having an alternative namespace with the refactored one. 
(deprecate and and add the refactored neutron in the same commit).  So this 
would be my suggestion.


From: Jeremy Daggett [jeremy.dagg...@gmail.commailto:jeremy.dagg...@gmail.com]
Sent: Wednesday, April 02, 2014 12:23 PM
To: user@jclouds.apache.orgmailto:user@jclouds.apache.org; 
d...@jclouds.apache.orgmailto:d...@jclouds.apache.org
Subject: Re: neutron refactoring - breaking changes

Hi Zack,

I feel that we should also change the Java package namespace as a part of this 
refactoring.  It does not follow the standard OpenStack API naming convention 
and their additive progression from API release to release.

All OpenStack APIs should be referenced with a major version only: v2 versus 
v2_0

That said, I suggest that we do this refactoring now, to unify the OpenStack 
version numbers across jclouds.  The package namespace should change from:

org.jclouds.openstack.neutron.v2_0 - org.jclouds.openstack.neutron.v2

In my experience with the APIs over the past several years, v2_0 doesn't 
really mean anything to me and it ties the implementation to a specific 
version. APIs are additive, so v2.2 is still the v2 API with new 
features/additions.

I would be more than happy to submit a PR to make this happen. We also should 
do this with the other incubating Glance APIs and any future work with 
OpenStack.

Comments, questions, concerns?

/jd


On Tue, Apr 1, 2014 at 8:47 AM, Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com
 wrote:
Hello,

There is a list of proposed changes to refactor neutron in 
jclouds-labs-openstack as detailed here:
http://www.mail-archive.com/dev@jclouds.apache.org/msg04477.html

I would like to hear back from users who are currently using the API - is this 
something we should or should not be doing? The problem is that we do not have 
a happy deprecation path for these changes. One good suggestion I've heard so 
far was to deprecate in 1.7 and put the breaking changes in 1.8.

Thanks!
Zack

RE: neutron refactoring - breaking changes

[Zack Shoylev]

I would hold off on separate renaming PRs and here is why:

One of the best ways to deprecate the existing neutron implementation is to 
deprecate it while having an alternative namespace with the refactored one. 
(deprecate and and add the refactored neutron in the same commit).  So this 
would be my suggestion.


From: Jeremy Daggett [jeremy.dagg...@gmail.com]
Sent: Wednesday, April 02, 2014 12:23 PM
To: user@jclouds.apache.org; d...@jclouds.apache.org
Subject: Re: neutron refactoring - breaking changes

Hi Zack,

I feel that we should also change the Java package namespace as a part of this 
refactoring.  It does not follow the standard OpenStack API naming convention 
and their additive progression from API release to release.

All OpenStack APIs should be referenced with a major version only: v2 versus 
v2_0

That said, I suggest that we do this refactoring now, to unify the OpenStack 
version numbers across jclouds.  The package namespace should change from:

org.jclouds.openstack.neutron.v2_0 - org.jclouds.openstack.neutron.v2

In my experience with the APIs over the past several years, v2_0 doesn't 
really mean anything to me and it ties the implementation to a specific 
version. APIs are additive, so v2.2 is still the v2 API with new 
features/additions.

I would be more than happy to submit a PR to make this happen. We also should 
do this with the other incubating Glance APIs and any future work with 
OpenStack.

Comments, questions, concerns?

/jd


On Tue, Apr 1, 2014 at 8:47 AM, Zack Shoylev 
zack.shoy...@rackspace.commailto:zack.shoy...@rackspace.com wrote:
Hello,

There is a list of proposed changes to refactor neutron in 
jclouds-labs-openstack as detailed here:
http://www.mail-archive.com/dev@jclouds.apache.org/msg04477.html

I would like to hear back from users who are currently using the API - is this 
something we should or should not be doing? The problem is that we do not have 
a happy deprecation path for these changes. One good suggestion I've heard so 
far was to deprecate in 1.7 and put the breaking changes in 1.8.

Thanks!
Zack

neutron refactoring - breaking changes

[Zack Shoylev]

Hello,

There is a list of proposed changes to refactor neutron in 
jclouds-labs-openstack as detailed here: 
http://www.mail-archive.com/dev@jclouds.apache.org/msg04477.html

I would like to hear back from users who are currently using the API - is this 
something we should or should not be doing? The problem is that we do not have 
a happy deprecation path for these changes. One good suggestion I've heard so 
far was to deprecate in 1.7 and put the breaking changes in 1.8.

Thanks!
Zack