Connect to hubiC
HI list, I’d like to interact with OVH’s hubiC storage. It’s based on OpenStack so it’s likely to be usable with jClouds. Unfortunately, they change the authentication process : auth is based on oauth2. What is the best approach to this ? Can I use oauth2 API then OpenStack API to interact with the storage or should a dedicated provider be developped ? Thanks for your help, Ionel -- Beicip-Franlab SA - 232 av. Napoléon Bonaparte - BP 2132-92502 Rueil-Malmaison Cedex Capital: EUR 6 000 000 - TVA FR 54 679 804 047- RCS Nanterre 679 804 047 This message and any attachments (the message) are confidential and intended solely for the addressees. Any unauthorised use, dissemination or reproduction is strictly prohibited. The sender does not accept liability for any errors or omissions in the contents of this message arising as a result of e-mail transmission.
Re: Life-span of a Context
HI Ignasi,
Thanks for your reply.
Ionel
On 02/15/2017 03:30 PM, Ignasi Barrera wrote:
You should try to keep the context open as long as possible, and reuse it when
needed.
A context itself does not hold any connection to external services. It just provides the container for the IoC framework used in jclouds
(Guice), and the different ExecutorService instances jclouds uses to perform async oeprations. creating the context creates all those
resourecs, and closing it releases them all, so you'd better open the context (it is meant to be thread-safe), and leave it open as long
as it makes sense, and try to reuse it where possible.
On 15 February 2017 at 15:23, GARDAIS Ionel <ionel.gard...@tech-advantage.com
<mailto:ionel.gard...@tech-advantage.com>> wrote:
Hi,
We are using JClouds on a server-side application.
That is, it is supposed to be long-lived, compared to a end-user
application that could be closed after several minutes/hours of use.
public byte[] getFile(String basePath, String containerName, String
blobPath) {
log.debug("load File in basePath: {}, containerName: {},
blobPath:{}", basePath, containerName, basePath);
Properties properties = new java.util.Properties();
properties.setProperty(FilesystemConstants.PROPERTY_BASEDIR, basePath);
BlobStoreContext context =
ContextBuilder.newBuilder("filesystem").overrides(properties)
.buildView(BlobStoreContext.class);
BlobStore blobStore = context.getBlobStore();
byte[] bFile = new byte[0];
Blob blob = blobStore.getBlob(containerName, blobPath);
if (null != blob) {
log.debug("Blob headers : {}", blob.getAllHeaders());
log.debug("Blob metadata : {}", blob.getMetadata());
try {
bFile = IOUtils.toByteArray(blob.getPayload().openStream());
} catch (IOException e) {
log.warn("Error reading file to input stream.", e);
}
} else {
log.warn("Blob at {}/{} does not exist.", containerName,
blobPath);
}
context.close();
return bFile;
}
The base directory is global for the whole application.
I wonder if the context and the blobstore should then be stored in a
singleton a retrieved when needed, instead of being instantiated
every time ?
(thus saving few milliseconds of works).
How long a context should remain open ?
How often should it be closed ?
Also, when using putBlob(), is the payload flushed directly or when the
context is closed ?
Thanks,
Ionel
--
TECH'advantage SA - 1 rue Isabey 92500 RUEIL MALMAISON
Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832
301 00027
--
Ionel GARDAIS
Tech'Advantage CIO - IT Team manager
--
TECH'advantage SA - 1 rue Isabey 92500 RUEIL MALMAISON
Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301
00027
Life-span of a Context
Hi,
We are using JClouds on a server-side application.
That is, it is supposed to be long-lived, compared to a end-user application
that could be closed after several minutes/hours of use.
public byte[] getFile(String basePath, String containerName, String
blobPath) {
log.debug("load File in basePath: {}, containerName: {}, blobPath:{}",
basePath, containerName, basePath);
Properties properties = new java.util.Properties();
properties.setProperty(FilesystemConstants.PROPERTY_BASEDIR, basePath);
BlobStoreContext context =
ContextBuilder.newBuilder("filesystem").overrides(properties)
.buildView(BlobStoreContext.class);
BlobStore blobStore = context.getBlobStore();
byte[] bFile = new byte[0];
Blob blob = blobStore.getBlob(containerName, blobPath);
if (null != blob) {
log.debug("Blob headers : {}", blob.getAllHeaders());
log.debug("Blob metadata : {}", blob.getMetadata());
try {
bFile = IOUtils.toByteArray(blob.getPayload().openStream());
} catch (IOException e) {
log.warn("Error reading file to input stream.", e);
}
} else {
log.warn("Blob at {}/{} does not exist.", containerName, blobPath);
}
context.close();
return bFile;
}
The base directory is global for the whole application.
I wonder if the context and the blobstore should then be stored in a singleton a retrieved when needed, instead of being instantiated every
time ?
(thus saving few milliseconds of works).
How long a context should remain open ?
How often should it be closed ?
Also, when using putBlob(), is the payload flushed directly or when the context
is closed ?
Thanks,
Ionel
--
TECH'advantage SA - 1 rue Isabey 92500 RUEIL MALMAISON
Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301
00027
directoryExists vs blobExists
Hi, Got a question about using the filesystem API. As a blob wrote using putBlob(« container », "folder/file.t xt »); directoryExists(« container », « folder") returns true blobExists(« container », « folder") returns false blobExists(« container », « folder/") returns false As directoryExists() is deprecated, what should be used ? Thanks, Ionel -- 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301 BEGIN:VCARD VERSION:3.0 FN:GARDAIS\, Ionel N:GARDAIS;Ionel;;; ADR;TYPE=work,postal,parcel:;;1 Rue Isabey;RUEIL MALMAISON;IdF;92500;FRANCE TEL;TYPE=work,voice:0147088131 EMAIL;TYPE=internet:ionel.gard...@tech-advantage.com URL;TYPE=work:http://www.techad.fr ORG:TECH advantage TITLE:CIO REV:2016-12-16T11:52:22Z UID:5a4525af-5d0c-4a32-a77b-c565580b116e:114277 END:VCARD
Re: directoryExists vs blobExists
Thanks for the clarification. So directoryExists(container, directory) is to be replaced by list(container, prefix(directory)).isEmpty() -- Ionel GARDAIS Tech'Advantage CIO - IT Team manager - Mail original - De: "Andrew Gaul" <g...@apache.org> À: "user" <user@jclouds.apache.org> Envoyé: Mardi 19 Septembre 2017 10:17:31 Objet: Re: directoryExists vs blobExists Prefix allows clients to filter results which only match a prefix. For example, prefix a/ with a/1, a/2 and b/3 keys would only match a/1 and a/2. Delimiter filters results based on a partial suffix. For example, delimiter / with the previous keys would return a and b. Prefix and delimiter have more utility and match how blobstores actually work. File systems have an actual hierarchy such that creating a/b will create two entries and must prohibit operations like removing a before removing a/b. Object stores remove these invariants to increase scalability. jclouds previously mapped directory operations onto pseudo-file system operations, inconsistently and poorly. On Tue, Sep 19, 2017 at 08:37:26AM +0200, GARDAIS Ionel wrote: > Thanks Andrew. > > Do I understand correctly that "prefix and delimiter" is "path ending with a > /" (like in "folder/") ? > > Regards, > -- > Ionel GARDAIS > Tech'Advantage CIO - IT Team manager > > - Mail original - > De: "Andrew Gaul" <g...@apache.org> > À: "user" <user@jclouds.apache.org> > Envoyé: Lundi 18 Septembre 2017 19:53:08 > Objet: Re: directoryExists vs blobExists > > jclouds deprecated and will remove support for directories from the > portable abstraction since most object stores do not not support them. > Directories should still work through the provider level but I recommend > using the prefix and delimiter options instead. Tracking issue: > > https://issues.apache.org/jira/browse/JCLOUDS-1066 > > On Mon, Sep 18, 2017 at 05:26:31PM +0200, GARDAIS Ionel wrote: > > Hi, > > > > Got a question about using the filesystem API. > > As a blob wrote using putBlob(« container », "folder/file.t xt »); > > > > directoryExists(« container », « folder") returns true > > > > blobExists(« container », « folder") returns false > > blobExists(« container », « folder/") returns false > > > > As directoryExists() is deprecated, what should be used ? > > > > Thanks, > > Ionel > > > > -- > > > > 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON > > > > Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301 > > > > > BEGIN:VCARD > > VERSION:3.0 > > FN:GARDAIS\, Ionel > > N:GARDAIS;Ionel;;; > > ADR;TYPE=work,postal,parcel:;;1 Rue Isabey;RUEIL MALMAISON;IdF;92500;FRANCE > > TEL;TYPE=work,voice:0147088131 > > EMAIL;TYPE=internet:ionel.gard...@tech-advantage.com > > URL;TYPE=work:http://www.techad.fr > > ORG:TECH advantage > > TITLE:CIO > > REV:2016-12-16T11:52:22Z > > UID:5a4525af-5d0c-4a32-a77b-c565580b116e:114277 > > END:VCARD > > > -- > Andrew Gaul > http://gaul.org/ > -- > 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON > Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301 > -- Andrew Gaul http://gaul.org/ -- 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301
Re: directoryExists vs blobExists
Thanks Andrew. Do I understand correctly that "prefix and delimiter" is "path ending with a /" (like in "folder/") ? Regards, -- Ionel GARDAIS Tech'Advantage CIO - IT Team manager - Mail original - De: "Andrew Gaul" <g...@apache.org> À: "user" <user@jclouds.apache.org> Envoyé: Lundi 18 Septembre 2017 19:53:08 Objet: Re: directoryExists vs blobExists jclouds deprecated and will remove support for directories from the portable abstraction since most object stores do not not support them. Directories should still work through the provider level but I recommend using the prefix and delimiter options instead. Tracking issue: https://issues.apache.org/jira/browse/JCLOUDS-1066 On Mon, Sep 18, 2017 at 05:26:31PM +0200, GARDAIS Ionel wrote: > Hi, > > Got a question about using the filesystem API. > As a blob wrote using putBlob(« container », "folder/file.t xt »); > > directoryExists(« container », « folder") returns true > > blobExists(« container », « folder") returns false > blobExists(« container », « folder/") returns false > > As directoryExists() is deprecated, what should be used ? > > Thanks, > Ionel > > -- > > 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON > > Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301 > > BEGIN:VCARD > VERSION:3.0 > FN:GARDAIS\, Ionel > N:GARDAIS;Ionel;;; > ADR;TYPE=work,postal,parcel:;;1 Rue Isabey;RUEIL MALMAISON;IdF;92500;FRANCE > TEL;TYPE=work,voice:0147088131 > EMAIL;TYPE=internet:ionel.gard...@tech-advantage.com > URL;TYPE=work:http://www.techad.fr > ORG:TECH advantage > TITLE:CIO > REV:2016-12-16T11:52:22Z > UID:5a4525af-5d0c-4a32-a77b-c565580b116e:114277 > END:VCARD -- Andrew Gaul http://gaul.org/ -- 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301
