good for so long. So, does jclouds
automatically reconnect to the provider if the session times out?
John
--
Andrew Gaul
http://gaul.org/
these as singletons,
and send data as appropriate.
Thanks,
John
On Sun, Jan 26, 2014 at 9:44 PM, Andrew Gaul g...@apache.org wrote:
Instantiating a BlobStoreContext performs several requests such
authentication and populating metadata like bucket locations. This
process has some overhead so
Does anyone use the jclouds Clojure bindings, specifically
blobstore2.clj? These have not seen many changes over the years and
have made it harder to evolve the underlying Java jclouds APIs, e.g.,
https://github.com/jclouds/jclouds/pull/44 . Can someone volunteer to
maintain these?
--
Andrew
upload?
Thank you
--
Andrew Gaul
http://gaul.org/
, prefer not to spread our finite volunteer resources across
maintaining multiple stable branches and validating releases. A few
users have asked for a 1.5.11 release as well! We should push them to
upgrade to a more recent version of jclouds instead.
--
Andrew Gaul
http://gaul.org/
On Wed, Feb 19, 2014 at 01:04:43PM -0800, Andrew Gaul wrote:
Does anyone use the jclouds Clojure bindings, specifically
blobstore2.clj? These have not seen many changes over the years and
have made it harder to evolve the underlying Java jclouds APIs, e.g.,
https://github.com/jclouds/jclouds
at the existing Google support which
implements OAuth. This will give you a good head start on any
implementation. Good luck!
--
Andrew Gaul
http://gaul.org/
in some
persistent store, and vend signed URLs to clients when they request and
have permission to read/write an given object. jclouds does not offer
any support for LDAP/AD and the greater Java ecosystem generally has
poor support for this.
--
Andrew Gaul
http://gaul.org/
developing a commercial product
which includes jclouds code. Their developers actively participate in
jclouds development and contribute code. Please contact Maginatics at
http://maginatics.com/ for information on their offering.
--
Andrew Gaul
http://gaul.org/
this
Regards
Nipun Udara
--
Andrew Gaul
http://gaul.org/
.
--
Andrew Gaul
http://gaul.org/
.
jclouds should provide real support for aborting putBlob. Can you open
a JIRA issue to track this? It intersects with some of our
AsyncBlobStore discussion.
--
Andrew Gaul
http://gaul.org/
On Wed, May 28, 2014 at 11:57:18AM -0700, Andrew Gaul wrote:
jclouds presently supports Java 6, 7, and 8 which imposes extra
development costs and prevents uptake of new language and library
features including try-with-resources, NIO.2, and HTTP client
improvements. Oracle ceased public
/PipedOutputStream if you really
must use an OutputStream?
--
Andrew Gaul
http://gaul.org/
*
Senior Software Engineer
*Opower * http://www.opower.com/
*We’re hiring! See jobs here http://www.opower.com/careers *
On Mon, Aug 4, 2014 at 7:53 PM, Andrew Gaul g...@apache.org wrote:
On Mon, Aug 04, 2014 at 04:39:15PM -0400, Steve Kingsland wrote:
I'm trying to use
If you use 1.7.0 or newer, can you share the failing operations with
wire logs? Also make sure you use the generic s3 provider and not the
aws-s3 specific provider.
--
Andrew Gaul
http://gaul.org/
? The jclouds filesystem provider
does not provide any indexing; the underlying operating system provides
this.
--
Andrew Gaul
http://gaul.org/
backend or run create local S3 blobstore using the
filesystem backend. I appreciate any feedback:
https://github.com/andrewgaul/s3proxy
--
Andrew Gaul
http://gaul.org/
will upgrade to jclouds 2.0 as soon as we release it. Thanks
for the feedback!
--
Andrew Gaul
http://gaul.org/
on device
It does not say *which* device is out of space. Could you check on your
target system to find out which device has no space left [1]?
ap
[1] https://kb.iu.edu/d/agfe
--
Andrew Gaul
http://gaul.org/
/jira/browse/JCLOUDS-427
--
Andrew Gaul
http://gaul.org/
confirm.
Thanks and Regards
Manisha
--
~Regards
*Manisha Eleperuma*
Software Engineer, Cloud TG
WSO2, Inc.: http://wso2.com
lean.enterprise.middleware
*blog: http://manisha-eleperuma.blogspot.com/
http://manisha-eleperuma.blogspot.com/*
--
Andrew Gaul
http://gaul.org/
)
Try closing the results of bloblgetPayload().getInput() (lines 428 and
431). You can do this via Java 7 try-with-resources or with
ByteStreams2.toByteArrayAndClose. These InputStream are actually
FileInputStream and likely the cause of your leaks.
--
Andrew Gaul
http://gaul.org/
but nothing was downloaded.
Thank you.
Regards,
SB
--
Andrew Gaul
http://gaul.org/
And now the equivalent for Swift:
https://github.com/bouncestorage/swiftproxy
On Tue, Aug 12, 2014 at 12:02:01AM -0700, Andrew Gaul wrote:
Over the last few weeks I have hacked up S3Proxy, which provides an S3
interface on top of the jclouds BlobStore portable abstraction. This
enables new
Anyone interested in jclouds attending OpenStack Summit in Vancouver
this week? Let's meet either for lunch or after hours!
--
Andrew Gaul
http://gaul.org/
with
jclouds 1.8.0? By bulk delete I mean, one HTTP request to delete a
collection of object names.
Thanks,
Forrest T.
--
Andrew Gaul
http://gaul.org/
; I analized some releases and i noticed that not using java 7 features like
> multi catch and in java 8 do not use lambda expressions and others
> features, so i came by this email to know why the developers not using this
> features?
--
Andrew Gaul
http://gaul.org/
point me to a page which has this information.
>
>
> Thanks
> Trupti
--
Andrew Gaul
http://gaul.org/
compatibility or u meant JCloud instead?
>
> > On 28-Oct-2015, at 11:37 am, Andrew Gaul <g...@apache.org> wrote:
> >
> > OpenStack Swift maintains backwards compatibility with previous releases
> > so the existing code should work with newer Swift ve
request signature so that when a
> non-java client retrieves the resource, it will download with the provided
> file name?
>
> V/r,
>
> Jeremy
--
Andrew Gaul
http://gaul.org/
rred chunk plus
> the chunk number?
>
> Maybe someone can shed some light on this :).
>
> Thanks
> Veit
>
--
Andrew Gaul
http://gaul.org/
using within the API?
>
> I can assuming running a script to access the dependencies' version but my
> goal is to print out the jclouds version number as a header when I start a
> debug trace.
>
> Any thoughts?
>
> Thanks,
> Forrest T.
--
Andrew Gaul
http://gaul.org/
together here?
> >
> > As Andrew G's answer hinted, I could imagine using jclouds in a proxy-like
> > setup, but since jclouds itself expects to call an OpenStack API, I can't
> > immediately see how it would be used to create a "standalone" OpenStack
> > implementation.
> >
> > Regards
> >
> > ap
> >
--
Andrew Gaul
http://gaul.org/
sonally prefer to get something like a
> ContextClosedException instead of a method call that never returns.
>
> Regards,
> Bram
>
>
> On Fri, Jun 10, 2016 at 6:34 PM Andrew Gaul <g...@apache.org> wrote:
>
> > Bram, you cannot use the BlobStoreContext
signature for the blobstore APIs
> > will be much appreciated. *
> >
> > *NB: With the last commit mentioned in the above reply, we do not see the
> > region not supported exception anymore. *
> >
> > *Thanks!*
> >
> > *Regards*
> >
>
.contentDisposition(objectname).build();
>
> // Upload the Blob
> String eTag = blobStore.putBlob(containername, blob);
> System.out.println(eTag);}}
> Error : requested location eu-central-1, which is not in the configured
> locations
> Solution to rectify the issue required
>
> RegardsArchana
--
Andrew Gaul
http://gaul.org/
ved by using executorservice.
> The question here is, does multipartUpload i.e uploading of each part is
> happening in parallel ?
> Does sequential upload of part deprecated ?
> RegardsArchana
>
>
>
> On Saturday, 4 February 2017, 1:30, Andrew Gaul <g...@apache.org&
ket policies.
>
> Does jClouds blob store support bucket policies? I couldn't find any
> mentions in the docs and my source code exploration didn't find
> anything that fits the description. If this exists an example would
> be most welcome.
>
> Best regards, cen
>
>
--
Andrew Gaul
http://gaul.org/
errors:
>
> 1) No implementation for org.jclouds.openstack.swift.v1.SwiftApi was bound.
> while locating org.jclouds.openstack.swift.v1.SwiftApi
>
>
> The same piece of code works perfectly when run with a live swift
> instance and "openstack-swift" as provider.
> Best,
> Aleksandra
&g
e[1024];
>
> int length = -1;
>
> while ((length = in.read(bytes, 0, 1024)) != -1) {
>
> fout.write(bytes, 0, length);
>
> }
>
> fout.flush();
>
> fout.close();
>
> in.close();
>
> System.out.println("Download complete...");
>
>
>
--
Andrew Gaul
http://gaul.org/
ir/blob-3
>
> * Azureblob
> - blob-1
> - blob-2
> - dir
> - dir/blob-3
>
> On a separate note, there is a discrepancy between aws and azure directories
> since we have ‘/‘ in the returned results of the former. This was not the
> case in 1.9
day, 26 October 2016, 10:18, Andrew Gaul <g...@apache.org> wrote:
>
>
> Can you provide more details on your use case, e.g., which provider?
> All providers use an identity and credential. Long ago someone asked
> about HTTP basic access authentication which we do not support
Hi
>
> Is there any way to authenticate blob store using private key alone,
> instead of passing credentials(identity, key) ? Does jclouds support that
> kind of authentication ?
>
> RegardsArchana
--
Andrew Gaul
http://gaul.org/
.list returns:
>
> * blob1
> * blob2
> * dir/blob3
>
> Thank you,
> Ashkan
--
Andrew Gaul
http://gaul.org/
nce we open a stream we can not be able to change the
> content length of blob.
>
> I want to know is there a way to change the content length of MataData
> after compression of all the slices completes.
>
> Regards,
> Dileep
--
Andrew Gaul
http://gaul.org/
ne example Java code to upload/download file from HDFS file
> system using Jclouds.
>
> If you can help me that will be great.
> Thanks in advance.
--
Andrew Gaul
http://gaul.org/
penstack.org/user-guide/cli-swift-large-object-creation.html
> RegardsArchana
>
--
Andrew Gaul
http://gaul.org/
t; https://github.com/jclouds/jclouds/blob/master/apis/openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/binders/BindManifestToJsonPayload.java
>
> RegardsArchana
--
Andrew Gaul
http://gaul.org/
> RegardsArchana
>
--
Andrew Gaul
http://gaul.org/
ompared 1.9.1 jcloluds CommonSwiftClient.java:putObjectManifest with
> jclouds 2.0 StaticLargeObject.java:replaceManifest, I figured out that
> "X-Object-Manifest" is used as required in jclouds 1.9.1 and that has been
> omitted in 2.0
> Please share sample test case th
/tokens HTTP/1.1 failed with
> response: HTTP/1.1 400 Bad Request; content:
> [{"badRequest":{"code":400,"message":"Invalid json request body"}}]
>
> In which they explain 400 as “Missing required parameters. This error also
> occurs if you
gt; RegardsArchana
>
> On Friday, 7 April 2017, 1:00, Andrew Gaul <g...@apache.org> wrote:
>
>
> jclouds is a community project so DLO support will be added when someone
> contributes it, likely you. I would look at how other libraries bridge
> this gap and submit a pull
:38AM +0200, Ignasi Barrera wrote:
> Given the silence in this thread, I'd say go ahead and remove it.
>
> On 3 July 2017 at 23:47, Andrew Gaul <g...@apache.org> wrote:
> > Does anyone use the EMC Atmos provider? As I understand it, EMC has not
> > developed this in so
contributions!
On Fri, Jul 21, 2017 at 04:00:20AM +, Archana C wrote:
> Hi,
> jClouds version till 2.0.1 supports signature V2 for S3 compatible object
> storage. Is there a plan to support signature V4 for S3 compatible object
> storage ?
> RegardsArchana
--
Andrew Gaul
http://gaul.org/
-part upload and server-side filtering. The recent
disappearance of Atmos Online leaves me unable to test the provider for
the jclouds 2.0.2 release. I would like to remove the Atmos provider if
no one is using it.
--
Andrew Gaul
http://gaul.org/
These bindings add to jclouds maintenance and make evolving core APIs
harder. Further they really belong outside of the jclouds repository.
I would like to remove these in 2.1.0 unless someone makes a compelling
case.
--
Andrew Gaul
http://gaul.org/
lier for 1.9* version this was not
> issue as MultipartUploadSlicingAlgorithm.java was not made final class.
>
> Thanks,
> Rishika
--
Andrew Gaul
http://gaul.org/
in your applications.
On Tue, Mar 21, 2017 at 08:00:00PM -0400, Andrew Gaul wrote:
> Honestly, we could promote GCS to core today -- let's skip the
> intermediate step. The URL signers are not much work which I could
> implement some time next week. The InputStream hack uses multipart
rprise.middleware
>
> m: +94773325954
> e: lahi...@wso2.com b: http://lahiruwrites.blogspot.com/
> in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
--
Andrew Gaul
http://gaul.org/
le get removed, the segments
> remains.
> Is there something missing ?
> RegardsArchana
--
Andrew Gaul
http://gaul.org/
I filed https://bugs.launchpad.net/swift/+bug/1691523 to track this.
On Wed, May 17, 2017 at 10:53:56AM -0700, Andrew Gaul wrote:
> This is likely a limitation of the Swift API.
>
> On Wed, May 17, 2017 at 12:10:47PM +, Archana C wrote:
> > Hi
> > I am trying t
ed way for bulk-delete DLO uploads.
> RegardsArchana
>
> On Wednesday, 17 May 2017, 23:24, Andrew Gaul <g...@apache.org> wrote:
>
>
> This is likely a limitation of the Swift API.
>
> On Wed, May 17, 2017 at 12:10:47PM +, Archana C wrote:
> > H
YPE=work,voice:0147088131
> EMAIL;TYPE=internet:ionel.gard...@tech-advantage.com
> URL;TYPE=work:http://www.techad.fr
> ORG:TECH advantage
> TITLE:CIO
> REV:2016-12-16T11:52:22Z
> UID:5a4525af-5d0c-4a32-a77b-c565580b116e:114277
> END:VCARD
--
Andrew Gaul
http://gaul.org/
lder/") ?
>
> Regards,
> --
> Ionel GARDAIS
> Tech'Advantage CIO - IT Team manager
>
> - Mail original -
> De: "Andrew Gaul" <g...@apache.org>
> À: "user" <user@jclouds.apache.org>
> Envoyé: Lundi 18 Septembre
TYPE=work,voice:0147088131
> EMAIL;TYPE=internet:ionel.gard...@tech-advantage.com
> URL;TYPE=work:http://www.techad.fr
> ORG:TECH advantage
> TITLE:CIO
> REV:2017-09-28T12:56:01Z
> UID:5a4525af-5d0c-4a32-a77b-c565580b116e:114277
> END:VCARD
--
Andrew Gaul
http://gaul.org/
to get a provider's suggested
> delimiters for "directory emulation" (like the forward slash in the above
> example) and "file extension emulation" (like the period in the above
> example)?
--
Andrew Gaul
http://gaul.org/
On Fri, Aug 17, 2018 at 10:04:29PM -, john.calc...@gmail.com wrote:
> I'm calling putBlob to put a blob to an S3 ninja server. The server is
> returning a ridiculous 500 server error response page - it contains a ton of
> javascript and the jclouds SAX parser is choking on the content of a
gt; So I would like to know, If there is a way to remove this HttpHeader in
> > Jcloud? Please provide some sample, if it has..
> >
> >
> >
> > Regards
> >
> > Pratheesh
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
--
Andrew Gaul
http://gaul.org/
Most of the magic lies in RestAnnotationProcessor if you
want to see how it works.
--
Andrew Gaul
http://gaul.org/
an exception for a missing
> object and, not knowing what it's for, just retry the operation.
All blobstores allow deletes of non-existent keys.
--
Andrew Gaul
http://gaul.org/
lobstores like Ceph can address this
issue with conditional PUT but this is not supported elsewhere.
--
Andrew Gaul
http://gaul.org/
-host as documented here:
https://github.com/gaul/s3proxy/blob/master/src/main/java/org/gaul/s3proxy/S3ProxyConstants.java#L50
If this does not help, could you follow up with a GitHub issue at
https://github.com/gaul/s3proxy since this does not relate to the
jclouds backend?
--
Andrew Gaul
http://gaul.org/
s backend.
[1] https://github.com/gaul/s3proxy
--
Andrew Gaul
http://gaul.org/
to require Guava 21 which in turn requires Java 8.
Unfortunately I do not see another way forward and this is an issue of
when not if. Any comments on this proposed change?
--
Andrew Gaul
http://gaul.org/
jclouds does not support Azure file storage today. We would happily
accept pull requests to add this! This is simpler than Azure blob which
has its own protocol; Azure file uses SMB2 so we only need a
provisioning API and some code to hook up to Azure compute.
--
Andrew Gaul
http://gaul.org/
gt; >>> On 21 June 2018 at 11:58, Ranjith R wrote:
> >>>
> >>>> I was looking at https://issues.apache.org/jira/browse/JCLOUDS-480 and
> >>>> it talks about the default signing for AWS being v4 and other s3 clones
> >>>> being v2. I just want to know if I can use v4 for a s3 clone? Is there
> >>>> any example that I can look at?
> >>>>
> >>>> Thanks,
> >>>> Ranjith
> >>>>
> >>>> On Mon, Jun 18, 2018 at 7:21 PM Ranjith R wrote:
> >>>>
> >>>>> Hi All - I know signature v4 signing is implemented for Amazon S3
> >>>>> (aws-s3). Just wanted to know if I can use v4 signing for a non amazon
> >>>>> cloud which supports S3 API and sigV4 (s3). If it does, what changes
> >>>>> should be done from the client side?
> >>>>>
> >>>>> Thanks,
> >>>>> Ranjith
> >>>>>
> >>>>
> >>>
> >
--
Andrew Gaul
http://gaul.org/
hese 500 errors are caused either by
> disk-full or container-quota-full conditions.
Sorry for the late reply, but I have also heard users complain about 500
errors when overwriting existing blobs. Apparently it can be configured
in a write-once manner. Unfortunately I don't have any direct
experience with HGST.
--
Andrew Gaul
http://gaul.org/
"container","id");
>
> doing the nullcheck myself and then getting the inputstream from the blob's
> payload.
Sorry for the late reply but I find the implementation of
BlobStore.streamBlob confusing since it lives in the portable
abstraction yet only has one implementation. I recommend using getBlob
which actually does stream the payload.
--
Andrew Gaul
http://gaul.org/
d and merged. Apparently
> changes need to go into master.
> How long before master becomes stable and can be used for building/
> releasing s3proxy?
> Anyways, I'll give it a shot.
> Best,
--
Andrew Gaul
http://gaul.org/
//github.com/vyasa-analytics/s3proxy/pull/1
Could you send this to https://github.com/gaul/s3proxy ?
It is unfortunate that clients do not fail back from v2 to v1
automatically like some do for v4 to v2 authentication, but S3Proxy
should just add support for v2.
--
Andrew Gaul
http://gaul.org/
welcome your help and feedback. For more information on how to report
problems, and to get involved, visit the project website at:
https://jclouds.apache.org/
The Apache jclouds Team
--
Andrew Gaul
http://gaul.org/
) is supported by
> > jclouds? My preliminary research says no, but if anyone else knows better,
> > I'd appreciate it.
> >
> > John
> >
--
Andrew Gaul
http://gaul.org/
G-ACCESS-KEY should be a clue to instead send the request to GCS.
>
> So, my question: Is this migration not implemented in blobstore or am
> I just calling the example incorrectly?
You should set the provider to "s3" instead of "aws-s3" and set the
endpoint to https:/
e.getBlob takes an optional argument that you can populate via:
new GetOptions().range(start, size)
Note that this range is inclusive.
--
Andrew Gaul
http://gaul.org/
welcome your help and feedback. For more information on how to report
problems, and to get involved, visit the project website at:
https://jclouds.apache.org/
The Apache jclouds Team
--
Andrew Gaul
http://gaul.org/
has configured for the AWS S3
> provider.
>
> Thanks,
> John
--
Andrew Gaul
http://gaul.org/
05-21 16:53:12.020 +,36635150704660 {} DEBUG o.j.h.i.HttpWire [user
> thread 18] over limit 33554416/262144: wrote temp file
>
> I'm concerned about the trailing messages - "over limit ... wrote temp file"
> - what does this mean?
>
> Thanks,
> John
--
Andrew Gaul
http://gaul.org/
; storage_account_name = ""
> file_system (or container) = ""
>
> Note: Data lake uses domain "dfs.core.windows.net" instead of "
> blob.core.windows.net". I used to change the endpoint.
>
> I found the sample below for blob connectivity but its not working.
> https://jclouds.apache.org/guides/azure-storage/
>
> Regards,
> Rauf
--
Andrew Gaul
http://gaul.org/
und this ancient request in legacy depot:
> https://github.com/jclouds/legacy-jclouds/issues/1280
>
> Has anything like this ever been added to jclouds?
>
> Thanks
> Timo
--
Andrew Gaul
http://gaul.org/
dependencies have troubled users? I would like
jclouds 2.3.0 to address all of our common dependency issues since we
will break some (and unbreak more) users due to the Guava upgrade.
--
Andrew Gaul
http://gaul.org/
[?:1.8.0_181]
> at java.nio.file.Files.getPosixFilePermissions(Files.java:2004)
> ~[?:1.8.0_181]
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:686)
> ~[filesystem-2.2.1.jar:2.2.1]
> ... 14 more
> ```
>
> JClouds seems to be having trouble reading the existing POSIX attributes on
> the file it just wrote (so it can modify them).
>
> Has anyone else seen this, or have there been changes between 2.1 and 2.2.0
> to the filesystem implementation?
--
Andrew Gaul
http://gaul.org/
ional
> information you think might be relevant. My hope is that I can do the work,
> saving you that effort, if you will spend just a few minutes jotting down how
> you would approach the patch since you obviously understand the code better
> than most.
>
> Thanks very much fo
rovider with path-based naming or
> > 2) coerce jclouds into using the generic S3 provider with V4 signing
> >
> > Thanks in advance.
> > John
> >
--
Andrew Gaul
http://gaul.org/
s logs the creation of the container and
> the test-container directory.
>
> We upgraded from 2.2.0 to 2.2.1 recently and this started happening. The test
> has been in place for years, without seeing this issue. Suddenly, after
> upgrading, it starts happening.
>
> Thanks,
> John
--
Andrew Gaul
http://gaul.org/
2.3.0-SNAPSHOT:
https://jclouds.apache.org/start/install/#maven
You may need to wait a few hours for the latest snapshot to be
generated.
--
Andrew Gaul
http://gaul.org/
I opened a pull request:
https://github.com/apache/jclouds/pull/94
On Fri, Jan 08, 2021 at 10:01:56AM +0900, Andrew Gaul wrote:
> Hi John, the code in LocalBlobStore.putBlob looks like this:
>
> String eTag = storageStrategy.putBlob(containerName, blob);
> s
should keep chipping away at it.
I do think that using Guava in the public interfaces makes it difficult
to shade this dependency and thus ListenableFuture might be an easy
place to start.
--
Andrew Gaul
http://gaul.org/
trates. My
experience with these automatic tools is that they work better for
applications than frameworks. We would also want to align with other
Apache projects -- do we have some similar infrastructure already?
--
Andrew Gaul
http://gaul.org/
p instead of Map to hint at immutability but this
unnecessarily makes the interface reliant on Guava. I would have to
look at interface-maven-plugin again to see everything but unfortunately
I ran into issues with the way jclouds uses public access across
subpackages.
--
Andrew Gaul
http://gaul.org/
1 - 100 of 116 matches
Mail list logo
