Here is Kevin's example using PipedInputStream and PipedOutputStream:
https://groups.google.com/d/msg/jclouds/F2pCt9i7TSg/AUF4AqOO0TMJ
I don't have the need to use different threads, though, so instead I'd do
something like this?
public OutputStream getOutputStream(String containerName, String
resourceName) throws IOException {
PipedInputStream in = new PipedInputStream();
PipedOutputStream out = new PipedOutputStream(in);
BlobStore blobStore = this.blobStoreContext.getBlobStore();
Blob blob = blobStore.blobBuilder(resourceName).payload(in).build();
blobStore.putBlob(containerName, blob);
return out;
}
And then when close() or flush() is called on the returned OutputStream,
the blob is uploaded like magic? Is it OK that I'm not setting the content
length?
*Steve Kingsland*
Senior Software Engineer
*Opower * <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers> *
On Mon, Aug 4, 2014 at 8:38 PM, Steve Kingsland <[email protected]>
wrote:
> Thanks Andrew, I will. Can you provide any guidance, pseudo-code,
> examples, etc. on how I would use a PipedOutputStream to buffer the content
> that's being written, and upload it to a BlobStore?
>
> To put it differently: how can I use these classes to return an
> OutputStream that is capable of putting a blob in a blob store, all by
> itself?
>
>
>
> *Steve Kingsland*
>
> Senior Software Engineer
>
> * Opower * <http://www.opower.com/>
>
>
> *We’re hiring! See jobs here <http://www.opower.com/careers> *
>
>
> On Mon, Aug 4, 2014 at 8:30 PM, Andrew Gaul <[email protected]> wrote:
>
>> Please look at PipedInputStream/PipedOutputStream which should address
>> this use case.
>>
>> On Mon, Aug 04, 2014 at 08:10:49PM -0400, Steve Kingsland wrote:
>> > My use case is:
>> >
>> > 1. the calling code is generating content in memory, and wants an
>> > OutputStream to write it to (currently it's going to disk);
>> >
>> > 2. the putBlob() method wants a btye[], InputStream, etc. that it can
>> read
>> > from.
>> >
>> > My problem is that *both* parties want to control the transaction. Here
>> is
>> > what my calling code looks like:
>> >
>> > OutputStream documentOutputStream = null;
>> > try {
>> > documentOutputStream = this.
>> > *documentResourceFactory.getDocumentOutputStream(documentPath);*
>> >
>> > renderAndWriteDocument(renderContext, documentOutputStream);
>> > }
>> > catch (IOException e) {
>> > ...
>> > }
>> > finally {
>> > Closeables.closeQuietly(documentOutputStream);
>> > }
>> >
>> > I'm trying to create an implementation of DocumentResourceFactory that
>> > returns an OutputStream for writing the document to an Object Store
>> using
>> > jclouds, instead of writing it to the local file system. I guess a
>> > stream-based API isn't really supported for writing to object stores...
>> >
>> > In my case, the files are small enough that I'm OK buffering them in
>> > memory. So what I'm planning to do, if there are no better options, is
>> to
>> > create an OutputStream implementation that buffers the file contents,
>> and
>> > uploads it to the blob store when flush()/close() is called. But that
>> > doesn't sound great, so I'm hoping maybe someone else has a better idea?
>> >
>> >
>> >
>> > *Steve Kingsland*
>> >
>> > Senior Software Engineer
>> >
>> > *Opower * <http://www.opower.com/>
>> >
>> >
>> > *We’re hiring! See jobs here <http://www.opower.com/careers> *
>> >
>> >
>> > On Mon, Aug 4, 2014 at 7:53 PM, Andrew Gaul <[email protected]> wrote:
>> >
>> > > On Mon, Aug 04, 2014 at 04:39:15PM -0400, Steve Kingsland wrote:
>> > > > I'm trying to use jclouds to write to an S3-compatible object store
>> > > (Ceph),
>> > > > and I'd like to use an OutputStream to write the payload for a
>> Blob. How
>> > > do
>> > > > I do this?
>> > > >
>> > > > I'm working on an existing system which uses a stream-based
>> abstraction
>> > > > around all of the file I/O, that looks like this:
>> > > >
>> > > > public interface ResourceFactory {
>> > > > InputStream getInputStream(String resourcePath) throws
>> IOException;
>> > > >
>> > > > OutputStream getOutputStream(String resourcePath) throws
>> IOException;
>> > > > }
>> > > >
>> > > > I was able to implement getInputStream() for *reading* a blob from
>> > > jclouds,
>> > > > but I'm not sure how to return an OutputStream for *writing* a blob.
>> > > >
>> > > > I know this question has already been asked
>> > > > <https://groups.google.com/forum/#!topic/jclouds/F2pCt9i7TSg>, but
>> it
>> > > seems
>> > > > like a common-enough use case that it shouldn't be terribly
>> complicated
>> > > to
>> > > > implement. Can anyone provide suggestions for how to accomplish
>> this?
>> > > >
>> > > > The best I could find is Payload#writeTo
>> > > > <
>> > >
>> http://demobox.github.io/jclouds-maven-site-1.7.2/1.7.2/jclouds/apidocs/org/jclouds/io/WriteTo.html
>> > > >,
>> > > > which accepts an OutputStream but is @Deprecated. Thanks in advance!
>> > >
>> > > Steve, I am not sure I understand your use case. putBlob consumes an
>> > > input *source*, e.g., ByteSource or InputStream. Why do you want to
>> > > provide it an output *sink*, e.g., OutputStream? If you have a
>> special
>> > > need, could you provide a custom implementation of ByteSource or
>> > > InputStream, or use PipedInputStream/PipedOutputStream if you really
>> > > must use an OutputStream?
>> > >
>> > > --
>> > > Andrew Gaul
>> > > http://gaul.org/
>> > >
>>
>> --
>> Andrew Gaul
>> http://gaul.org/
>>
>
>
