Thanks Andrew, I will. Can you provide any guidance, pseudo-code, examples, etc. on how I would use a PipedOutputStream to buffer the content that's being written, and upload it to a BlobStore?
To put it differently: how can I use these classes to return an OutputStream that is capable of putting a blob in a blob store, all by itself? *Steve Kingsland* Senior Software Engineer * Opower * <http://www.opower.com/> *We’re hiring! See jobs here <http://www.opower.com/careers> * On Mon, Aug 4, 2014 at 8:30 PM, Andrew Gaul <[email protected]> wrote: > Please look at PipedInputStream/PipedOutputStream which should address > this use case. > > On Mon, Aug 04, 2014 at 08:10:49PM -0400, Steve Kingsland wrote: > > My use case is: > > > > 1. the calling code is generating content in memory, and wants an > > OutputStream to write it to (currently it's going to disk); > > > > 2. the putBlob() method wants a btye[], InputStream, etc. that it can > read > > from. > > > > My problem is that *both* parties want to control the transaction. Here > is > > what my calling code looks like: > > > > OutputStream documentOutputStream = null; > > try { > > documentOutputStream = this. > > *documentResourceFactory.getDocumentOutputStream(documentPath);* > > > > renderAndWriteDocument(renderContext, documentOutputStream); > > } > > catch (IOException e) { > > ... > > } > > finally { > > Closeables.closeQuietly(documentOutputStream); > > } > > > > I'm trying to create an implementation of DocumentResourceFactory that > > returns an OutputStream for writing the document to an Object Store using > > jclouds, instead of writing it to the local file system. I guess a > > stream-based API isn't really supported for writing to object stores... > > > > In my case, the files are small enough that I'm OK buffering them in > > memory. So what I'm planning to do, if there are no better options, is to > > create an OutputStream implementation that buffers the file contents, and > > uploads it to the blob store when flush()/close() is called. But that > > doesn't sound great, so I'm hoping maybe someone else has a better idea? > > > > > > > > *Steve Kingsland* > > > > Senior Software Engineer > > > > *Opower * <http://www.opower.com/> > > > > > > *We’re hiring! See jobs here <http://www.opower.com/careers> * > > > > > > On Mon, Aug 4, 2014 at 7:53 PM, Andrew Gaul <[email protected]> wrote: > > > > > On Mon, Aug 04, 2014 at 04:39:15PM -0400, Steve Kingsland wrote: > > > > I'm trying to use jclouds to write to an S3-compatible object store > > > (Ceph), > > > > and I'd like to use an OutputStream to write the payload for a Blob. > How > > > do > > > > I do this? > > > > > > > > I'm working on an existing system which uses a stream-based > abstraction > > > > around all of the file I/O, that looks like this: > > > > > > > > public interface ResourceFactory { > > > > InputStream getInputStream(String resourcePath) throws > IOException; > > > > > > > > OutputStream getOutputStream(String resourcePath) throws > IOException; > > > > } > > > > > > > > I was able to implement getInputStream() for *reading* a blob from > > > jclouds, > > > > but I'm not sure how to return an OutputStream for *writing* a blob. > > > > > > > > I know this question has already been asked > > > > <https://groups.google.com/forum/#!topic/jclouds/F2pCt9i7TSg>, but > it > > > seems > > > > like a common-enough use case that it shouldn't be terribly > complicated > > > to > > > > implement. Can anyone provide suggestions for how to accomplish this? > > > > > > > > The best I could find is Payload#writeTo > > > > < > > > > http://demobox.github.io/jclouds-maven-site-1.7.2/1.7.2/jclouds/apidocs/org/jclouds/io/WriteTo.html > > > >, > > > > which accepts an OutputStream but is @Deprecated. Thanks in advance! > > > > > > Steve, I am not sure I understand your use case. putBlob consumes an > > > input *source*, e.g., ByteSource or InputStream. Why do you want to > > > provide it an output *sink*, e.g., OutputStream? If you have a special > > > need, could you provide a custom implementation of ByteSource or > > > InputStream, or use PipedInputStream/PipedOutputStream if you really > > > must use an OutputStream? > > > > > > -- > > > Andrew Gaul > > > http://gaul.org/ > > > > > -- > Andrew Gaul > http://gaul.org/ >
