I have stored large numbers of large files in Accumulo using a derivative of the File System Archive ( https://accumulo.apache.org/1.4/examples/dirlist.html). I had code that accepted streams but stores chunks instead of the whole file into the Value. Attached to my ColumnQualifier was essentially a chunk index.
On Thu, Apr 10, 2014 at 7:10 AM, pdread <[email protected]> wrote: > Hi > > This has been bothering me for some time, and I suspect its a dumb > question, > but what the heck. > > The accumulo client API only accepts byte[] or Text as its Mutation input. > Would it be possible to > use a Stream instead (devlopers?)? If I'm processing streams, which I am, > and I have to handle files to the tune > of 10GB, which I would like to store in Accumulo but I have read I cannot, > it would save memory > footprint on my tomcats if I could stream my data into accumulo and not > deal > with bytes/text. > > Oh and accumulo developers while you're at adding this new feature it would > be nice if the bulk loads could > append instead of just replace the tables....would be nice. > > Thanks > > Paul > > > > -- > View this message in context: > http://apache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981.html > Sent from the Users mailing list archive at Nabble.com. >
