[
http://jira.amdatu.org/jira/browse/AMDATU-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11651#comment-11651
]
Marcel Offermans commented on AMDATU-428:
-----------------------------------------
If the question is, can someone write an adapter that implements the BlobStore
API on top of a local Cassandra instance, that would of course be possible,
just like you can create a "transient" and "filesystem" based implementation
(see http://code.google.com/p/jclouds/wiki/BlobStore#Transient_Provider for a
short overview). I also think the question is valid if it makes sense to do so.
Is Cassandra really the right tool for that job? FWIW, Cassandra's own FAQ
states "Currently Cassandra isn't optimized specifically for large file or BLOB
storage. However, files of around 64Mb and smaller can be easily stored in the
database without splitting them into smaller chunks." (see
http://wiki.apache.org/cassandra/FAQ#large_file_and_blob_storage). Also, a
document on limitations states: "Cassandra's public API is based on Thrift,
which offers no streaming abilities -- any value written or fetched has to fit
in memory. This is inherent to Thrift's design and is therefore unlikely to !
change. So adding large object support to Cassandra would need a special API
that manually split the large objects up into pieces. A potential approach is
described in http://issues.apache.org/jira/browse/CASSANDRA-265. As a
workaround in the meantime, you can manually split files into chunks of
whatever size you are comfortable with -- at least one person is using 64MB --
and making a file correspond to a row, with the chunks as column values." (see
http://wiki.apache.org/cassandra/CassandraLimitations)
Sounds to me it's not worth it.
On the other hand, the work that's being done on "table" stores sounds very
interesting!
> Blob Store support
> ------------------
>
> Key: AMDATU-428
> URL: http://jira.amdatu.org/jira/browse/AMDATU-428
> Project: Amdatu
> Issue Type: New Feature
> Reporter: Paul Bakker
> Attachments: TestContext.patch
>
>
> We need some way to work with Blob Store providers such as Amazon S3.
> I've implemented this and checked in to the sandbox:
> https://subversion.amdatu.org/svn/amdatu/sandbox/paulb
> The service is built on jclouds, so any Blob Store support by jclouds can be
> used. The service exposes the jclouds API directly. Jcloud already is an
> abstraction so it doesn't make sense to create another abstraction on top of
> this. If we would provide our own abstraction we would either copy the
> jclouds API or loose a lot of jcloud's functionality. If we need to support
> additional stores we can add them to jclouds.
> The jclouds API is exposed via an OSGI service (BlobStoreService). Jclouds
> has to be configured with a username/password etc. to work with a specific
> blob store. For this I have created a Managed Service Factory. As soon as the
> Factory finds configuration for a specific store it instantiates a service
> specifically for that store. To use this service you have to go through the
> following steps:
> -Add configuration (using Config Admin) for the provider containing:
> provider, identity and secret. The provider must be one of the providers
> supported by jclouds.
> -Lookup the service
> {code}
> dependencyManager.add(createComponent().setImplementation(BlobStoreClient.class)
>
> .add(createServiceDependency().setService(BlobStoreService.class,
> BlobStoreService.Providers.AWS_S3.getAsServiceFilter()).setRequired(true)));
> {code}
> From there you just use the jclouds API, for example:
> {code}
> BlobStoreContext blobStoreContext = blobStoreService.getContext();
> BlobStore blobStore = blobStoreContext.getBlobStore();
> PageSet<? extends StorageMetadata> amdatu = blobStore.list("amdatu");
> for (StorageMetadata storageMetadata : amdatu) {
> System.out.println(storageMetadata.getName());
> }
> {code}
> Let me know what you think!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
http://jira.amdatu.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________
Amdatu-developers mailing list
[email protected]
http://lists.amdatu.org/mailman/listinfo/amdatu-developers
