Dev team, We have a chance to speak directly to Matt Stump CTO of SourceNinja. Matt took SourceNinja through a postgres, solr, hadoop, redis migration to Cassandra. Should be very pertinent to the technology selection discussion.
We have limited availability for callers (i.e. 10 total): Friday, July 20th, 2012 @ 14:00 PDT, 17:00 EDT. If you would like to attend, please RSVP to me for logistics ASAP. Best, L You may remember Matt Stump's story: SourceNinja started with Rails and Postgres, but lacked the ability to process information in the background. Added background processing with Resque/redis — added another database cluster to their deployment. Ran into postgres limitations - could not perform natural language searching, trigram, fuzzy searching. Added solr - now up to three clusters with lots of synchronization issues. Ran into issue with backend workers and had to add a Hadoop cluster - now up to four database clusters! Complexity exploded: Ran into coherence issues where they would have to run solr reindexes for days on end. Each cluster behaves differently, operational complexity, replication, fail over, backup, admin. Reevaluated what kind of solution they needed: Flexible schema. Easy to scale and increased redundancy. Fast enough for web requests. Consolidate existing services (i.e. four independent clusters). Full text indexing. Hadoop support. Selected DataStax Enterprise Edition (i.e. Cassandra). <insert happy ending here> :)
_______________________________________________ oae-dev mailing list [email protected] http://collab.sakaiproject.org/mailman/listinfo/oae-dev
