Re: Distributed Cache Server?
har nah, think I'll give BT a miss for this purpose. Yeah, know it was a loaded question (no specifics), but was really only thinking out aloud. The plans are: - Multiple copies of data on different machines (just thinking 2 copies of the data) - Load spread over multiple machines... anything between 2->lots (again, not being specific). - Data itself I'm expecting to be low a few Gb at tops (but ofcourse, as this extends I might want to increase the size) - Haven't thought a LOT about automatic handling of errors.. yet. - and its assumed to be on a trusted network. Apart from memcached, I haven't seen anything similar to what I'm after (plus to be perfectly honest. I wouldn't mind coding it up anyway) ;) Was just "feeling about" to see if anything else is out there. KenF ps. btw, only part of it is theoretical, a lot of this I've already coded, but not targetted towards large scale (100's machines etc)... So am thinking what I'd need to consider to increase this to a larger scale. Roger Binns wrote: > <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > > Does anyone know if a "distributed caching system" has been developed > > for use with Python? > > BitTorrent :-) > > > Yes, "distributed caching system" is a bit of a general term, but am > > really just talking about something as simple as key + value (arbitrary > > class) which can be split over a number of machines in an efficient > > manner. > > You'll need to define what the sweet spot is that you are aiming for. > Are we talking tens of thousands of keys or billions? How big is the > data (megabytes, gigabytes, terabytes?) Do you need transactional > integrity (eg when are updates seen by other readers)? Do you want > redundancy (data duplicated on multiple machines)? How many machines > are we talking about? Should failure be automatically detected? Is > there a need for security or treating the machines as untrusted? > > Roger -- http://mail.python.org/mailman/listinfo/python-list
Re: Distributed Cache Server?
[EMAIL PROTECTED] wrote: > Does anyone know if a "distributed caching system" has been developed > for use with Python? I've seen mention of memcached, but was really > after something natively python. [...] Too bad, because memcached implements all the difficult parts already (like failover if one node dies), has Python bindings: ftp://ftp.tummy.com/pub/python-memcached/ and just plain works. > [...] I've started development of such a system, but am wondering if > something already exists out there. I don't know of any pure-Python equivalent. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Distributed Cache Server?
<[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Does anyone know if a "distributed caching system" has been developed > for use with Python? BitTorrent :-) > Yes, "distributed caching system" is a bit of a general term, but am > really just talking about something as simple as key + value (arbitrary > class) which can be split over a number of machines in an efficient > manner. You'll need to define what the sweet spot is that you are aiming for. Are we talking tens of thousands of keys or billions? How big is the data (megabytes, gigabytes, terabytes?) Do you need transactional integrity (eg when are updates seen by other readers)? Do you want redundancy (data duplicated on multiple machines)? How many machines are we talking about? Should failure be automatically detected? Is there a need for security or treating the machines as untrusted? Roger -- http://mail.python.org/mailman/listinfo/python-list
Distributed Cache Server?
Does anyone know if a "distributed caching system" has been developed for use with Python? I've seen mention of memcached, but was really after something natively python. Yes, "distributed caching system" is a bit of a general term, but am really just talking about something as simple as key + value (arbitrary class) which can be split over a number of machines in an efficient manner. I've started development of such a system, but am wondering if something already exists out there. KenF -- http://mail.python.org/mailman/listinfo/python-list
