If you need larger than 1MB objects and you are only serving these objects to 5 users at a time. It really sounds like memcache is the wrong tool for your project. You might want to look into something like MongoDB which has a larger object limit of 4MB. There are a number of key value stores that can handle even larger size objects.
Could you explain a little more on what type of binary data you are manipulating ? On Mon, Sep 20, 2010 at 8:25 PM, MikeG. <[email protected]> wrote: > Hi, > > I'm starting a project in which I would like to have the entire DB in > cache. > The reason is that my transactions are reading large amount of data > from the DB to generate > a deliverable blob. > > I have no concurrency issues. I will always have around 5 users max > concurrently and most > of the time it will be single user. SO I look at the context as > virtually single user. > > To avoid this massive DB reading (of large count of large chunks) I > would like to have it > permanently in memory. > When any modification of a record happened it is not being written to > the DB until the > big blob final product is delivered and the local system goes idle. > Only then modifications are written to the DB. Upon successful > completion of DB update the local system sends a > message to the recipient of the product to inform it that the DB is > now in sync with the product > at hand and it can be consumed. > > Now, to hold 1TB in memory I need a cluster and not a small one. I > have decided to use > memory mapped files such that my RAM is virtual memory. Easy to get > large file system of > several TBs. > > My question - is there any limit memcached has as far as cache size? > > Also, memcached (the C implementation) has a 1MB record size limit. 1) > What's the reason > for that? 2) Can it be changed (with a hacked private version) 3) Does > Jmemcached has the > same limit? > > Thanks, > -Michael >
