Niclas, It really depends on your needs, but I can assure you that I did some basic testing using 10 users operating around 200 entities (multiple times) i... and I had timeouts (> 60 seconds / cpu was around 30%), when done synchronously .
So 1) is always possible ;-) 2) if you really believe that, AMZN PUT options are underpriced these last days... 3) on the opposite, they basically said, when asked for limits about bandwidth, that they had no limit (tests report around 22MB/s) I am not so sure you understood exactly what I meant, don't hesitate to write back tomorrow. cheers phil Le 30 nov. 2009 à 13:41, Niclas Hedhman a écrit : > > On Philippe's request... > > I'm not at computer anymore, so I'll answer properly tomorrow... > > BUT, it sounds odd that S3 should be slower than a "home made" layer across > the Amazon network... what comes to mind, why isn't such layer already > present transparently? I can see only three reasons; > > 1. Something in your conclusions are wrong, > > 2. Amazon developers ain't smart enough, > > 3. Amazon intentionally want it to be slow. > > Anything else you can think of? > > -- Cheers > Niclas > >> ---------- Forwarded message ---------- >> From: "philippe van dyck" <[email protected]> >> Date: 30 Nov 2009 20:27 >> Subject: Re: Code for S3 ES >> To: "Niclas Hedhman" <[email protected]> >> >> We dropped off the list, I don't mind going back on it (I let you post this >> back) >> >> >> Le 30 nov. 2009 à 12:27, Niclas Hedhman a écrit : >> > On Mon, Nov 30, 2009 at 4:54 PM, philippe van dyck <[email protected]> >> > wrote: > >> Was my first s... >> >> Indeed, if you don't manage the sequence yourself and let the jclouds >> library and httpnio manage the threads you may loose the order needed to >> apply the new "layer" of updates to your entity store. >> Those operations, even a read operation, are ordered. That is the reason I >> use a queue. >> >> Accessing S3 is quite slow, and I tried previously to access it >> synchronously... and it was much too slow (I got timeouts in Wicket page >> display). >> >> So I definitely needed an asynchronous way to access S3, that is why the >> "cache" was added. >> > >> So at first, I though about using the underlying library but I also >> > >> needed to implement a lo... >> >> Bad news, you simply cannot lock entries in S3 (no way to create a >> semaphore) ... you need a cluster-wide mechanism to do so (sounds like some >> NASA project). >> > >> I preferred to use one queue of updates / key. It is simple but not >> > >> very efficient regarding ... >> >> Well, I had a stupid but efficient idea... what about synchronizing >> MapEntityStore.applyChanges ? >> >> Then I had a better one : final public BlockingQueue<BlockingQueue<T>> pipe >> >> As long as updates are sent *in the right order* asynchronously, I don't >> mind queueing the 'updates queues' in a pipe. >> >> Sound a bit overkill... but it works ! >> >> One more thing, when I empty a queue of updates (containing read operations >> with waiting threads), I do it in parallel, using a thread / entity >> reference. >> >> To be sure we both understand each other, I am talking about a pipe >> containing queues of updates to be applied one after the other, in parallel >> if the entity references are different. >> > >> Another added value of Infinispan is that it has it's own transaction >> > >> manager (XA!)... I plan... >> >> Well, the XA stuff is managed cluster-wide by infinispan (don't ask, looks >> like a big locking mess - JGroups stuff) >> > >> Agreed, caching should be available as a generic mechanism elsewhere >> > >> and used in the ES the '... >> >> We may need slowly but surely to stop talking about 'caching' ... since we >> actually use it to store indefinitely stuff 'à la' NoSQL ? WDYT ? >> > >> I am right now trying to understand how Infinispan manage failures .... >> > >> and I don't like a sy... >> >> No real alternative on the market right now. >> >> Quite easy to understand once you played with it. (Cloud diving is a new >> sport) >> >> Be warned, this is a Ringgit eating machine ;-) >> >> Phil >> > > > Cheers > -- > Niclas Hedhman, Software Developer > http://www.qi4j.org >> > > > - New Energy for Jav... >> >> > > _______________________________________________ > qi4j-dev mailing list > [email protected] > http://lists.ops4j.org/mailman/listinfo/qi4j-dev
_______________________________________________ qi4j-dev mailing list [email protected] http://lists.ops4j.org/mailman/listinfo/qi4j-dev
