Tommaso, Thanks for that information.
I think we should dig into why (and which of) the components were locking up. Is it possible you were hitting https://github.com/Metaswitch/crest/issues/105? I think the fix for this was in yesterday's release. If running Cassandra under Oracle's JDK causes these failures, I'd suggest moving back to OpenJDK. Unfortunately, I'm not a big expert on Cassandra's internals, so I don't know what feature of it is causing problems with Oracle's JDK. Good suggestion about fixing up the homestead dependencies. The reason we haven't so far is that homestead could be located on a separate cluster from Cassandra (for scaling the two tiers independently). However, since this isn't a very common deployment topology, we're considering tying the packages together anyway. (As it turns out, it's hard to express the dependencies correctly when using the Datastax Cassandra repository and packages - we'd need to move to Apache's packages to achieve this.) Please let me know how you get on, and whether you're still seeing the lock-up with the new software release. Thanks, Matt ________________________________________ From: Tommaso Cucinotta [[email protected]] Sent: 22 November 2013 18:59 To: Matt Williams Cc: [email protected] Subject: Re: [Clearwater] cassandra 2.0.2 ? On 22/11/13 14:52, Matt Williams wrote: >> I'm using Oracle JVM 1.7 > > Thanks! All our testing has been with OpenJDK. Good to know, thanks. > Do you know why Oracle JVM was installed? Well, I can see that CW seemingly gets stuck from time to time, and needs a restart of components (e.g., after growing workload it stops giving me back 200s, so I reduce workload but no 200 any more for minutes..., then I manually ssh into bono, sprout, homestead and restart'em all (including cassandra)...., and after a while I can see these 200s coming back again. I just thought that having the right java might have helped (esp. due to Cassandra), so I just replaced myself OpenJDK with Oracle. Would u advise to go back to OpenJDK ? > Was it installed explicitly or has Ubuntu resolved its dependencies in a > different way on your system from ours? it didn't come from .deb deps resolution (see description above). Yet on this side, why don't you put in the homestead .deb the right Cassandra+dsc version deps ? > If you install OpenJDK do you still see the problem? can't remember, probably not, but as I messed around with cassandra upgrade, then downgrade, plus OpenJDK->Oracle migration, who knows :-)... > ...assuming you are installing on Ubuntu? Please can you confirm? ...and > which version? Yes, the recommended one: $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=12.04 DISTRIB_CODENAME=precise DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS" > Does this issue occur immediately after installing Cassandra (i.e. without > Clearwater) or does it only start once you've installed homestead/homer? the -Xss issue seems specific to Cassandra and perhaps Oracle 1.7 JDK. T. -- Tommaso Cucinotta, Computer Engineer PhD Researcher at Alcatel-Lucent Bell Laboratories Blanchardstown Business & Technology Park Dublin - Ireland _______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
