Re: Client-side timeouts after dropping table

Thanks, filing this under "things I wish I'd realized sooner" :) On Tue, Sep 20, 2016 at 10:27 PM, Jonathan Haddad wrote: > 3.7 falls under the Tick Tock release cycle, which is almost completely > untested in production by experienced operators. In the cases where it has >

Re: Client-side timeouts after dropping table

I was able to get metrics, but nothing stands out. When the applications start up and a table is dropped, shortly thereafter on a subsequent write I get a NoHostAvailableException that is caused by an OperationTimedOutException. I am not 100% certain on which write the timeout occurs because there

Re: Client-side timeouts after dropping table

3.7 falls under the Tick Tock release cycle, which is almost completely untested in production by experienced operators. In the cases where it has been tested, there have been numerous bugs found which I (and I think most people on this list) consider to be show stoppers. Additionally, the Tick

Re: Client-side timeouts after dropping table

Can you elaborate on why not 3.7? On Tue, Sep 20, 2016 at 7:41 PM, Jonathan Haddad wrote: > If you haven't yet deployed to prod I strongly recommend *not* using 3.7. > > What network storage are you using? Outside of a handful of highly > experienced experts using EBS in

Re: Client-side timeouts after dropping table

Thanks Nate. We do not have monitoring set up yet, but I should be able to get the deployment updated with a metrics reporter. I'll update the thread with my findings. On Tue, Sep 20, 2016 at 10:30 PM, Nate McCall wrote: > If you can get to them in the test env. you want

Re: Client-side timeouts after dropping table

If you can get to them in the test env. you want to look in o.a.c.metrics.CommitLog for: - TotalCommitlogSize: if this hovers near commitlog_size_in_mb and never goes down, you are thrashing on segment allocation - WaitingOnCommit: this is the time spent waiting on calls to sync and will start to

Re: Client-side timeouts after dropping table

I have seen in various threads on the list that 3.0.x is probably best for prod. Just wondering though if there is anything in particular in 3.7 to be weary of. I need to check with one of our QA engineers to get specifics on the storage. Here is what I do know. We have a blade center running

Re: Client-side timeouts after dropping table

If you haven't yet deployed to prod I strongly recommend *not* using 3.7. What network storage are you using? Outside of a handful of highly experienced experts using EBS in very specific ways, it usually ends in failure. On Tue, Sep 20, 2016 at 3:30 PM John Sanda wrote: