Re: CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0
This is after you backported 9472 to 3.0? -- Jeff Jirsa > On Jul 27, 2017, at 10:33 PM, Andrew Whangwrote: > > Jay, > > We see ~20% write latency improvement on 3.0.13 in a write-heavy workload, > using offheap_objects. offheap_buffers only offered minimal improvement. > > On Thu, Jul 27, 2017 at 10:06 PM, Jay Zhuang > wrote: > >> Hi Andrew, >> >> Do you see performance gain from reintroducing off-heap memtables for >> 3.0.x? When we were on 2.2.x we saw big improvements from enabling >> off-heap memtables. >> >> Thanks, >> Jay >> >>> On 7/27/17 9:37 PM, Andrew Whang wrote: >>> I'm wondering if anyone has been able to patch CASSANDRA-9472 to 3.0, >>> without breaking unit tests. The patch was introduced in 3.4, but 3.0.x >>> contains unit tests and code from later 3.x releases, which makes >> debugging >>> unit test failures difficult - i.e. SSTableCorruptionDetectionTest, >> which >>> was introduced in 3.7 and is found in 3.0.14, but not in 3.4. >>> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: dev-h...@cassandra.apache.org >> >> - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0
Jay, We see ~20% write latency improvement on 3.0.13 in a write-heavy workload, using offheap_objects. offheap_buffers only offered minimal improvement. On Thu, Jul 27, 2017 at 10:06 PM, Jay Zhuangwrote: > Hi Andrew, > > Do you see performance gain from reintroducing off-heap memtables for > 3.0.x? When we were on 2.2.x we saw big improvements from enabling > off-heap memtables. > > Thanks, > Jay > > On 7/27/17 9:37 PM, Andrew Whang wrote: > > I'm wondering if anyone has been able to patch CASSANDRA-9472 to 3.0, > > without breaking unit tests. The patch was introduced in 3.4, but 3.0.x > > contains unit tests and code from later 3.x releases, which makes > debugging > > unit test failures difficult - i.e. SSTableCorruptionDetectionTest, > which > > was introduced in 3.7 and is found in 3.0.14, but not in 3.4. > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0
Hi Andrew, Do you see performance gain from reintroducing off-heap memtables for 3.0.x? When we were on 2.2.x we saw big improvements from enabling off-heap memtables. Thanks, Jay On 7/27/17 9:37 PM, Andrew Whang wrote: > I'm wondering if anyone has been able to patch CASSANDRA-9472 to 3.0, > without breaking unit tests. The patch was introduced in 3.4, but 3.0.x > contains unit tests and code from later 3.x releases, which makes debugging > unit test failures difficult - i.e. SSTableCorruptionDetectionTest, which > was introduced in 3.7 and is found in 3.0.14, but not in 3.4. > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0
I'm wondering if anyone has been able to patch CASSANDRA-9472 to 3.0, without breaking unit tests. The patch was introduced in 3.4, but 3.0.x contains unit tests and code from later 3.x releases, which makes debugging unit test failures difficult - i.e. SSTableCorruptionDetectionTest, which was introduced in 3.7 and is found in 3.0.14, but not in 3.4.
Re: Customize Cassandra to execute a query multiple times
Why not simply have a microservice that does this for you? It may expose an API that allows to either store queries and/or conditions that trigger the queries (maybe time elapsed, an alert generated, whatever...) and it would then connect to Cassandra and execute the stored query(ies). Given that there is no client waiting for a response, then latency is not even a (major) issue, so the extra network hop is probably of little consequence. Why would you want this to be an integral part of Cassandra? (like Jeff I also am struggling to come up with a concrete use case where this would be desirable) -- *Marco Massenzio* http://codetrips.com On Wed, Jul 26, 2017 at 11:06 PM, Ke Wangwrote: > Hi Jeff, > > Thanks for the prompt reply. > > I'm sorry I wasn't clear about my use case. Let's say I just want the > server to execute writes that are not directly generated from clients. It > can be previously stored queries (the case in my last email), or it can be > arbitrarily made-up write requests by the server itself. > > Can you describe a little bit more on how to implement serializing the > mutation into the table? > > Best, > Ke > > 2017-07-26 22:57 GMT-07:00 Jeff Jirsa : > > > > > > > On 2017-07-26 22:19 (-0700), Ke Wang wrote: > > > Hello all, > > > > > > Is there a way to customize Cassandra to execute a query multiple > times? > > > > > > > There's always a way... > > > > > My use case is the following. When the Cassandra server receives > queries > > > from remote clients, besides executing those queries, the server also > > > stores the queries. In the future, the server can re-execute stored > > queries > > > under certain circumstances. > > > > > > > Seems like an unusual request. Where are you storing the queries? Are > > these reads or writes? For writes you could just serialize the resulting > > partitionupdate/mutation into a table a la pre-3.0 hints, and then > re-play > > as needed (though tbh I don't really get the use case)? > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
Re: Customize Cassandra to execute a query multiple times
Hi Jeff, Thanks for the prompt reply. I'm sorry I wasn't clear about my use case. Let's say I just want the server to execute writes that are not directly generated from clients. It can be previously stored queries (the case in my last email), or it can be arbitrarily made-up write requests by the server itself. Can you describe a little bit more on how to implement serializing the mutation into the table? Best, Ke 2017-07-26 22:57 GMT-07:00 Jeff Jirsa: > > > On 2017-07-26 22:19 (-0700), Ke Wang wrote: > > Hello all, > > > > Is there a way to customize Cassandra to execute a query multiple times? > > > > There's always a way... > > > My use case is the following. When the Cassandra server receives queries > > from remote clients, besides executing those queries, the server also > > stores the queries. In the future, the server can re-execute stored > queries > > under certain circumstances. > > > > Seems like an unusual request. Where are you storing the queries? Are > these reads or writes? For writes you could just serialize the resulting > partitionupdate/mutation into a table a la pre-3.0 hints, and then re-play > as needed (though tbh I don't really get the use case)? > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >