Re: Additions to Cassandra ecosystem page?
No major opinion on the "cloud offerings" piece, but I agree people should know what they are getting into, and be able to make an informed decision. However, if someone is going down that path, I would hope they do the due-diligence to make sure it fits their requirements. 1 small update I would suggest. It seems like Datastax Spring Boot entry would go in development frameworks as opposed to the sidecar section. On Tue, Jun 22, 2021, 5:39 PM bened...@apache.org wrote: > Under Cloud Offerings, are we comfortable implicitly endorsing “API > compatible” offerings that aren’t actually Cassandra, and also don’t (as > far as I am aware) fully support Cassandra functionality? Should we at > least mention that this is the case? > > > From: Melissa Logan > Date: Tuesday, 22 June 2021 at 21:39 > To: u...@cassandra.apache.org , > dev@cassandra.apache.org > Subject: Additions to Cassandra ecosystem page? > Hi all, > > The Cassandra community recently updated its website and has added several > new entries to the Ecosystem page: https://cassandra.apache.org/ecosystem/ > . > > If you have edits or know of other third-party Cassandra projects, tools, > products, etc that may be useful to others -- please get in touch and we'll > add to the next round of site updates in July. > > Thanks! > > Melissa > Apache Cassandra Contributor >
Re: Obfuscation of passwords in audit loging, in or not in 4.0?
+1 to this being a serious bug. As a large user, if we used internal passwords, this would completely prevent me from using Cassandra native audit log capabilities. Disabling DCL is not a great option, as DCL is probably the most needed auditable event. If this is on by default (not sure of default settings) I also assume this would be classified as an immediate CVE... Right? I don't directly contribute, so I can't talk too much, but I can't see how 4.0.0 could go GA with this. On Thu, Jun 3, 2021, 7:24 PM Sumanth Pasupuleti < sumanth.pasupuleti...@gmail.com> wrote: > > I am on the side of "this sounds like a really bad bug" for the audit > pieces, maybe less so than FQL. Anyone using audit for real probably has > meaningful audit requirements, which means they're in an industry where > they get audited for security, which means logging passwords is a big deal. > > +1. Given we are shipping audit logging feature for the first time with > 4.0, it would be great if this rather low complex patch can be included in > the 4.0 RC and thereby ship a "complete feature". > > On Thu, Jun 3, 2021 at 4:04 PM Vinay Chella > wrote: > > > > I think it can be argued that this is a pretty serious bug for a newly > > introduced feature, and qualifies for inclusion in an RC, but I don’t > > personally have a strong opinion on if this should happen. > >+1 > > > > > One more point - if we keep the workaround, that should be documented > > with > > > big red letters for the users. > > > > Yes, heavy +1, if we are not merging it. Another idea, if we are not > > merging this in, is to put DCL(CREATE ROLE/USER, ALTER ROLE/USER etc.,) > > queries in the default configuration (cassandra.yaml) exclude list to > avoid > > oops for operators, since that is the only query type that log passwords > in > > plain text and all other places they are not. > > > > > > Thanks, > > Vinay > > > > > > On Thu, Jun 3, 2021 at 3:58 PM bened...@apache.org > > wrote: > > > > > I think it can be argued that this is a pretty serious bug for a newly > > > introduced feature, and qualifies for inclusion in an RC, but I don’t > > > personally have a strong opinion on if this should happen. > > > > > > I can’t imagine how this would be an _exception_ for inclusion in 4.0.1 > > > though. > > > > > > From: Mick Semb Wever > > > Date: Thursday, 3 June 2021 at 22:45 > > > To: dev@cassandra.apache.org > > > Subject: Re: Obfuscation of passwords in audit loging, in or not in > 4.0? > > > Thanks for raising this Stefan. > > > > > > > > > > > > > While I humbly think this is 4.0-worthy, the process we have, as far > > > > as I know, is that there should be only critical fixes in 4.0 so I > > > > guess this will go to 4.0.1, right? Or does this qualify to go to 4.0 > > > > still? > > > > > > > > > > > > > I believe the question here is whether this patch is worthy of an > > exception > > > to go to 4.0.x. (i.e. 4.0.1) > > > At this point in time all improvements would be by default slated for > 4.x > > > (i.e. 4.1) > > > > > > It does not qualify as a RC critical bug for 4.0.0. > > > > > > Looking at the patch it is simple, and one could almost consider it a > > > security fix on a new 4.0 feature, so I'd say it's a valid exception > for > > > 4.0.x. > > > Keen to hear what others think. And how we should go about requesting > > such > > > exceptions for non-bugs during each annual release cycle. > > > > > >
Re: Google Season of Docs 2020 participation
Joey, I really like this idea, and get asks for this type of documentation from new and moderately experienced developers alike. A "Cassandra by Example" approach helps... * Highlight popular or unique use cases showcasing strengths (and weaknesses) of Cassandra. * Gives practical examples on how to use Cassandra * Fosters critical thinking that can lead to innovative ideas and potentially more community engagement. * Can be used to showcase new or not well known features. I like your initial ideas, because they could also highlight common tasks like. * Successful use of denormalization * Resiliency design patterns * Use case based tuning (I know of some good blogs on tuning things like GC and compression) Good ideas! On Wed, Apr 29, 2020, 6:29 PM Joseph Lynch wrote: > Given the datastax donation I agree it makes sense for us to propose > projects that are unlikely to overlap. What do people think about > documentation that is slightly more instructional as opposed to > informational? > > Perhaps some kind of tutorial series on "Practical examples of > building useful systems with Cassandra": > * How to use Cassandra to store sensor data (maybe from a fictional > IOT device). This could include data models, functioning Java/Python > code running a HTTP service, and benchmarking examples. > * How to use Cassandra as a global record store for storing strongly > typed configuration for services. This could include architecture, > data models, explore use of different consistency level, etc ... > * How to safely use Cassandra's CRDT data models (or just LWT for low > scale) to implement a state machine (aka locking) service. > > Or we could go the operations direction: > * How to get a small test cluster running on Kubernetes (or in an ASG > on AWS / azure / ). This > could include setup, scaling, repairing, monitoring, etc ... > > Or something along those lines... > > Cheers, > -Joey > > > On Monday, April 27, 2020, 10:37:08 p.m. UTC, Dinesh Joshi < > djo...@apache.org> wrote: > > > > Folks, > > > > GSoD 2020 is upon us. The organizational applications are due soon May > 4th 2020 and I'd like us to participate in it again. GSoD 2019 brought in > great deal of improvements to the C* docs and I believe GSoD 2020 will be > able to bring in more enhancements. I realize we are also talking about > docs donations from DataStax but the docs project can be focused on 4.0 > which would not overlap with the donation. If you have opinions, please let > me know. > > > > Cheers, > > > > Dinesh > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: Sidecar meeting notes from 2020-03-10
Hi... In the wiki, the was mention of a DS sidecar show and tell slides. Are those available for public consumption? Also, I'll take a look and see where we may be able to contribute. We have our own suite of management tools for the scale we operate at. Many of these are specific to our requirments, but I'd like to contribute where we can. Also, I don't follow slack closely so maybe I missed it, but is there any way for me to be involved in the next in-person meet up. Hopefully I can provide feedback and requirments we have/see. Great work so far, and thanks! On Fri, Mar 13, 2020, 9:05 PM Vinay Chella wrote: > Sidecar discussions so far have been on the dev list, CEP-1[1], JIRA[2], > and seem to be working at its current volume. All the work is currently > tracked under 'sidecar' component in Apache Cassandra jira[2]. > > So with this context and the low volume, I am +1 on Jon's approach. > > [1]: CEP-1 - > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652224 > [2]: Sidecar componet - > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20component%20%3D%20Sidecar > > > Thanks, > Vinay > > > On Fri, Mar 13, 2020 at 2:12 PM Brandon Williams wrote: > > > I agree with Jon, and dev goes directly to my inbox. > > > > On Fri, Mar 13, 2020 at 4:07 PM Jon Haddad wrote: > > > > > > I think we're low volume enough across the board that we can continue > > > sidecar discussion on dev, and if we start to see too much noise, split > > > things up. > > > > > > On Fri, Mar 13, 2020 at 1:46 PM Patrick McFadin > > wrote: > > > > > > > I think Vinay just pinged everyone working on the project, but to > > Josh's > > > > point. is this even traffic for the cassandra dev ml? Or does a new > one > > > > need to be created? > > > > > > > > On Fri, Mar 13, 2020 at 1:29 PM Nate McCall > > wrote: > > > > > > > > > Where was this announced? I didnt hear anything about it (it's > > possible I > > > > > missed an email but don't see anything). > > > > > > > > > > On Sat, Mar 14, 2020 at 8:26 AM Patrick McFadin < > pmcfa...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > This week, a small group of us met on Zoom on how to contribute > > best to > > > > > the > > > > > > sidecar project (CEP-1). DataStax is open sourcing several > > components, > > > > > one > > > > > > of which includes a management sidecar. This was open > conversation > > > > about > > > > > > what's being released and how best to participate in CEP-1 in the > > > > future. > > > > > > Thanks to Vinay Chella for organizing such a diverse group! > > > > > > > > > > > > As a matter of tracking any discussions, notes were added to > CEP-1 > > in > > > > the > > > > > > Apache cwiki. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-1+Online+meeting+2020-03-10 > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
Re: Update defaults for 4.0?
If someone isn't explicitly setting vnodes, and the default changes, it will vary from the number of assigned tokens for existing clusters, right? Won't this cause the node to fail to start? I am in favor of changing these defaults, but should provide very clear guidance on vnodes (unless I am wrong). I'm sure there are others that would be safe to change. I'll review our defaults we typically set and report back tomorrow. On Tue, Jan 21, 2020, 7:22 PM Jeremy Hanna wrote: > I mentioned this in the contributor meeting as a topic to bring up on the > list - should we take the opportunity to update defaults for Cassandra 4.0? > > The rationale is two-fold: > 1) There are best practices and tribal knowledge around certain properties > where people just know to update those properties immediately as a starting > point. If it's pretty much a given that we set something as a starting > point different than the current defaults, why not make that the new > default? > 2) We should align the defaults with what we test with. There may be > exceptions if we have one-off tests but on the whole, we should be testing > with defaults. > > As a starting point, compaction throughput and number of vnodes seem like > good candidates but it would be great to get feedback for any others. > > For compaction throughput ( > https://jira.apache.org/jira/browse/CASSANDRA-14902), I've made a basic > case on the ticket to default to 64 just as a starting point because the > decision for 16 was made when spinning disk was most common. Hence most > people I know change that and I think without too much bikeshedding, 64 is > a reasonable starting point. A case could be made that empirically the > compaction throughput throttle may have less effect than many people think, > but I still think an updated default would make sense. > > For number of vnodes, Michael Shuler made the point in the discussion that > we already test with 32, which is a far better number than the 256 > default. I know many new users that just leave the 256 default and then > discover later that it's better to go lower. I think 32 is a good > balance. One could go lower with the new algorithm but I think 32 is much > better than 256 without being too skewed, and it's what we currently test. > > Jeff brought up a good point that we want to be careful with defaults > since changing them could come as an unpleasant surprise to people who > don't explicitly set them. As a general rule, we should always update > release notes to clearly state that a default has changed. For these two > defaults in particular, I think it's safe. For compaction throughput I > think a release not is sufficient in case they want to modify it. For > number of vnodes, it won't affect existing deployments with data - it would > be for new clusters, which would honestly benefit from this anyway. > > The other point is whether it's too late to go into 4.0. For these two > changes, I think significant testing can still be done with these new > defaults before release and I think testing more explicitly with 32 vnodes > in particular will give people more confidence in the lower number with a > wider array of testing (where we don't already use 32 explicitly). > > In summary, are people okay with considering updating these defaults and > possibly others in the alpha stage of a new major release? Are there other > properties to consider? > > Jeremy > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: "4.0: TBD" -> "4.0: Est. Q4 2019"?
Nice work... I like this and have no additions/comments at this time On Wed, Sep 18, 2019, 4:18 PM sankalp kohli wrote: > We added and changed a lot of things to this doc during a discussion in > NGCC. Can everyone take a look at it and provide feedback. > > On Wed, Sep 11, 2019 at 10:51 PM Dinesh Joshi wrote: > > > I have left some comments on the document. Apart from a few > clarifications > > and some minor changes, I feel its in a good shape. I think we should > move > > forward with it. We can refine the process, definitions & criteria as we > > learn. > > > > Dinesh > > > > > On Sep 11, 2019, at 11:15 AM, Sumanth Pasupuleti < > > sumanth.pasupuleti...@gmail.com> wrote: > > > > > > One more call for any additional comments/ feedback on the release > > > lifecycle document > > > > > > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > > > > > > Thanks, > > > Sumanth > > > > > > On Sat, Jul 27, 2019 at 1:01 AM Sumanth Pasupuleti < > > > sumanth.pasupuleti...@gmail.com> wrote: > > > > > >> Submitted patch to add release lifecycle information to the website > > >> https://issues.apache.org/jira/browse/CASSANDRA-15249 > > >> > > >> On Tue, Jun 25, 2019 at 6:57 AM Oleksandr Petrov < > > >> oleksandr.pet...@gmail.com> wrote: > > >> > > >>> Maybe a bit off-topic: > > >>> > > >>> Before we cut a release, we should make sure we take care of beta > > protocol > > >>> [1], include released driver versions [2] and remove compact storage > > >>> remainders [3]. Third one is optional, but I'd argue we should do it > > >>> sooner > > >>> rather than later. > > >>> > > >>> [1] https://issues.apache.org/jira/browse/CASSANDRA-14973 > > >>> [2] https://issues.apache.org/jira/browse/CASSANDRA-13951 > > >>> [3] https://issues.apache.org/jira/browse/CASSANDRA-13994 > > >>> > > >>> > > >>> > > >>> On Sat, Jun 22, 2019 at 1:25 AM Sumanth Pasupuleti < > > >>> sumanth.pasupuleti...@gmail.com> wrote: > > >>> > > Thanks for the feedback Scott. I have incorporated all the > incremental > > feedback I have thus far. > > > > Looking for any additional feedback folks may have. > > > > > > >>> > > > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > > > > On Tue, Jun 11, 2019 at 11:54 AM Scott Andreas < > sc...@paradoxica.net> > > wrote: > > > > > Thanks for starting this discussion, Sumanth! Added a round of > > >>> comments > > as > > > well. > > > > > > Summarizing my non-binding feedback: I feel that many of the items > > >>> under > > > "Alpha" and "Beta" should be achieved prior to the release of an > > >>> alpha, > > > especially those related to correctness/safety, scope lock, feature > > > completeness, deprecation, and backwards compatibility. > Establishing > > a > > > higher standard for official project releases (even at the alpha > and > > >>> beta > > > stage) will help us really polish the final build together. > > > > > > Ideally, I feel that contributors should have completed extensive > > > testing/validation to ensure that no critical or severe bugs exist > > >>> prior > > to > > > the release of an alpha (e.g., data loss, consistency violations, > > incorrect > > > responses to queries, etc). Perhaps we can add a line to this > effect. > > > > > > Ensuring that we've met that bar prior to alpha will help us focus > > the > > > final stages of the release on gathering feedback from users + > > >>> developers > > > to validate tooling and automation; compatibility with less > > >>> commonly-used > > > client libraries, testing new features, evaluating performance and > > > stability under their workloads, etc. > > > > > > – Scott > > > > > > On 6/11/19, 6:45 AM, "Sumanth Pasupuleti" < > > > sumanth.pasupuleti...@gmail.com> wrote: > > > > > >Thanks for the feedback on the product stages/ release life > cycle > > > document. > > >I have incorporated the suggestions and looking for any > additional > > > feedback > > >folks may have. > > > > > > > > > > >>> > > > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > > > > > >Thanks, > > >Sumanth > > > > > >On Tue, May 28, 2019 at 10:43 PM Scott Andreas < > > >>> sc...@paradoxica.net > > > > > > wrote: > > > > > >> Echoing Jon’s point here – > > >> > > >> JH: “My thinking is I'd like to be able to recommend 4.0.0 as a > > > production > > >> ready > > >> database for business critical cases” > > >> > > >> I feel that this is a standard that is both appropriate and > > > achievable, > > >> and one I’m legitimately excited about. > > >> > > >> Re: the current state of the test plan wiki in Confluence, I owe > > > another > > >> pass through. There has
Re: "4.0: TBD" -> "4.0: Est. Q4 2019"?
Is it common to have such strict release criteria from RC to GA... Release Candidate (RC) - Thorough testing is performed, and if no bugs are found within a testing period of one month, release is promoted to GA. - If bugs are found, fixes are made and above step is repeated That seems overly tough criteria (no bugs for a month?). Maybe this needs to be clarified to state the severity of bug in which a promotion will not happen, and the clock will start over? Or am I misinterpretting. Otherwise, yes, this is something I can get onboard with. Thanks! On Wed, Sep 11, 2019, 11:16 AM Sumanth Pasupuleti < sumanth.pasupuleti...@gmail.com> wrote: > One more call for any additional comments/ feedback on the release > lifecycle document > > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > > Thanks, > Sumanth > > On Sat, Jul 27, 2019 at 1:01 AM Sumanth Pasupuleti < > sumanth.pasupuleti...@gmail.com> wrote: > > > Submitted patch to add release lifecycle information to the website > > https://issues.apache.org/jira/browse/CASSANDRA-15249 > > > > On Tue, Jun 25, 2019 at 6:57 AM Oleksandr Petrov < > > oleksandr.pet...@gmail.com> wrote: > > > >> Maybe a bit off-topic: > >> > >> Before we cut a release, we should make sure we take care of beta > protocol > >> [1], include released driver versions [2] and remove compact storage > >> remainders [3]. Third one is optional, but I'd argue we should do it > >> sooner > >> rather than later. > >> > >> [1] https://issues.apache.org/jira/browse/CASSANDRA-14973 > >> [2] https://issues.apache.org/jira/browse/CASSANDRA-13951 > >> [3] https://issues.apache.org/jira/browse/CASSANDRA-13994 > >> > >> > >> > >> On Sat, Jun 22, 2019 at 1:25 AM Sumanth Pasupuleti < > >> sumanth.pasupuleti...@gmail.com> wrote: > >> > >> > Thanks for the feedback Scott. I have incorporated all the incremental > >> > feedback I have thus far. > >> > > >> > Looking for any additional feedback folks may have. > >> > > >> > > >> > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > >> > > >> > On Tue, Jun 11, 2019 at 11:54 AM Scott Andreas > >> > wrote: > >> > > >> > > Thanks for starting this discussion, Sumanth! Added a round of > >> comments > >> > as > >> > > well. > >> > > > >> > > Summarizing my non-binding feedback: I feel that many of the items > >> under > >> > > "Alpha" and "Beta" should be achieved prior to the release of an > >> alpha, > >> > > especially those related to correctness/safety, scope lock, feature > >> > > completeness, deprecation, and backwards compatibility. > Establishing a > >> > > higher standard for official project releases (even at the alpha and > >> beta > >> > > stage) will help us really polish the final build together. > >> > > > >> > > Ideally, I feel that contributors should have completed extensive > >> > > testing/validation to ensure that no critical or severe bugs exist > >> prior > >> > to > >> > > the release of an alpha (e.g., data loss, consistency violations, > >> > incorrect > >> > > responses to queries, etc). Perhaps we can add a line to this > effect. > >> > > > >> > > Ensuring that we've met that bar prior to alpha will help us focus > the > >> > > final stages of the release on gathering feedback from users + > >> developers > >> > > to validate tooling and automation; compatibility with less > >> commonly-used > >> > > client libraries, testing new features, evaluating performance and > >> > > stability under their workloads, etc. > >> > > > >> > > – Scott > >> > > > >> > > On 6/11/19, 6:45 AM, "Sumanth Pasupuleti" < > >> > > sumanth.pasupuleti...@gmail.com> wrote: > >> > > > >> > > Thanks for the feedback on the product stages/ release life > cycle > >> > > document. > >> > > I have incorporated the suggestions and looking for any > additional > >> > > feedback > >> > > folks may have. > >> > > > >> > > > >> > > >> > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit# > >> > > > >> > > Thanks, > >> > > Sumanth > >> > > > >> > > On Tue, May 28, 2019 at 10:43 PM Scott Andreas < > >> sc...@paradoxica.net > >> > > > >> > > wrote: > >> > > > >> > > > Echoing Jon’s point here – > >> > > > > >> > > > JH: “My thinking is I'd like to be able to recommend 4.0.0 as > a > >> > > production > >> > > > ready > >> > > > database for business critical cases” > >> > > > > >> > > > I feel that this is a standard that is both appropriate and > >> > > achievable, > >> > > > and one I’m legitimately excited about. > >> > > > > >> > > > Re: the current state of the test plan wiki in Confluence, I > owe > >> > > another > >> > > > pass through. There has been a lot of progress here, but I’ve > >> let > >> > > perfect > >> > > > be the enemy of the good in getting updates out. I’ll complete > >> that > >> > > pass > >> > > > later this week. > >> > > > > >> > > >
Re: TLP tools for stress testing and building test clusters in AWS
Wednesday would work for me. We use and (slightly) contribute to tlp tools. We are platform testing and beginning 4.0 testing ourselves, so an in person overview would be great! On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko wrote: > Wednesday and Thursday, either, at 9 AM pacific WFM. > > > On 13 Apr 2019, at 13:31, Stefan Miklosovic < > stefan.mikloso...@instaclustr.com> wrote: > > > > Hi Jon, > > > > I would like be on that call too but I am off on Thursday. > > > > I am from Australia so 5pm London time is ours 2am next day so your > > Wednesday morning is my Thursday night. Wednesday early morning so > > your Tuesday morning and London's afternoon would be the best. > > > > Recording the thing would be definitely helpful too. > > > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > >> > >> I'd be more than happy to hop on a call next week to give you both > >> (and anyone else interested) a tour of our dev tools. Maybe something > >> early morning on my end, which should be your evening, could work? > >> > >> I can set up a Zoom conference to get everyone acquainted. We can > >> record and post it for any who can't make it. > >> > >> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm > >> London)? If anyone's interested please reply with what dates work. > >> I'll be sure to post the details back here with the zoom link in case > >> anyone wants to join that didn't get a chance to reply, as well as a > >> link to the recorded call. > >> > >> Jon > >> > >> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > >> wrote: > >>> > >>> +1 > >>> > >>> I’m also just as excited to see some standardised workloads and test > bed. At the moment we’re benefiting from some large contributors doing > their own proprietary performance testing, which is super valuable and > something we’ve lacked before. But I’m also keen to see some more > representative workloads that are reproducible by anybody in the community > take shape. > >>> > >>> > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > wrote: > > Hey Jon, > > This sounds exciting and pretty useful, thanks. > > Looking forward to using tlp-stress for validating 15066 performance. > > We should touch base some time next week to pick a comprehensive set > of workloads and versions, perhaps? > > > > On 12 Apr 2019, at 16:34, Jon Haddad wrote: > > > > I don't want to derail the discussion about Stabilizing Internode > > Messaging, so I'm starting this as a separate thread. There was a > > comment that Josh made [1] about doing performance testing with real > > clusters as well as a lot of microbenchmarks, and I'm 100% in support > > of this. We've been working on some tooling at TLP for the last > > several months to make this a lot easier. One of the goals has been > > to help improve the 4.0 testing process. > > > > The first tool we have is tlp-stress [2]. It's designed with a "get > > started in 5 minutes" mindset. My goal was to ship a stress tool > that > > ships with real workloads out of the box that can be easily tweaked, > > similar to how fio allows you to design a disk workload and tweak it > > with paramaters. Included are stress workloads that stress LWTs (two > > different types), materialized views, counters, time series, and > > key-value workloads. Each workload can be modified easily to change > > compaction strategies, concurrent operations, number of partitions. > > We can run workloads for a set number of iterations or a custom > > duration. We've used this *extensively* at TLP to help our customers > > and most of our blog posts that discuss performance use it as well. > > It exports data to both a CSV format and auto sets up prometheus for > > metrics collection / aggregation. As an example, we were able to > > determine that the compression length set on the paxos tables imposes > > a significant overhead when using the Locking LWT workload, which > > simulates locking and unlocking of rows. See CASSANDRA-15080 for > > details. > > > > We have documentation [3] on the TLP website. > > > > The second tool we've been working on is tlp-cluster [4]. This tool > > is designed to help provision AWS instances for the purposes of > > testing. To be clear, I don't expect, or want, this tool to be used > > for production environments. It's designed to assist with the > > Cassandra build process by generating deb packages or re-using the > > ones that have already been uploaded. Here's a short list of the > > things you'll care about: > > > > 1. Create instances in AWS for Cassandra using any instance size and > > number of nodes. Also create tlp-stress instances and a box for > > monitoring > > 2. Use any available build of Cassandra, with a quick option to > change > > YAML
