Re: TLP tools for stress testing and building test clusters in AWS

2019-05-05 Thread Anthony Grasso
Hi Stefan.

Sorry for the late reply. Just wanted to close the loop on this.

Glad you found the demo useful.

On related node, we found out what the issues was when we were going
through the demo. It was to do with one of the AZs in our account being
unavailable. Even though AWS says an AZ exists, it may still be unavailable
for use. When provisioning a cluster, you might get an error like this in
the log:

aws_instance.cassandra.3: Error launching source instance: Unsupported:
> Your requested instance type (r3.2xlarge) is not supported in your
> requested Availability Zone (us-west-2d). Please retry your request by not
> specifying an Availability Zone or choosing us-west-2c, us-west-2a,
> us-west-2b.


To address this, we've added an --azs flag which can be used as a
workaround when an AZ unavailable. You can use it by supplying the letter
of the AZ you want. For example --azs abc.

Regards,
Anthony


On Wed, 17 Apr 2019 at 15:49, Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> wrote:

> Thanks Anthony for going that proverbial extra mile to cover people in
> different time zones too.
>
> I believe other people will find your talk as helpful as we did.
>
> Regards
>
> On Wed, 17 Apr 2019 at 10:08, Anthony Grasso 
> wrote:
> >
> > Hi Stefan and devs,
> >
> > I have set up a zoom link for the TLP tool set intro that will be on in
> an
> > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772
> >
> > This link is open so if anyone else wishes to join they are welcome to do
> > so. I will be covering the same topics Jon is covering in his meeting
> > tomorrow.
> >
> > Regards,
> > Anthony
> >
> >
> > On Wed, 17 Apr 2019 at 08:29, Anthony Grasso 
> > wrote:
> >
> > > Hi Stefan,
> > >
> > > Thanks for sending the invite out!
> > >
> > > Just wondering what do you think of the idea of having a Zoom meeting
> that
> > > anyone can join? This way anyone else interested can join us as well.
> I can
> > > set that up if you like?
> > >
> > > Cheers,
> > > Anthony
> > >
> > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> > > stefan.mikloso...@instaclustr.com> wrote:
> > >
> > >> Hi Anthony,
> > >>
> > >> sounds good. I ve sent you Hangouts meeting invitation privately.
> > >>
> > >> Regards
> > >>
> > >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso <
> anthony.gra...@gmail.com>
> > >> wrote:
> > >> >
> > >> > Hi Stefan,
> > >> >
> > >> > I have been working with Jon on developing the tool set. I can do a
> Zoom
> > >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We
> can go
> > >> > through all the same information that Jon is going to go through in
> his
> > >> > call. Note that I am in the same timezone as you, so if tomorrow
> > >> morning is
> > >> > no good we can always do the afternoon.
> > >> >
> > >> > Cheers,
> > >> > Anthony
> > >> >
> > >> >
> > >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> > >> > stefan.mikloso...@instaclustr.com> wrote:
> > >> >
> > >> > > Hi Jon,
> > >> > >
> > >> > > I would like be on that call too but I am off on Thursday.
> > >> > >
> > >> > > I am from Australia so 5pm London time is ours 2am next day so
> your
> > >> > > Wednesday morning is my Thursday night. Wednesday early morning so
> > >> > > your Tuesday morning and London's afternoon would be the best.
> > >> > >
> > >> > > Recording the thing would be definitely helpful too.
> > >> > >
> > >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad 
> wrote:
> > >> > > >
> > >> > > > I'd be more than happy to hop on a call next week to give you
> both
> > >> > > > (and anyone else interested) a tour of our dev tools.  Maybe
> > >> something
> > >> > > > early morning on my end, which should be your evening, could
> work?
> > >> > > >
> > >> > > > I can set up a Zoom conference to get everyone acquainted.  We
> can
> > >> > > > record and post it for any who can't make it.
> > >> > > >
> > >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM
> Pacific
> > >> (5pm
> > >> > > > London)?  If anyone's interested please reply with what dates
> work.
> > >> > > > I'll be sure to post the details back here with the zoom link in
> > >> case
> > >> > > > anyone wants to join that didn't get a chance to reply, as well
> as a
> > >> > > > link to the recorded call.
> > >> > > >
> > >> > > > Jon
> > >> > > >
> > >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> > >> > > >  wrote:
> > >> > > > >
> > >> > > > > +1
> > >> > > > >
> > >> > > > > I’m also just as excited to see some standardised workloads
> and
> > >> test
> > >> > > bed.  At the moment we’re benefiting from some large contributors
> > >> doing
> > >> > > their own proprietary performance testing, which is super
> valuable and
> > >> > > something we’ve lacked before.  But I’m also keen to see some more
> > >> > > representative workloads that are reproducible by anybody in the
> > >> community
> > >> > > take shape.
> > >> > > > >
> > >> > > > >
> > >> > > > > > On 12 Apr 2019, at 18:09, 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-17 Thread Jon Haddad
Hey folks.  I've opened the 9am zoom session.

You can join here: https://zoom.us/j/189920888


On Tue, Apr 16, 2019 at 10:49 PM Stefan Miklosovic
 wrote:
>
> Thanks Anthony for going that proverbial extra mile to cover people in
> different time zones too.
>
> I believe other people will find your talk as helpful as we did.
>
> Regards
>
> On Wed, 17 Apr 2019 at 10:08, Anthony Grasso  wrote:
> >
> > Hi Stefan and devs,
> >
> > I have set up a zoom link for the TLP tool set intro that will be on in an
> > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772
> >
> > This link is open so if anyone else wishes to join they are welcome to do
> > so. I will be covering the same topics Jon is covering in his meeting
> > tomorrow.
> >
> > Regards,
> > Anthony
> >
> >
> > On Wed, 17 Apr 2019 at 08:29, Anthony Grasso 
> > wrote:
> >
> > > Hi Stefan,
> > >
> > > Thanks for sending the invite out!
> > >
> > > Just wondering what do you think of the idea of having a Zoom meeting that
> > > anyone can join? This way anyone else interested can join us as well. I 
> > > can
> > > set that up if you like?
> > >
> > > Cheers,
> > > Anthony
> > >
> > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> > > stefan.mikloso...@instaclustr.com> wrote:
> > >
> > >> Hi Anthony,
> > >>
> > >> sounds good. I ve sent you Hangouts meeting invitation privately.
> > >>
> > >> Regards
> > >>
> > >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
> > >> wrote:
> > >> >
> > >> > Hi Stefan,
> > >> >
> > >> > I have been working with Jon on developing the tool set. I can do a 
> > >> > Zoom
> > >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
> > >> > through all the same information that Jon is going to go through in his
> > >> > call. Note that I am in the same timezone as you, so if tomorrow
> > >> morning is
> > >> > no good we can always do the afternoon.
> > >> >
> > >> > Cheers,
> > >> > Anthony
> > >> >
> > >> >
> > >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> > >> > stefan.mikloso...@instaclustr.com> wrote:
> > >> >
> > >> > > Hi Jon,
> > >> > >
> > >> > > I would like be on that call too but I am off on Thursday.
> > >> > >
> > >> > > I am from Australia so 5pm London time is ours 2am next day so your
> > >> > > Wednesday morning is my Thursday night. Wednesday early morning so
> > >> > > your Tuesday morning and London's afternoon would be the best.
> > >> > >
> > >> > > Recording the thing would be definitely helpful too.
> > >> > >
> > >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> > >> > > >
> > >> > > > I'd be more than happy to hop on a call next week to give you both
> > >> > > > (and anyone else interested) a tour of our dev tools.  Maybe
> > >> something
> > >> > > > early morning on my end, which should be your evening, could work?
> > >> > > >
> > >> > > > I can set up a Zoom conference to get everyone acquainted.  We can
> > >> > > > record and post it for any who can't make it.
> > >> > > >
> > >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
> > >> (5pm
> > >> > > > London)?  If anyone's interested please reply with what dates work.
> > >> > > > I'll be sure to post the details back here with the zoom link in
> > >> case
> > >> > > > anyone wants to join that didn't get a chance to reply, as well as 
> > >> > > > a
> > >> > > > link to the recorded call.
> > >> > > >
> > >> > > > Jon
> > >> > > >
> > >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> > >> > > >  wrote:
> > >> > > > >
> > >> > > > > +1
> > >> > > > >
> > >> > > > > I’m also just as excited to see some standardised workloads and
> > >> test
> > >> > > bed.  At the moment we’re benefiting from some large contributors
> > >> doing
> > >> > > their own proprietary performance testing, which is super valuable 
> > >> > > and
> > >> > > something we’ve lacked before.  But I’m also keen to see some more
> > >> > > representative workloads that are reproducible by anybody in the
> > >> community
> > >> > > take shape.
> > >> > > > >
> > >> > > > >
> > >> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> > >> > >  wrote:
> > >> > > > > >
> > >> > > > > > Hey Jon,
> > >> > > > > >
> > >> > > > > > This sounds exciting and pretty useful, thanks.
> > >> > > > > >
> > >> > > > > > Looking forward to using tlp-stress for validating 15066
> > >> performance.
> > >> > > > > >
> > >> > > > > > We should touch base some time next week to pick a
> > >> comprehensive set
> > >> > > of workloads and versions, perhaps?
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad 
> > >> wrote:
> > >> > > > > >>
> > >> > > > > >> I don't want to derail the discussion about Stabilizing
> > >> Internode
> > >> > > > > >> Messaging, so I'm starting this as a separate thread.  There
> > >> was a
> > >> > > > > >> comment that Josh made [1] about doing performance testing
> > >> with real
> > >> > > > > >> clusters as well as a lot of 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Stefan Miklosovic
Thanks Anthony for going that proverbial extra mile to cover people in
different time zones too.

I believe other people will find your talk as helpful as we did.

Regards

On Wed, 17 Apr 2019 at 10:08, Anthony Grasso  wrote:
>
> Hi Stefan and devs,
>
> I have set up a zoom link for the TLP tool set intro that will be on in an
> hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772
>
> This link is open so if anyone else wishes to join they are welcome to do
> so. I will be covering the same topics Jon is covering in his meeting
> tomorrow.
>
> Regards,
> Anthony
>
>
> On Wed, 17 Apr 2019 at 08:29, Anthony Grasso 
> wrote:
>
> > Hi Stefan,
> >
> > Thanks for sending the invite out!
> >
> > Just wondering what do you think of the idea of having a Zoom meeting that
> > anyone can join? This way anyone else interested can join us as well. I can
> > set that up if you like?
> >
> > Cheers,
> > Anthony
> >
> > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> >> Hi Anthony,
> >>
> >> sounds good. I ve sent you Hangouts meeting invitation privately.
> >>
> >> Regards
> >>
> >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
> >> wrote:
> >> >
> >> > Hi Stefan,
> >> >
> >> > I have been working with Jon on developing the tool set. I can do a Zoom
> >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
> >> > through all the same information that Jon is going to go through in his
> >> > call. Note that I am in the same timezone as you, so if tomorrow
> >> morning is
> >> > no good we can always do the afternoon.
> >> >
> >> > Cheers,
> >> > Anthony
> >> >
> >> >
> >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> >> > stefan.mikloso...@instaclustr.com> wrote:
> >> >
> >> > > Hi Jon,
> >> > >
> >> > > I would like be on that call too but I am off on Thursday.
> >> > >
> >> > > I am from Australia so 5pm London time is ours 2am next day so your
> >> > > Wednesday morning is my Thursday night. Wednesday early morning so
> >> > > your Tuesday morning and London's afternoon would be the best.
> >> > >
> >> > > Recording the thing would be definitely helpful too.
> >> > >
> >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> >> > > >
> >> > > > I'd be more than happy to hop on a call next week to give you both
> >> > > > (and anyone else interested) a tour of our dev tools.  Maybe
> >> something
> >> > > > early morning on my end, which should be your evening, could work?
> >> > > >
> >> > > > I can set up a Zoom conference to get everyone acquainted.  We can
> >> > > > record and post it for any who can't make it.
> >> > > >
> >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
> >> (5pm
> >> > > > London)?  If anyone's interested please reply with what dates work.
> >> > > > I'll be sure to post the details back here with the zoom link in
> >> case
> >> > > > anyone wants to join that didn't get a chance to reply, as well as a
> >> > > > link to the recorded call.
> >> > > >
> >> > > > Jon
> >> > > >
> >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> >> > > >  wrote:
> >> > > > >
> >> > > > > +1
> >> > > > >
> >> > > > > I’m also just as excited to see some standardised workloads and
> >> test
> >> > > bed.  At the moment we’re benefiting from some large contributors
> >> doing
> >> > > their own proprietary performance testing, which is super valuable and
> >> > > something we’ve lacked before.  But I’m also keen to see some more
> >> > > representative workloads that are reproducible by anybody in the
> >> community
> >> > > take shape.
> >> > > > >
> >> > > > >
> >> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> >> > >  wrote:
> >> > > > > >
> >> > > > > > Hey Jon,
> >> > > > > >
> >> > > > > > This sounds exciting and pretty useful, thanks.
> >> > > > > >
> >> > > > > > Looking forward to using tlp-stress for validating 15066
> >> performance.
> >> > > > > >
> >> > > > > > We should touch base some time next week to pick a
> >> comprehensive set
> >> > > of workloads and versions, perhaps?
> >> > > > > >
> >> > > > > >
> >> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad 
> >> wrote:
> >> > > > > >>
> >> > > > > >> I don't want to derail the discussion about Stabilizing
> >> Internode
> >> > > > > >> Messaging, so I'm starting this as a separate thread.  There
> >> was a
> >> > > > > >> comment that Josh made [1] about doing performance testing
> >> with real
> >> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in
> >> > > support
> >> > > > > >> of this.  We've been working on some tooling at TLP for the
> >> last
> >> > > > > >> several months to make this a lot easier.  One of the goals
> >> has been
> >> > > > > >> to help improve the 4.0 testing process.
> >> > > > > >>
> >> > > > > >> The first tool we have is tlp-stress [2].  It's designed with
> >> a "get
> >> > > > > >> started in 5 minutes" mindset.  My goal was to ship a stress
> 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Nate McCall
Just noticed the date - to be clear, this is like in ~45 mins?

On Wed, Apr 17, 2019 at 12:11 PM Nate McCall  wrote:
>
> > I have set up a zoom link for the TLP tool set intro that will be on in an
> > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772
> >
> Super rad - thanks for setting this up, Anthony!

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org



Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Nate McCall
> I have set up a zoom link for the TLP tool set intro that will be on in an
> hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772
>
Super rad - thanks for setting this up, Anthony!

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org



Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Anthony Grasso
Hi Stefan and devs,

I have set up a zoom link for the TLP tool set intro that will be on in an
hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772

This link is open so if anyone else wishes to join they are welcome to do
so. I will be covering the same topics Jon is covering in his meeting
tomorrow.

Regards,
Anthony


On Wed, 17 Apr 2019 at 08:29, Anthony Grasso 
wrote:

> Hi Stefan,
>
> Thanks for sending the invite out!
>
> Just wondering what do you think of the idea of having a Zoom meeting that
> anyone can join? This way anyone else interested can join us as well. I can
> set that up if you like?
>
> Cheers,
> Anthony
>
> On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
>
>> Hi Anthony,
>>
>> sounds good. I ve sent you Hangouts meeting invitation privately.
>>
>> Regards
>>
>> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
>> wrote:
>> >
>> > Hi Stefan,
>> >
>> > I have been working with Jon on developing the tool set. I can do a Zoom
>> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
>> > through all the same information that Jon is going to go through in his
>> > call. Note that I am in the same timezone as you, so if tomorrow
>> morning is
>> > no good we can always do the afternoon.
>> >
>> > Cheers,
>> > Anthony
>> >
>> >
>> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
>> > stefan.mikloso...@instaclustr.com> wrote:
>> >
>> > > Hi Jon,
>> > >
>> > > I would like be on that call too but I am off on Thursday.
>> > >
>> > > I am from Australia so 5pm London time is ours 2am next day so your
>> > > Wednesday morning is my Thursday night. Wednesday early morning so
>> > > your Tuesday morning and London's afternoon would be the best.
>> > >
>> > > Recording the thing would be definitely helpful too.
>> > >
>> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
>> > > >
>> > > > I'd be more than happy to hop on a call next week to give you both
>> > > > (and anyone else interested) a tour of our dev tools.  Maybe
>> something
>> > > > early morning on my end, which should be your evening, could work?
>> > > >
>> > > > I can set up a Zoom conference to get everyone acquainted.  We can
>> > > > record and post it for any who can't make it.
>> > > >
>> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
>> (5pm
>> > > > London)?  If anyone's interested please reply with what dates work.
>> > > > I'll be sure to post the details back here with the zoom link in
>> case
>> > > > anyone wants to join that didn't get a chance to reply, as well as a
>> > > > link to the recorded call.
>> > > >
>> > > > Jon
>> > > >
>> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
>> > > >  wrote:
>> > > > >
>> > > > > +1
>> > > > >
>> > > > > I’m also just as excited to see some standardised workloads and
>> test
>> > > bed.  At the moment we’re benefiting from some large contributors
>> doing
>> > > their own proprietary performance testing, which is super valuable and
>> > > something we’ve lacked before.  But I’m also keen to see some more
>> > > representative workloads that are reproducible by anybody in the
>> community
>> > > take shape.
>> > > > >
>> > > > >
>> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
>> > >  wrote:
>> > > > > >
>> > > > > > Hey Jon,
>> > > > > >
>> > > > > > This sounds exciting and pretty useful, thanks.
>> > > > > >
>> > > > > > Looking forward to using tlp-stress for validating 15066
>> performance.
>> > > > > >
>> > > > > > We should touch base some time next week to pick a
>> comprehensive set
>> > > of workloads and versions, perhaps?
>> > > > > >
>> > > > > >
>> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad 
>> wrote:
>> > > > > >>
>> > > > > >> I don't want to derail the discussion about Stabilizing
>> Internode
>> > > > > >> Messaging, so I'm starting this as a separate thread.  There
>> was a
>> > > > > >> comment that Josh made [1] about doing performance testing
>> with real
>> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in
>> > > support
>> > > > > >> of this.  We've been working on some tooling at TLP for the
>> last
>> > > > > >> several months to make this a lot easier.  One of the goals
>> has been
>> > > > > >> to help improve the 4.0 testing process.
>> > > > > >>
>> > > > > >> The first tool we have is tlp-stress [2].  It's designed with
>> a "get
>> > > > > >> started in 5 minutes" mindset.  My goal was to ship a stress
>> tool
>> > > that
>> > > > > >> ships with real workloads out of the box that can be easily
>> tweaked,
>> > > > > >> similar to how fio allows you to design a disk workload and
>> tweak it
>> > > > > >> with paramaters.  Included are stress workloads that stress
>> LWTs
>> > > (two
>> > > > > >> different types), materialized views, counters, time series,
>> and
>> > > > > >> key-value workloads.  Each workload can be modified easily to
>> change
>> > > > > >> compaction strategies, 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Jon Haddad
The one I sent out is open, no separate invite required.

On Tue, Apr 16, 2019 at 3:47 PM Dinesh Joshi  wrote:
>
> I'm slightly confused. The zoom meeting mentioned in this thread is only open 
> to who have registered interest here? If so, can someone please add me?
>
> Dinesh
>
> > On Apr 16, 2019, at 3:29 PM, Anthony Grasso  
> > wrote:
> >
> > Hi Stefan,
> >
> > Thanks for sending the invite out!
> >
> > Just wondering what do you think of the idea of having a Zoom meeting that
> > anyone can join? This way anyone else interested can join us as well. I can
> > set that up if you like?
> >
> > Cheers,
> > Anthony
> >
> > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> >> Hi Anthony,
> >>
> >> sounds good. I ve sent you Hangouts meeting invitation privately.
> >>
> >> Regards
> >>
> >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
> >> wrote:
> >>>
> >>> Hi Stefan,
> >>>
> >>> I have been working with Jon on developing the tool set. I can do a Zoom
> >>> call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
> >>> through all the same information that Jon is going to go through in his
> >>> call. Note that I am in the same timezone as you, so if tomorrow morning
> >> is
> >>> no good we can always do the afternoon.
> >>>
> >>> Cheers,
> >>> Anthony
> >>>
> >>>
> >>> On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> >>> stefan.mikloso...@instaclustr.com> wrote:
> >>>
>  Hi Jon,
> 
>  I would like be on that call too but I am off on Thursday.
> 
>  I am from Australia so 5pm London time is ours 2am next day so your
>  Wednesday morning is my Thursday night. Wednesday early morning so
>  your Tuesday morning and London's afternoon would be the best.
> 
>  Recording the thing would be definitely helpful too.
> 
>  On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> >
> > I'd be more than happy to hop on a call next week to give you both
> > (and anyone else interested) a tour of our dev tools.  Maybe
> >> something
> > early morning on my end, which should be your evening, could work?
> >
> > I can set up a Zoom conference to get everyone acquainted.  We can
> > record and post it for any who can't make it.
> >
> > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
> >> (5pm
> > London)?  If anyone's interested please reply with what dates work.
> > I'll be sure to post the details back here with the zoom link in case
> > anyone wants to join that didn't get a chance to reply, as well as a
> > link to the recorded call.
> >
> > Jon
> >
> > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> >  wrote:
> >>
> >> +1
> >>
> >> I’m also just as excited to see some standardised workloads and
> >> test
>  bed.  At the moment we’re benefiting from some large contributors doing
>  their own proprietary performance testing, which is super valuable and
>  something we’ve lacked before.  But I’m also keen to see some more
>  representative workloads that are reproducible by anybody in the
> >> community
>  take shape.
> >>
> >>
> >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
>   wrote:
> >>>
> >>> Hey Jon,
> >>>
> >>> This sounds exciting and pretty useful, thanks.
> >>>
> >>> Looking forward to using tlp-stress for validating 15066
> >> performance.
> >>>
> >>> We should touch base some time next week to pick a comprehensive
> >> set
>  of workloads and versions, perhaps?
> >>>
> >>>
>  On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> 
>  I don't want to derail the discussion about Stabilizing
> >> Internode
>  Messaging, so I'm starting this as a separate thread.  There
> >> was a
>  comment that Josh made [1] about doing performance testing with
> >> real
>  clusters as well as a lot of microbenchmarks, and I'm 100% in
>  support
>  of this.  We've been working on some tooling at TLP for the last
>  several months to make this a lot easier.  One of the goals has
> >> been
>  to help improve the 4.0 testing process.
> 
>  The first tool we have is tlp-stress [2].  It's designed with a
> >> "get
>  started in 5 minutes" mindset.  My goal was to ship a stress
> >> tool
>  that
>  ships with real workloads out of the box that can be easily
> >> tweaked,
>  similar to how fio allows you to design a disk workload and
> >> tweak it
>  with paramaters.  Included are stress workloads that stress LWTs
>  (two
>  different types), materialized views, counters, time series, and
>  key-value workloads.  Each workload can be modified easily to
> >> change
>  compaction strategies, concurrent operations, number of
> >> partitions.
>  We can run workloads for a set 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Dinesh Joshi
I'm slightly confused. The zoom meeting mentioned in this thread is only open 
to who have registered interest here? If so, can someone please add me?

Dinesh

> On Apr 16, 2019, at 3:29 PM, Anthony Grasso  wrote:
> 
> Hi Stefan,
> 
> Thanks for sending the invite out!
> 
> Just wondering what do you think of the idea of having a Zoom meeting that
> anyone can join? This way anyone else interested can join us as well. I can
> set that up if you like?
> 
> Cheers,
> Anthony
> 
> On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
> 
>> Hi Anthony,
>> 
>> sounds good. I ve sent you Hangouts meeting invitation privately.
>> 
>> Regards
>> 
>> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
>> wrote:
>>> 
>>> Hi Stefan,
>>> 
>>> I have been working with Jon on developing the tool set. I can do a Zoom
>>> call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
>>> through all the same information that Jon is going to go through in his
>>> call. Note that I am in the same timezone as you, so if tomorrow morning
>> is
>>> no good we can always do the afternoon.
>>> 
>>> Cheers,
>>> Anthony
>>> 
>>> 
>>> On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
>>> stefan.mikloso...@instaclustr.com> wrote:
>>> 
 Hi Jon,
 
 I would like be on that call too but I am off on Thursday.
 
 I am from Australia so 5pm London time is ours 2am next day so your
 Wednesday morning is my Thursday night. Wednesday early morning so
 your Tuesday morning and London's afternoon would be the best.
 
 Recording the thing would be definitely helpful too.
 
 On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> 
> I'd be more than happy to hop on a call next week to give you both
> (and anyone else interested) a tour of our dev tools.  Maybe
>> something
> early morning on my end, which should be your evening, could work?
> 
> I can set up a Zoom conference to get everyone acquainted.  We can
> record and post it for any who can't make it.
> 
> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
>> (5pm
> London)?  If anyone's interested please reply with what dates work.
> I'll be sure to post the details back here with the zoom link in case
> anyone wants to join that didn't get a chance to reply, as well as a
> link to the recorded call.
> 
> Jon
> 
> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
>  wrote:
>> 
>> +1
>> 
>> I’m also just as excited to see some standardised workloads and
>> test
 bed.  At the moment we’re benefiting from some large contributors doing
 their own proprietary performance testing, which is super valuable and
 something we’ve lacked before.  But I’m also keen to see some more
 representative workloads that are reproducible by anybody in the
>> community
 take shape.
>> 
>> 
>>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
  wrote:
>>> 
>>> Hey Jon,
>>> 
>>> This sounds exciting and pretty useful, thanks.
>>> 
>>> Looking forward to using tlp-stress for validating 15066
>> performance.
>>> 
>>> We should touch base some time next week to pick a comprehensive
>> set
 of workloads and versions, perhaps?
>>> 
>>> 
 On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
 
 I don't want to derail the discussion about Stabilizing
>> Internode
 Messaging, so I'm starting this as a separate thread.  There
>> was a
 comment that Josh made [1] about doing performance testing with
>> real
 clusters as well as a lot of microbenchmarks, and I'm 100% in
 support
 of this.  We've been working on some tooling at TLP for the last
 several months to make this a lot easier.  One of the goals has
>> been
 to help improve the 4.0 testing process.
 
 The first tool we have is tlp-stress [2].  It's designed with a
>> "get
 started in 5 minutes" mindset.  My goal was to ship a stress
>> tool
 that
 ships with real workloads out of the box that can be easily
>> tweaked,
 similar to how fio allows you to design a disk workload and
>> tweak it
 with paramaters.  Included are stress workloads that stress LWTs
 (two
 different types), materialized views, counters, time series, and
 key-value workloads.  Each workload can be modified easily to
>> change
 compaction strategies, concurrent operations, number of
>> partitions.
 We can run workloads for a set number of iterations or a custom
 duration.  We've used this *extensively* at TLP to help our
 customers
 and most of our blog posts that discuss performance use it as
>> well.
 It exports data to both a CSV format and auto sets up
>> prometheus for
 metrics collection / aggregation.  As an example, we were able
>> to

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Anthony Grasso
Hi Stefan,

Thanks for sending the invite out!

Just wondering what do you think of the idea of having a Zoom meeting that
anyone can join? This way anyone else interested can join us as well. I can
set that up if you like?

Cheers,
Anthony

On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> wrote:

> Hi Anthony,
>
> sounds good. I ve sent you Hangouts meeting invitation privately.
>
> Regards
>
> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso 
> wrote:
> >
> > Hi Stefan,
> >
> > I have been working with Jon on developing the tool set. I can do a Zoom
> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
> > through all the same information that Jon is going to go through in his
> > call. Note that I am in the same timezone as you, so if tomorrow morning
> is
> > no good we can always do the afternoon.
> >
> > Cheers,
> > Anthony
> >
> >
> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> > > Hi Jon,
> > >
> > > I would like be on that call too but I am off on Thursday.
> > >
> > > I am from Australia so 5pm London time is ours 2am next day so your
> > > Wednesday morning is my Thursday night. Wednesday early morning so
> > > your Tuesday morning and London's afternoon would be the best.
> > >
> > > Recording the thing would be definitely helpful too.
> > >
> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> > > >
> > > > I'd be more than happy to hop on a call next week to give you both
> > > > (and anyone else interested) a tour of our dev tools.  Maybe
> something
> > > > early morning on my end, which should be your evening, could work?
> > > >
> > > > I can set up a Zoom conference to get everyone acquainted.  We can
> > > > record and post it for any who can't make it.
> > > >
> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
> (5pm
> > > > London)?  If anyone's interested please reply with what dates work.
> > > > I'll be sure to post the details back here with the zoom link in case
> > > > anyone wants to join that didn't get a chance to reply, as well as a
> > > > link to the recorded call.
> > > >
> > > > Jon
> > > >
> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> > > >  wrote:
> > > > >
> > > > > +1
> > > > >
> > > > > I’m also just as excited to see some standardised workloads and
> test
> > > bed.  At the moment we’re benefiting from some large contributors doing
> > > their own proprietary performance testing, which is super valuable and
> > > something we’ve lacked before.  But I’m also keen to see some more
> > > representative workloads that are reproducible by anybody in the
> community
> > > take shape.
> > > > >
> > > > >
> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> > >  wrote:
> > > > > >
> > > > > > Hey Jon,
> > > > > >
> > > > > > This sounds exciting and pretty useful, thanks.
> > > > > >
> > > > > > Looking forward to using tlp-stress for validating 15066
> performance.
> > > > > >
> > > > > > We should touch base some time next week to pick a comprehensive
> set
> > > of workloads and versions, perhaps?
> > > > > >
> > > > > >
> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> > > > > >>
> > > > > >> I don't want to derail the discussion about Stabilizing
> Internode
> > > > > >> Messaging, so I'm starting this as a separate thread.  There
> was a
> > > > > >> comment that Josh made [1] about doing performance testing with
> real
> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in
> > > support
> > > > > >> of this.  We've been working on some tooling at TLP for the last
> > > > > >> several months to make this a lot easier.  One of the goals has
> been
> > > > > >> to help improve the 4.0 testing process.
> > > > > >>
> > > > > >> The first tool we have is tlp-stress [2].  It's designed with a
> "get
> > > > > >> started in 5 minutes" mindset.  My goal was to ship a stress
> tool
> > > that
> > > > > >> ships with real workloads out of the box that can be easily
> tweaked,
> > > > > >> similar to how fio allows you to design a disk workload and
> tweak it
> > > > > >> with paramaters.  Included are stress workloads that stress LWTs
> > > (two
> > > > > >> different types), materialized views, counters, time series, and
> > > > > >> key-value workloads.  Each workload can be modified easily to
> change
> > > > > >> compaction strategies, concurrent operations, number of
> partitions.
> > > > > >> We can run workloads for a set number of iterations or a custom
> > > > > >> duration.  We've used this *extensively* at TLP to help our
> > > customers
> > > > > >> and most of our blog posts that discuss performance use it as
> well.
> > > > > >> It exports data to both a CSV format and auto sets up
> prometheus for
> > > > > >> metrics collection / aggregation.  As an example, we were able
> to
> > > > > >> determine that the compression length set on the paxos tables
> > > imposes
> > > 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Jon Haddad
Yes, sorry about that. Wednesday morning 9am PT

On Tue, Apr 16, 2019 at 3:26 AM Benedict Elliott Smith 
wrote:

> Just to confirm, this is on Wednesday?
>
> > On 15 Apr 2019, at 22:38, Jon Haddad  wrote:
> >
> > Hey all,
> >
> > I've set up a Zoom call for 9AM Pacific time.  Everyone's welcome to
> join.
> >
> > https://zoom.us/j/189920888
> >
> > Looking forward to a good discussion on how we can all pitch in on
> > getting 4.0 out the door.
> >
> > Jon
> >
> > On Sat, Apr 13, 2019 at 9:14 AM Jonathan Koppenhofer
> >  wrote:
> >>
> >> Wednesday would work for me.
> >>
> >> We use and (slightly) contribute to tlp tools. We are platform testing
> and
> >> beginning 4.0 testing ourselves, so an in person overview would be
> great!
> >>
> >> On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko
> 
> >> wrote:
> >>
> >>> Wednesday and Thursday, either, at 9 AM pacific WFM.
> >>>
>  On 13 Apr 2019, at 13:31, Stefan Miklosovic <
> >>> stefan.mikloso...@instaclustr.com> wrote:
> 
>  Hi Jon,
> 
>  I would like be on that call too but I am off on Thursday.
> 
>  I am from Australia so 5pm London time is ours 2am next day so your
>  Wednesday morning is my Thursday night. Wednesday early morning so
>  your Tuesday morning and London's afternoon would be the best.
> 
>  Recording the thing would be definitely helpful too.
> 
>  On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> >
> > I'd be more than happy to hop on a call next week to give you both
> > (and anyone else interested) a tour of our dev tools.  Maybe
> something
> > early morning on my end, which should be your evening, could work?
> >
> > I can set up a Zoom conference to get everyone acquainted.  We can
> > record and post it for any who can't make it.
> >
> > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific
> (5pm
> > London)?  If anyone's interested please reply with what dates work.
> > I'll be sure to post the details back here with the zoom link in case
> > anyone wants to join that didn't get a chance to reply, as well as a
> > link to the recorded call.
> >
> > Jon
> >
> > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> >  wrote:
> >>
> >> +1
> >>
> >> I’m also just as excited to see some standardised workloads and test
> >>> bed.  At the moment we’re benefiting from some large contributors doing
> >>> their own proprietary performance testing, which is super valuable and
> >>> something we’ve lacked before.  But I’m also keen to see some more
> >>> representative workloads that are reproducible by anybody in the
> community
> >>> take shape.
> >>
> >>
> >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> >>>  wrote:
> >>>
> >>> Hey Jon,
> >>>
> >>> This sounds exciting and pretty useful, thanks.
> >>>
> >>> Looking forward to using tlp-stress for validating 15066
> performance.
> >>>
> >>> We should touch base some time next week to pick a comprehensive
> set
> >>> of workloads and versions, perhaps?
> >>>
> >>>
>  On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> 
>  I don't want to derail the discussion about Stabilizing Internode
>  Messaging, so I'm starting this as a separate thread.  There was a
>  comment that Josh made [1] about doing performance testing with
> real
>  clusters as well as a lot of microbenchmarks, and I'm 100% in
> support
>  of this.  We've been working on some tooling at TLP for the last
>  several months to make this a lot easier.  One of the goals has
> been
>  to help improve the 4.0 testing process.
> 
>  The first tool we have is tlp-stress [2].  It's designed with a
> "get
>  started in 5 minutes" mindset.  My goal was to ship a stress tool
> >>> that
>  ships with real workloads out of the box that can be easily
> tweaked,
>  similar to how fio allows you to design a disk workload and tweak
> it
>  with paramaters.  Included are stress workloads that stress LWTs
> (two
>  different types), materialized views, counters, time series, and
>  key-value workloads.  Each workload can be modified easily to
> change
>  compaction strategies, concurrent operations, number of
> partitions.
>  We can run workloads for a set number of iterations or a custom
>  duration.  We've used this *extensively* at TLP to help our
> customers
>  and most of our blog posts that discuss performance use it as
> well.
>  It exports data to both a CSV format and auto sets up prometheus
> for
>  metrics collection / aggregation.  As an example, we were able to
>  determine that the compression length set on the paxos tables
> imposes
>  a significant overhead when using the Locking LWT workload, which
>  simulates locking and 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Stefan Miklosovic
Hi Anthony,

sounds good. I ve sent you Hangouts meeting invitation privately.

Regards

On Tue, 16 Apr 2019 at 14:53, Anthony Grasso  wrote:
>
> Hi Stefan,
>
> I have been working with Jon on developing the tool set. I can do a Zoom
> call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
> through all the same information that Jon is going to go through in his
> call. Note that I am in the same timezone as you, so if tomorrow morning is
> no good we can always do the afternoon.
>
> Cheers,
> Anthony
>
>
> On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
>
> > Hi Jon,
> >
> > I would like be on that call too but I am off on Thursday.
> >
> > I am from Australia so 5pm London time is ours 2am next day so your
> > Wednesday morning is my Thursday night. Wednesday early morning so
> > your Tuesday morning and London's afternoon would be the best.
> >
> > Recording the thing would be definitely helpful too.
> >
> > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> > >
> > > I'd be more than happy to hop on a call next week to give you both
> > > (and anyone else interested) a tour of our dev tools.  Maybe something
> > > early morning on my end, which should be your evening, could work?
> > >
> > > I can set up a Zoom conference to get everyone acquainted.  We can
> > > record and post it for any who can't make it.
> > >
> > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> > > London)?  If anyone's interested please reply with what dates work.
> > > I'll be sure to post the details back here with the zoom link in case
> > > anyone wants to join that didn't get a chance to reply, as well as a
> > > link to the recorded call.
> > >
> > > Jon
> > >
> > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> > >  wrote:
> > > >
> > > > +1
> > > >
> > > > I’m also just as excited to see some standardised workloads and test
> > bed.  At the moment we’re benefiting from some large contributors doing
> > their own proprietary performance testing, which is super valuable and
> > something we’ve lacked before.  But I’m also keen to see some more
> > representative workloads that are reproducible by anybody in the community
> > take shape.
> > > >
> > > >
> > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> >  wrote:
> > > > >
> > > > > Hey Jon,
> > > > >
> > > > > This sounds exciting and pretty useful, thanks.
> > > > >
> > > > > Looking forward to using tlp-stress for validating 15066 performance.
> > > > >
> > > > > We should touch base some time next week to pick a comprehensive set
> > of workloads and versions, perhaps?
> > > > >
> > > > >
> > > > >> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> > > > >>
> > > > >> I don't want to derail the discussion about Stabilizing Internode
> > > > >> Messaging, so I'm starting this as a separate thread.  There was a
> > > > >> comment that Josh made [1] about doing performance testing with real
> > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in
> > support
> > > > >> of this.  We've been working on some tooling at TLP for the last
> > > > >> several months to make this a lot easier.  One of the goals has been
> > > > >> to help improve the 4.0 testing process.
> > > > >>
> > > > >> The first tool we have is tlp-stress [2].  It's designed with a "get
> > > > >> started in 5 minutes" mindset.  My goal was to ship a stress tool
> > that
> > > > >> ships with real workloads out of the box that can be easily tweaked,
> > > > >> similar to how fio allows you to design a disk workload and tweak it
> > > > >> with paramaters.  Included are stress workloads that stress LWTs
> > (two
> > > > >> different types), materialized views, counters, time series, and
> > > > >> key-value workloads.  Each workload can be modified easily to change
> > > > >> compaction strategies, concurrent operations, number of partitions.
> > > > >> We can run workloads for a set number of iterations or a custom
> > > > >> duration.  We've used this *extensively* at TLP to help our
> > customers
> > > > >> and most of our blog posts that discuss performance use it as well.
> > > > >> It exports data to both a CSV format and auto sets up prometheus for
> > > > >> metrics collection / aggregation.  As an example, we were able to
> > > > >> determine that the compression length set on the paxos tables
> > imposes
> > > > >> a significant overhead when using the Locking LWT workload, which
> > > > >> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> > > > >> details.
> > > > >>
> > > > >> We have documentation [3] on the TLP website.
> > > > >>
> > > > >> The second tool we've been working on is tlp-cluster [4].  This tool
> > > > >> is designed to help provision AWS instances for the purposes of
> > > > >> testing.  To be clear, I don't expect, or want, this tool to be used
> > > > >> for production environments.  It's designed to assist with the
> > > > >> Cassandra build 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-16 Thread Benedict Elliott Smith
Just to confirm, this is on Wednesday?

> On 15 Apr 2019, at 22:38, Jon Haddad  wrote:
> 
> Hey all,
> 
> I've set up a Zoom call for 9AM Pacific time.  Everyone's welcome to join.
> 
> https://zoom.us/j/189920888
> 
> Looking forward to a good discussion on how we can all pitch in on
> getting 4.0 out the door.
> 
> Jon
> 
> On Sat, Apr 13, 2019 at 9:14 AM Jonathan Koppenhofer
>  wrote:
>> 
>> Wednesday would work for me.
>> 
>> We use and (slightly) contribute to tlp tools. We are platform testing and
>> beginning 4.0 testing ourselves, so an in person overview would be great!
>> 
>> On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko 
>> wrote:
>> 
>>> Wednesday and Thursday, either, at 9 AM pacific WFM.
>>> 
 On 13 Apr 2019, at 13:31, Stefan Miklosovic <
>>> stefan.mikloso...@instaclustr.com> wrote:
 
 Hi Jon,
 
 I would like be on that call too but I am off on Thursday.
 
 I am from Australia so 5pm London time is ours 2am next day so your
 Wednesday morning is my Thursday night. Wednesday early morning so
 your Tuesday morning and London's afternoon would be the best.
 
 Recording the thing would be definitely helpful too.
 
 On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> 
> I'd be more than happy to hop on a call next week to give you both
> (and anyone else interested) a tour of our dev tools.  Maybe something
> early morning on my end, which should be your evening, could work?
> 
> I can set up a Zoom conference to get everyone acquainted.  We can
> record and post it for any who can't make it.
> 
> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> London)?  If anyone's interested please reply with what dates work.
> I'll be sure to post the details back here with the zoom link in case
> anyone wants to join that didn't get a chance to reply, as well as a
> link to the recorded call.
> 
> Jon
> 
> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
>  wrote:
>> 
>> +1
>> 
>> I’m also just as excited to see some standardised workloads and test
>>> bed.  At the moment we’re benefiting from some large contributors doing
>>> their own proprietary performance testing, which is super valuable and
>>> something we’ve lacked before.  But I’m also keen to see some more
>>> representative workloads that are reproducible by anybody in the community
>>> take shape.
>> 
>> 
>>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
>>>  wrote:
>>> 
>>> Hey Jon,
>>> 
>>> This sounds exciting and pretty useful, thanks.
>>> 
>>> Looking forward to using tlp-stress for validating 15066 performance.
>>> 
>>> We should touch base some time next week to pick a comprehensive set
>>> of workloads and versions, perhaps?
>>> 
>>> 
 On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
 
 I don't want to derail the discussion about Stabilizing Internode
 Messaging, so I'm starting this as a separate thread.  There was a
 comment that Josh made [1] about doing performance testing with real
 clusters as well as a lot of microbenchmarks, and I'm 100% in support
 of this.  We've been working on some tooling at TLP for the last
 several months to make this a lot easier.  One of the goals has been
 to help improve the 4.0 testing process.
 
 The first tool we have is tlp-stress [2].  It's designed with a "get
 started in 5 minutes" mindset.  My goal was to ship a stress tool
>>> that
 ships with real workloads out of the box that can be easily tweaked,
 similar to how fio allows you to design a disk workload and tweak it
 with paramaters.  Included are stress workloads that stress LWTs (two
 different types), materialized views, counters, time series, and
 key-value workloads.  Each workload can be modified easily to change
 compaction strategies, concurrent operations, number of partitions.
 We can run workloads for a set number of iterations or a custom
 duration.  We've used this *extensively* at TLP to help our customers
 and most of our blog posts that discuss performance use it as well.
 It exports data to both a CSV format and auto sets up prometheus for
 metrics collection / aggregation.  As an example, we were able to
 determine that the compression length set on the paxos tables imposes
 a significant overhead when using the Locking LWT workload, which
 simulates locking and unlocking of rows.  See CASSANDRA-15080 for
 details.
 
 We have documentation [3] on the TLP website.
 
 The second tool we've been working on is tlp-cluster [4].  This tool
 is designed to help provision AWS instances for the purposes of
 testing.  To be clear, I don't expect, or want, 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-15 Thread Anthony Grasso
Hi Stefan,

I have been working with Jon on developing the tool set. I can do a Zoom
call tomorrow (Wednesday) at 11am AEST if that works for you? We can go
through all the same information that Jon is going to go through in his
call. Note that I am in the same timezone as you, so if tomorrow morning is
no good we can always do the afternoon.

Cheers,
Anthony


On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> wrote:

> Hi Jon,
>
> I would like be on that call too but I am off on Thursday.
>
> I am from Australia so 5pm London time is ours 2am next day so your
> Wednesday morning is my Thursday night. Wednesday early morning so
> your Tuesday morning and London's afternoon would be the best.
>
> Recording the thing would be definitely helpful too.
>
> On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> >
> > I'd be more than happy to hop on a call next week to give you both
> > (and anyone else interested) a tour of our dev tools.  Maybe something
> > early morning on my end, which should be your evening, could work?
> >
> > I can set up a Zoom conference to get everyone acquainted.  We can
> > record and post it for any who can't make it.
> >
> > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> > London)?  If anyone's interested please reply with what dates work.
> > I'll be sure to post the details back here with the zoom link in case
> > anyone wants to join that didn't get a chance to reply, as well as a
> > link to the recorded call.
> >
> > Jon
> >
> > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> >  wrote:
> > >
> > > +1
> > >
> > > I’m also just as excited to see some standardised workloads and test
> bed.  At the moment we’re benefiting from some large contributors doing
> their own proprietary performance testing, which is super valuable and
> something we’ve lacked before.  But I’m also keen to see some more
> representative workloads that are reproducible by anybody in the community
> take shape.
> > >
> > >
> > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
>  wrote:
> > > >
> > > > Hey Jon,
> > > >
> > > > This sounds exciting and pretty useful, thanks.
> > > >
> > > > Looking forward to using tlp-stress for validating 15066 performance.
> > > >
> > > > We should touch base some time next week to pick a comprehensive set
> of workloads and versions, perhaps?
> > > >
> > > >
> > > >> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> > > >>
> > > >> I don't want to derail the discussion about Stabilizing Internode
> > > >> Messaging, so I'm starting this as a separate thread.  There was a
> > > >> comment that Josh made [1] about doing performance testing with real
> > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in
> support
> > > >> of this.  We've been working on some tooling at TLP for the last
> > > >> several months to make this a lot easier.  One of the goals has been
> > > >> to help improve the 4.0 testing process.
> > > >>
> > > >> The first tool we have is tlp-stress [2].  It's designed with a "get
> > > >> started in 5 minutes" mindset.  My goal was to ship a stress tool
> that
> > > >> ships with real workloads out of the box that can be easily tweaked,
> > > >> similar to how fio allows you to design a disk workload and tweak it
> > > >> with paramaters.  Included are stress workloads that stress LWTs
> (two
> > > >> different types), materialized views, counters, time series, and
> > > >> key-value workloads.  Each workload can be modified easily to change
> > > >> compaction strategies, concurrent operations, number of partitions.
> > > >> We can run workloads for a set number of iterations or a custom
> > > >> duration.  We've used this *extensively* at TLP to help our
> customers
> > > >> and most of our blog posts that discuss performance use it as well.
> > > >> It exports data to both a CSV format and auto sets up prometheus for
> > > >> metrics collection / aggregation.  As an example, we were able to
> > > >> determine that the compression length set on the paxos tables
> imposes
> > > >> a significant overhead when using the Locking LWT workload, which
> > > >> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> > > >> details.
> > > >>
> > > >> We have documentation [3] on the TLP website.
> > > >>
> > > >> The second tool we've been working on is tlp-cluster [4].  This tool
> > > >> is designed to help provision AWS instances for the purposes of
> > > >> testing.  To be clear, I don't expect, or want, this tool to be used
> > > >> for production environments.  It's designed to assist with the
> > > >> Cassandra build process by generating deb packages or re-using the
> > > >> ones that have already been uploaded.  Here's a short list of the
> > > >> things you'll care about:
> > > >>
> > > >> 1. Create instances in AWS for Cassandra using any instance size and
> > > >> number of nodes.  Also create tlp-stress instances and a box for
> > > >> monitoring
> > > >> 2. 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-15 Thread Jon Haddad
Hey all,

I've set up a Zoom call for 9AM Pacific time.  Everyone's welcome to join.

https://zoom.us/j/189920888

Looking forward to a good discussion on how we can all pitch in on
getting 4.0 out the door.

Jon

On Sat, Apr 13, 2019 at 9:14 AM Jonathan Koppenhofer
 wrote:
>
> Wednesday would work for me.
>
> We use and (slightly) contribute to tlp tools. We are platform testing and
> beginning 4.0 testing ourselves, so an in person overview would be great!
>
> On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko 
> wrote:
>
> > Wednesday and Thursday, either, at 9 AM pacific WFM.
> >
> > > On 13 Apr 2019, at 13:31, Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> > >
> > > Hi Jon,
> > >
> > > I would like be on that call too but I am off on Thursday.
> > >
> > > I am from Australia so 5pm London time is ours 2am next day so your
> > > Wednesday morning is my Thursday night. Wednesday early morning so
> > > your Tuesday morning and London's afternoon would be the best.
> > >
> > > Recording the thing would be definitely helpful too.
> > >
> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> > >>
> > >> I'd be more than happy to hop on a call next week to give you both
> > >> (and anyone else interested) a tour of our dev tools.  Maybe something
> > >> early morning on my end, which should be your evening, could work?
> > >>
> > >> I can set up a Zoom conference to get everyone acquainted.  We can
> > >> record and post it for any who can't make it.
> > >>
> > >> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> > >> London)?  If anyone's interested please reply with what dates work.
> > >> I'll be sure to post the details back here with the zoom link in case
> > >> anyone wants to join that didn't get a chance to reply, as well as a
> > >> link to the recorded call.
> > >>
> > >> Jon
> > >>
> > >> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> > >>  wrote:
> > >>>
> > >>> +1
> > >>>
> > >>> I’m also just as excited to see some standardised workloads and test
> > bed.  At the moment we’re benefiting from some large contributors doing
> > their own proprietary performance testing, which is super valuable and
> > something we’ve lacked before.  But I’m also keen to see some more
> > representative workloads that are reproducible by anybody in the community
> > take shape.
> > >>>
> > >>>
> >  On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
> >  wrote:
> > 
> >  Hey Jon,
> > 
> >  This sounds exciting and pretty useful, thanks.
> > 
> >  Looking forward to using tlp-stress for validating 15066 performance.
> > 
> >  We should touch base some time next week to pick a comprehensive set
> > of workloads and versions, perhaps?
> > 
> > 
> > > On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> > >
> > > I don't want to derail the discussion about Stabilizing Internode
> > > Messaging, so I'm starting this as a separate thread.  There was a
> > > comment that Josh made [1] about doing performance testing with real
> > > clusters as well as a lot of microbenchmarks, and I'm 100% in support
> > > of this.  We've been working on some tooling at TLP for the last
> > > several months to make this a lot easier.  One of the goals has been
> > > to help improve the 4.0 testing process.
> > >
> > > The first tool we have is tlp-stress [2].  It's designed with a "get
> > > started in 5 minutes" mindset.  My goal was to ship a stress tool
> > that
> > > ships with real workloads out of the box that can be easily tweaked,
> > > similar to how fio allows you to design a disk workload and tweak it
> > > with paramaters.  Included are stress workloads that stress LWTs (two
> > > different types), materialized views, counters, time series, and
> > > key-value workloads.  Each workload can be modified easily to change
> > > compaction strategies, concurrent operations, number of partitions.
> > > We can run workloads for a set number of iterations or a custom
> > > duration.  We've used this *extensively* at TLP to help our customers
> > > and most of our blog posts that discuss performance use it as well.
> > > It exports data to both a CSV format and auto sets up prometheus for
> > > metrics collection / aggregation.  As an example, we were able to
> > > determine that the compression length set on the paxos tables imposes
> > > a significant overhead when using the Locking LWT workload, which
> > > simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> > > details.
> > >
> > > We have documentation [3] on the TLP website.
> > >
> > > The second tool we've been working on is tlp-cluster [4].  This tool
> > > is designed to help provision AWS instances for the purposes of
> > > testing.  To be clear, I don't expect, or want, this tool to be used
> > > for production environments.  It's designed 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-13 Thread Jonathan Koppenhofer
Wednesday would work for me.

We use and (slightly) contribute to tlp tools. We are platform testing and
beginning 4.0 testing ourselves, so an in person overview would be great!

On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko 
wrote:

> Wednesday and Thursday, either, at 9 AM pacific WFM.
>
> > On 13 Apr 2019, at 13:31, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
> >
> > Hi Jon,
> >
> > I would like be on that call too but I am off on Thursday.
> >
> > I am from Australia so 5pm London time is ours 2am next day so your
> > Wednesday morning is my Thursday night. Wednesday early morning so
> > your Tuesday morning and London's afternoon would be the best.
> >
> > Recording the thing would be definitely helpful too.
> >
> > On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
> >>
> >> I'd be more than happy to hop on a call next week to give you both
> >> (and anyone else interested) a tour of our dev tools.  Maybe something
> >> early morning on my end, which should be your evening, could work?
> >>
> >> I can set up a Zoom conference to get everyone acquainted.  We can
> >> record and post it for any who can't make it.
> >>
> >> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> >> London)?  If anyone's interested please reply with what dates work.
> >> I'll be sure to post the details back here with the zoom link in case
> >> anyone wants to join that didn't get a chance to reply, as well as a
> >> link to the recorded call.
> >>
> >> Jon
> >>
> >> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
> >>  wrote:
> >>>
> >>> +1
> >>>
> >>> I’m also just as excited to see some standardised workloads and test
> bed.  At the moment we’re benefiting from some large contributors doing
> their own proprietary performance testing, which is super valuable and
> something we’ve lacked before.  But I’m also keen to see some more
> representative workloads that are reproducible by anybody in the community
> take shape.
> >>>
> >>>
>  On 12 Apr 2019, at 18:09, Aleksey Yeshchenko
>  wrote:
> 
>  Hey Jon,
> 
>  This sounds exciting and pretty useful, thanks.
> 
>  Looking forward to using tlp-stress for validating 15066 performance.
> 
>  We should touch base some time next week to pick a comprehensive set
> of workloads and versions, perhaps?
> 
> 
> > On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> >
> > I don't want to derail the discussion about Stabilizing Internode
> > Messaging, so I'm starting this as a separate thread.  There was a
> > comment that Josh made [1] about doing performance testing with real
> > clusters as well as a lot of microbenchmarks, and I'm 100% in support
> > of this.  We've been working on some tooling at TLP for the last
> > several months to make this a lot easier.  One of the goals has been
> > to help improve the 4.0 testing process.
> >
> > The first tool we have is tlp-stress [2].  It's designed with a "get
> > started in 5 minutes" mindset.  My goal was to ship a stress tool
> that
> > ships with real workloads out of the box that can be easily tweaked,
> > similar to how fio allows you to design a disk workload and tweak it
> > with paramaters.  Included are stress workloads that stress LWTs (two
> > different types), materialized views, counters, time series, and
> > key-value workloads.  Each workload can be modified easily to change
> > compaction strategies, concurrent operations, number of partitions.
> > We can run workloads for a set number of iterations or a custom
> > duration.  We've used this *extensively* at TLP to help our customers
> > and most of our blog posts that discuss performance use it as well.
> > It exports data to both a CSV format and auto sets up prometheus for
> > metrics collection / aggregation.  As an example, we were able to
> > determine that the compression length set on the paxos tables imposes
> > a significant overhead when using the Locking LWT workload, which
> > simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> > details.
> >
> > We have documentation [3] on the TLP website.
> >
> > The second tool we've been working on is tlp-cluster [4].  This tool
> > is designed to help provision AWS instances for the purposes of
> > testing.  To be clear, I don't expect, or want, this tool to be used
> > for production environments.  It's designed to assist with the
> > Cassandra build process by generating deb packages or re-using the
> > ones that have already been uploaded.  Here's a short list of the
> > things you'll care about:
> >
> > 1. Create instances in AWS for Cassandra using any instance size and
> > number of nodes.  Also create tlp-stress instances and a box for
> > monitoring
> > 2. Use any available build of Cassandra, with a quick option to
> change
> > YAML 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-13 Thread Aleksey Yeshchenko
Wednesday and Thursday, either, at 9 AM pacific WFM.

> On 13 Apr 2019, at 13:31, Stefan Miklosovic 
>  wrote:
> 
> Hi Jon,
> 
> I would like be on that call too but I am off on Thursday.
> 
> I am from Australia so 5pm London time is ours 2am next day so your
> Wednesday morning is my Thursday night. Wednesday early morning so
> your Tuesday morning and London's afternoon would be the best.
> 
> Recording the thing would be definitely helpful too.
> 
> On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
>> 
>> I'd be more than happy to hop on a call next week to give you both
>> (and anyone else interested) a tour of our dev tools.  Maybe something
>> early morning on my end, which should be your evening, could work?
>> 
>> I can set up a Zoom conference to get everyone acquainted.  We can
>> record and post it for any who can't make it.
>> 
>> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
>> London)?  If anyone's interested please reply with what dates work.
>> I'll be sure to post the details back here with the zoom link in case
>> anyone wants to join that didn't get a chance to reply, as well as a
>> link to the recorded call.
>> 
>> Jon
>> 
>> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
>>  wrote:
>>> 
>>> +1
>>> 
>>> I’m also just as excited to see some standardised workloads and test bed.  
>>> At the moment we’re benefiting from some large contributors doing their own 
>>> proprietary performance testing, which is super valuable and something 
>>> we’ve lacked before.  But I’m also keen to see some more representative 
>>> workloads that are reproducible by anybody in the community take shape.
>>> 
>>> 
 On 12 Apr 2019, at 18:09, Aleksey Yeshchenko  
 wrote:
 
 Hey Jon,
 
 This sounds exciting and pretty useful, thanks.
 
 Looking forward to using tlp-stress for validating 15066 performance.
 
 We should touch base some time next week to pick a comprehensive set of 
 workloads and versions, perhaps?
 
 
> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> 
> I don't want to derail the discussion about Stabilizing Internode
> Messaging, so I'm starting this as a separate thread.  There was a
> comment that Josh made [1] about doing performance testing with real
> clusters as well as a lot of microbenchmarks, and I'm 100% in support
> of this.  We've been working on some tooling at TLP for the last
> several months to make this a lot easier.  One of the goals has been
> to help improve the 4.0 testing process.
> 
> The first tool we have is tlp-stress [2].  It's designed with a "get
> started in 5 minutes" mindset.  My goal was to ship a stress tool that
> ships with real workloads out of the box that can be easily tweaked,
> similar to how fio allows you to design a disk workload and tweak it
> with paramaters.  Included are stress workloads that stress LWTs (two
> different types), materialized views, counters, time series, and
> key-value workloads.  Each workload can be modified easily to change
> compaction strategies, concurrent operations, number of partitions.
> We can run workloads for a set number of iterations or a custom
> duration.  We've used this *extensively* at TLP to help our customers
> and most of our blog posts that discuss performance use it as well.
> It exports data to both a CSV format and auto sets up prometheus for
> metrics collection / aggregation.  As an example, we were able to
> determine that the compression length set on the paxos tables imposes
> a significant overhead when using the Locking LWT workload, which
> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> details.
> 
> We have documentation [3] on the TLP website.
> 
> The second tool we've been working on is tlp-cluster [4].  This tool
> is designed to help provision AWS instances for the purposes of
> testing.  To be clear, I don't expect, or want, this tool to be used
> for production environments.  It's designed to assist with the
> Cassandra build process by generating deb packages or re-using the
> ones that have already been uploaded.  Here's a short list of the
> things you'll care about:
> 
> 1. Create instances in AWS for Cassandra using any instance size and
> number of nodes.  Also create tlp-stress instances and a box for
> monitoring
> 2. Use any available build of Cassandra, with a quick option to change
> YAML config.  For example: tlp-stress use 3.11.4 -c
> concurrent_writes:256
> 3. Do custom builds just by pointing to a local Cassandra git repo.
> They can be used the same way as #2.
> 4. tlp-stress is automatically installed on the stress box.
> 5. Everything's installed with pure bash.  I considered something more
> complex, but since this is for development only, it turns out the
> simplest 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-13 Thread Stefan Miklosovic
Hi Jon,

I would like be on that call too but I am off on Thursday.

I am from Australia so 5pm London time is ours 2am next day so your
Wednesday morning is my Thursday night. Wednesday early morning so
your Tuesday morning and London's afternoon would be the best.

Recording the thing would be definitely helpful too.

On Sat, 13 Apr 2019 at 07:45, Jon Haddad  wrote:
>
> I'd be more than happy to hop on a call next week to give you both
> (and anyone else interested) a tour of our dev tools.  Maybe something
> early morning on my end, which should be your evening, could work?
>
> I can set up a Zoom conference to get everyone acquainted.  We can
> record and post it for any who can't make it.
>
> I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
> London)?  If anyone's interested please reply with what dates work.
> I'll be sure to post the details back here with the zoom link in case
> anyone wants to join that didn't get a chance to reply, as well as a
> link to the recorded call.
>
> Jon
>
> On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
>  wrote:
> >
> > +1
> >
> > I’m also just as excited to see some standardised workloads and test bed.  
> > At the moment we’re benefiting from some large contributors doing their own 
> > proprietary performance testing, which is super valuable and something 
> > we’ve lacked before.  But I’m also keen to see some more representative 
> > workloads that are reproducible by anybody in the community take shape.
> >
> >
> > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko  
> > > wrote:
> > >
> > > Hey Jon,
> > >
> > > This sounds exciting and pretty useful, thanks.
> > >
> > > Looking forward to using tlp-stress for validating 15066 performance.
> > >
> > > We should touch base some time next week to pick a comprehensive set of 
> > > workloads and versions, perhaps?
> > >
> > >
> > >> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> > >>
> > >> I don't want to derail the discussion about Stabilizing Internode
> > >> Messaging, so I'm starting this as a separate thread.  There was a
> > >> comment that Josh made [1] about doing performance testing with real
> > >> clusters as well as a lot of microbenchmarks, and I'm 100% in support
> > >> of this.  We've been working on some tooling at TLP for the last
> > >> several months to make this a lot easier.  One of the goals has been
> > >> to help improve the 4.0 testing process.
> > >>
> > >> The first tool we have is tlp-stress [2].  It's designed with a "get
> > >> started in 5 minutes" mindset.  My goal was to ship a stress tool that
> > >> ships with real workloads out of the box that can be easily tweaked,
> > >> similar to how fio allows you to design a disk workload and tweak it
> > >> with paramaters.  Included are stress workloads that stress LWTs (two
> > >> different types), materialized views, counters, time series, and
> > >> key-value workloads.  Each workload can be modified easily to change
> > >> compaction strategies, concurrent operations, number of partitions.
> > >> We can run workloads for a set number of iterations or a custom
> > >> duration.  We've used this *extensively* at TLP to help our customers
> > >> and most of our blog posts that discuss performance use it as well.
> > >> It exports data to both a CSV format and auto sets up prometheus for
> > >> metrics collection / aggregation.  As an example, we were able to
> > >> determine that the compression length set on the paxos tables imposes
> > >> a significant overhead when using the Locking LWT workload, which
> > >> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> > >> details.
> > >>
> > >> We have documentation [3] on the TLP website.
> > >>
> > >> The second tool we've been working on is tlp-cluster [4].  This tool
> > >> is designed to help provision AWS instances for the purposes of
> > >> testing.  To be clear, I don't expect, or want, this tool to be used
> > >> for production environments.  It's designed to assist with the
> > >> Cassandra build process by generating deb packages or re-using the
> > >> ones that have already been uploaded.  Here's a short list of the
> > >> things you'll care about:
> > >>
> > >> 1. Create instances in AWS for Cassandra using any instance size and
> > >> number of nodes.  Also create tlp-stress instances and a box for
> > >> monitoring
> > >> 2. Use any available build of Cassandra, with a quick option to change
> > >> YAML config.  For example: tlp-stress use 3.11.4 -c
> > >> concurrent_writes:256
> > >> 3. Do custom builds just by pointing to a local Cassandra git repo.
> > >> They can be used the same way as #2.
> > >> 4. tlp-stress is automatically installed on the stress box.
> > >> 5. Everything's installed with pure bash.  I considered something more
> > >> complex, but since this is for development only, it turns out the
> > >> simplest tool possible works well and it means it's easily
> > >> configurable.  Just drop in your own bash script 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-12 Thread Jon Haddad
I'd be more than happy to hop on a call next week to give you both
(and anyone else interested) a tour of our dev tools.  Maybe something
early morning on my end, which should be your evening, could work?

I can set up a Zoom conference to get everyone acquainted.  We can
record and post it for any who can't make it.

I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm
London)?  If anyone's interested please reply with what dates work.
I'll be sure to post the details back here with the zoom link in case
anyone wants to join that didn't get a chance to reply, as well as a
link to the recorded call.

Jon

On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith
 wrote:
>
> +1
>
> I’m also just as excited to see some standardised workloads and test bed.  At 
> the moment we’re benefiting from some large contributors doing their own 
> proprietary performance testing, which is super valuable and something we’ve 
> lacked before.  But I’m also keen to see some more representative workloads 
> that are reproducible by anybody in the community take shape.
>
>
> > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko  
> > wrote:
> >
> > Hey Jon,
> >
> > This sounds exciting and pretty useful, thanks.
> >
> > Looking forward to using tlp-stress for validating 15066 performance.
> >
> > We should touch base some time next week to pick a comprehensive set of 
> > workloads and versions, perhaps?
> >
> >
> >> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> >>
> >> I don't want to derail the discussion about Stabilizing Internode
> >> Messaging, so I'm starting this as a separate thread.  There was a
> >> comment that Josh made [1] about doing performance testing with real
> >> clusters as well as a lot of microbenchmarks, and I'm 100% in support
> >> of this.  We've been working on some tooling at TLP for the last
> >> several months to make this a lot easier.  One of the goals has been
> >> to help improve the 4.0 testing process.
> >>
> >> The first tool we have is tlp-stress [2].  It's designed with a "get
> >> started in 5 minutes" mindset.  My goal was to ship a stress tool that
> >> ships with real workloads out of the box that can be easily tweaked,
> >> similar to how fio allows you to design a disk workload and tweak it
> >> with paramaters.  Included are stress workloads that stress LWTs (two
> >> different types), materialized views, counters, time series, and
> >> key-value workloads.  Each workload can be modified easily to change
> >> compaction strategies, concurrent operations, number of partitions.
> >> We can run workloads for a set number of iterations or a custom
> >> duration.  We've used this *extensively* at TLP to help our customers
> >> and most of our blog posts that discuss performance use it as well.
> >> It exports data to both a CSV format and auto sets up prometheus for
> >> metrics collection / aggregation.  As an example, we were able to
> >> determine that the compression length set on the paxos tables imposes
> >> a significant overhead when using the Locking LWT workload, which
> >> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> >> details.
> >>
> >> We have documentation [3] on the TLP website.
> >>
> >> The second tool we've been working on is tlp-cluster [4].  This tool
> >> is designed to help provision AWS instances for the purposes of
> >> testing.  To be clear, I don't expect, or want, this tool to be used
> >> for production environments.  It's designed to assist with the
> >> Cassandra build process by generating deb packages or re-using the
> >> ones that have already been uploaded.  Here's a short list of the
> >> things you'll care about:
> >>
> >> 1. Create instances in AWS for Cassandra using any instance size and
> >> number of nodes.  Also create tlp-stress instances and a box for
> >> monitoring
> >> 2. Use any available build of Cassandra, with a quick option to change
> >> YAML config.  For example: tlp-stress use 3.11.4 -c
> >> concurrent_writes:256
> >> 3. Do custom builds just by pointing to a local Cassandra git repo.
> >> They can be used the same way as #2.
> >> 4. tlp-stress is automatically installed on the stress box.
> >> 5. Everything's installed with pure bash.  I considered something more
> >> complex, but since this is for development only, it turns out the
> >> simplest tool possible works well and it means it's easily
> >> configurable.  Just drop in your own bash script starting with a
> >> number in a XX_script_name.sh format and it gets run.
> >> 6. The monitoring box is running Prometheus.  It auto scrapes
> >> Cassandra using the Instaclustr metrics library.
> >> 7. Grafana is also installed automatically.  There's a couple sample
> >> graphs there now.  We plan on having better default graphs soon.
> >>
> >> For the moment it installs java 8 only but that should be easily
> >> fixable to use java 11 to test ZGC (it's on my radar).
> >>
> >> Documentation for tlp-cluster is here [5].
> >>
> >> There's still some 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-12 Thread Benedict Elliott Smith
+1

I’m also just as excited to see some standardised workloads and test bed.  At 
the moment we’re benefiting from some large contributors doing their own 
proprietary performance testing, which is super valuable and something we’ve 
lacked before.  But I’m also keen to see some more representative workloads 
that are reproducible by anybody in the community take shape.


> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko  
> wrote:
> 
> Hey Jon,
> 
> This sounds exciting and pretty useful, thanks.
> 
> Looking forward to using tlp-stress for validating 15066 performance.
> 
> We should touch base some time next week to pick a comprehensive set of 
> workloads and versions, perhaps?
> 
> 
>> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
>> 
>> I don't want to derail the discussion about Stabilizing Internode
>> Messaging, so I'm starting this as a separate thread.  There was a
>> comment that Josh made [1] about doing performance testing with real
>> clusters as well as a lot of microbenchmarks, and I'm 100% in support
>> of this.  We've been working on some tooling at TLP for the last
>> several months to make this a lot easier.  One of the goals has been
>> to help improve the 4.0 testing process.
>> 
>> The first tool we have is tlp-stress [2].  It's designed with a "get
>> started in 5 minutes" mindset.  My goal was to ship a stress tool that
>> ships with real workloads out of the box that can be easily tweaked,
>> similar to how fio allows you to design a disk workload and tweak it
>> with paramaters.  Included are stress workloads that stress LWTs (two
>> different types), materialized views, counters, time series, and
>> key-value workloads.  Each workload can be modified easily to change
>> compaction strategies, concurrent operations, number of partitions.
>> We can run workloads for a set number of iterations or a custom
>> duration.  We've used this *extensively* at TLP to help our customers
>> and most of our blog posts that discuss performance use it as well.
>> It exports data to both a CSV format and auto sets up prometheus for
>> metrics collection / aggregation.  As an example, we were able to
>> determine that the compression length set on the paxos tables imposes
>> a significant overhead when using the Locking LWT workload, which
>> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
>> details.
>> 
>> We have documentation [3] on the TLP website.
>> 
>> The second tool we've been working on is tlp-cluster [4].  This tool
>> is designed to help provision AWS instances for the purposes of
>> testing.  To be clear, I don't expect, or want, this tool to be used
>> for production environments.  It's designed to assist with the
>> Cassandra build process by generating deb packages or re-using the
>> ones that have already been uploaded.  Here's a short list of the
>> things you'll care about:
>> 
>> 1. Create instances in AWS for Cassandra using any instance size and
>> number of nodes.  Also create tlp-stress instances and a box for
>> monitoring
>> 2. Use any available build of Cassandra, with a quick option to change
>> YAML config.  For example: tlp-stress use 3.11.4 -c
>> concurrent_writes:256
>> 3. Do custom builds just by pointing to a local Cassandra git repo.
>> They can be used the same way as #2.
>> 4. tlp-stress is automatically installed on the stress box.
>> 5. Everything's installed with pure bash.  I considered something more
>> complex, but since this is for development only, it turns out the
>> simplest tool possible works well and it means it's easily
>> configurable.  Just drop in your own bash script starting with a
>> number in a XX_script_name.sh format and it gets run.
>> 6. The monitoring box is running Prometheus.  It auto scrapes
>> Cassandra using the Instaclustr metrics library.
>> 7. Grafana is also installed automatically.  There's a couple sample
>> graphs there now.  We plan on having better default graphs soon.
>> 
>> For the moment it installs java 8 only but that should be easily
>> fixable to use java 11 to test ZGC (it's on my radar).
>> 
>> Documentation for tlp-cluster is here [5].
>> 
>> There's still some things to work out in the tool, and we've been
>> working hard to smooth out the rough edges.  I still haven't announced
>> anything WRT tlp-cluster on the TLP blog, because I don't think it's
>> quite ready for public consumption, but I think the folks on this list
>> are smart enough to see the value in it even if it has a few warts
>> still.
>> 
>> I don't consider myself familiar enough with the networking patch to
>> give it a full review, but I am qualified to build tools to help test
>> it and go through the testing process myself.  From what I can tell
>> the patch is moving the codebase in a positive direction and I'd like
>> to help build confidence in it so we can get it merged in.
>> 
>> We'll continue to build out and improve the tooling with the goal of
>> making it easier for people to jump into the QA side of things.
>> 

Re: TLP tools for stress testing and building test clusters in AWS

2019-04-12 Thread Aleksey Yeshchenko
Hey Jon,

This sounds exciting and pretty useful, thanks.

Looking forward to using tlp-stress for validating 15066 performance.

We should touch base some time next week to pick a comprehensive set of 
workloads and versions, perhaps?


> On 12 Apr 2019, at 16:34, Jon Haddad  wrote:
> 
> I don't want to derail the discussion about Stabilizing Internode
> Messaging, so I'm starting this as a separate thread.  There was a
> comment that Josh made [1] about doing performance testing with real
> clusters as well as a lot of microbenchmarks, and I'm 100% in support
> of this.  We've been working on some tooling at TLP for the last
> several months to make this a lot easier.  One of the goals has been
> to help improve the 4.0 testing process.
> 
> The first tool we have is tlp-stress [2].  It's designed with a "get
> started in 5 minutes" mindset.  My goal was to ship a stress tool that
> ships with real workloads out of the box that can be easily tweaked,
> similar to how fio allows you to design a disk workload and tweak it
> with paramaters.  Included are stress workloads that stress LWTs (two
> different types), materialized views, counters, time series, and
> key-value workloads.  Each workload can be modified easily to change
> compaction strategies, concurrent operations, number of partitions.
> We can run workloads for a set number of iterations or a custom
> duration.  We've used this *extensively* at TLP to help our customers
> and most of our blog posts that discuss performance use it as well.
> It exports data to both a CSV format and auto sets up prometheus for
> metrics collection / aggregation.  As an example, we were able to
> determine that the compression length set on the paxos tables imposes
> a significant overhead when using the Locking LWT workload, which
> simulates locking and unlocking of rows.  See CASSANDRA-15080 for
> details.
> 
> We have documentation [3] on the TLP website.
> 
> The second tool we've been working on is tlp-cluster [4].  This tool
> is designed to help provision AWS instances for the purposes of
> testing.  To be clear, I don't expect, or want, this tool to be used
> for production environments.  It's designed to assist with the
> Cassandra build process by generating deb packages or re-using the
> ones that have already been uploaded.  Here's a short list of the
> things you'll care about:
> 
> 1. Create instances in AWS for Cassandra using any instance size and
> number of nodes.  Also create tlp-stress instances and a box for
> monitoring
> 2. Use any available build of Cassandra, with a quick option to change
> YAML config.  For example: tlp-stress use 3.11.4 -c
> concurrent_writes:256
> 3. Do custom builds just by pointing to a local Cassandra git repo.
> They can be used the same way as #2.
> 4. tlp-stress is automatically installed on the stress box.
> 5. Everything's installed with pure bash.  I considered something more
> complex, but since this is for development only, it turns out the
> simplest tool possible works well and it means it's easily
> configurable.  Just drop in your own bash script starting with a
> number in a XX_script_name.sh format and it gets run.
> 6. The monitoring box is running Prometheus.  It auto scrapes
> Cassandra using the Instaclustr metrics library.
> 7. Grafana is also installed automatically.  There's a couple sample
> graphs there now.  We plan on having better default graphs soon.
> 
> For the moment it installs java 8 only but that should be easily
> fixable to use java 11 to test ZGC (it's on my radar).
> 
> Documentation for tlp-cluster is here [5].
> 
> There's still some things to work out in the tool, and we've been
> working hard to smooth out the rough edges.  I still haven't announced
> anything WRT tlp-cluster on the TLP blog, because I don't think it's
> quite ready for public consumption, but I think the folks on this list
> are smart enough to see the value in it even if it has a few warts
> still.
> 
> I don't consider myself familiar enough with the networking patch to
> give it a full review, but I am qualified to build tools to help test
> it and go through the testing process myself.  From what I can tell
> the patch is moving the codebase in a positive direction and I'd like
> to help build confidence in it so we can get it merged in.
> 
> We'll continue to build out and improve the tooling with the goal of
> making it easier for people to jump into the QA side of things.
> 
> Jon
> 
> [1] 
> https://lists.apache.org/thread.html/742009c8a77999f4b62062509f087b670275f827d0c1895bf839eece@%3Cdev.cassandra.apache.org%3E
> [2] https://github.com/thelastpickle/tlp-stress
> [3] http://thelastpickle.com/tlp-stress/
> [4] https://github.com/thelastpickle/tlp-cluster
> [5] http://thelastpickle.com/tlp-cluster/
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>