Rocksandra performance test result

[Jay Zhuang]

We're shadowing some production traffics to a Rocksandra cluster (
https://github.com/Instagram/cassandra/tree/rocks_3.0), the P99 latency is
significantly improved (about 6x for read, 12x for write). Here are the
test details:

https://docs.google.com/document/d/1cEM8ZqB5tOYVdsh1LpqSZ-eLasumWfzn_Tk_lDjxOTk/

Thanks,
Jay

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Patrick Bannister]

Kurt's proposal to freeze the 2.7 implementation and offer a Python 3
implementation alongside it appeals to my desire to free ourselves of
design choices made for the sake of Python 2/3 cross compatibility. It also
has the advantage of keeping Python 2.7 users on a time tested product.

On the downside, offering two separate implementations would come at the
cost of maintaining both, which would siphon community resources away from
other areas more beneficial to the project. On that basis, I advocate for a
cross compatible implementation over releasing two implementations.

Patrick Bannister


On Fri, Jun 1, 2018 at 6:49 PM, kurt greaves  wrote:

> Seems pretty straightforward to me. Create a python 3 version as soon as
> possible and make it available, keep the python 2.7 version as default
> until the next major release after 4.0 (assuming around/after python 2.7
> EOL), then switch default and leave continued support for 2.7 cqlsh up to
> the community and whoever needs it.
>
> On Sat., 2 Jun. 2018, 04:45 J. D. Jordan, 
> wrote:
>
> > All for using six and supporting both. Sorry, I read your initial email
> as
> > wanting to drop support for 2 at the end of the year.
> >
> > > On Jun 1, 2018, at 1:01 PM, Jonathan Haddad  wrote:
> > >
> > > And that's why I said supporting both with six is the right path
> > > forward, later dropping support for 2.  I'm not advocating we drop 2
> > > support now, and I'm not asking for any sort of commitment.  I didn't
> > > think adding support for 3 would be so controversial.
> > > On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
> > >  wrote:
> > >>
> > >> The community of people doing python development and the community of
> > people running Cassandra servers are not the same.  I am not fine riding
> > the coat tails of libraries used in python development.  As others have
> > stated we need to be following the lead of the OS vendors that people
> will
> > be deploying Cassandra on top of.  And those will not be dropping Python
> 2
> > at the end of the year.
> > >>
> > >> -Jeremiah
> > >>
> > >>> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad 
> > wrote:
> > >>>
> > >>> Both can work.  I did a lot of the work on the port of the Python
> > >>> driver's object mapper (formerly cqlengine) to Python 3.  It's
> > >>> reasonably straightforward if you use the six library.
> > >>>
> > >>> Both pandas and numpy are dropping support for Python 2 at the end of
> > >>> this year.  I'm fine with riding on their coattails.
> >  On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman <
> r...@windofkeltia.com>
> > wrote:
> > 
> >  Support for, but not the very script, right? Because, as gently
> > pointed
> >  out by several realists here, Python 2 is far from dead and arguably
> >  still the majority usage. That's only just now beginning to change.
> I
> >  think it will be more than 2 years before people begin asking what
> >  Python 2 was.
> > 
> > 
> > > On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > > Supporting both as a next step is logical, removing support for 2
> in
> > the
> > > next year or two seems reasonable enough. Gotta rip the band aid
> off
> > at
> > > some point.
> > >
> > >> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman <
> mibur...@redhat.com>
> > wrote:
> > >>
> > >> Hi,
> > >>
> > >> Deprecating in this context does not mean removing it or it being
> > >> replaced by 3 (RHEL 7.x will remain with Python 2.x as default).
> It
> > >> refers to future versions (>7), but there are none at this point.
> It
> > >> appears Ubuntu has deviated from Debian in this sense, but Debian
> > has
> > >> not changed yet (likely Debian 10 will, but that's not out yet and
> > has
> > >> no announced release date).
> > >>
> > >> Thus, 2.x still remains the most used version for servers. And
> > servers
> > >> deployed at this point of time will use these versions for years.
> > >>
> > >>   - Micke
> > >>
> > >>
> > >>> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> >  On 2018/06/01 07:40:04, Michael Burman 
> > wrote:
> >  IIRC, there's no major distribution yet that defaults to Python
> 3
> > (I
> >  think Ubuntu & Debian are still defaulting to Python 2 also).
> > This will
> >  happen eventually (maybe), but not yet. Discarding Python 2
> > support
> >  would mean more base-OS work for most people wanting to run
> > Cassandra
> >  and that's not a positive thing.
> > 
> > >>> Ubuntu since 16.04 defaults to Python 3:
> > >>>
> >  Python2 is not installed anymore by default on the server, cloud
> > and
> > >> the touch images, long live Python3! Python3 itself has been
> > upgraded to
> > >> the 3.5 series. -
> > >>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.
> ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=
> 

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[kurt greaves]

Seems pretty straightforward to me. Create a python 3 version as soon as
possible and make it available, keep the python 2.7 version as default
until the next major release after 4.0 (assuming around/after python 2.7
EOL), then switch default and leave continued support for 2.7 cqlsh up to
the community and whoever needs it.

On Sat., 2 Jun. 2018, 04:45 J. D. Jordan,  wrote:

> All for using six and supporting both. Sorry, I read your initial email as
> wanting to drop support for 2 at the end of the year.
>
> > On Jun 1, 2018, at 1:01 PM, Jonathan Haddad  wrote:
> >
> > And that's why I said supporting both with six is the right path
> > forward, later dropping support for 2.  I'm not advocating we drop 2
> > support now, and I'm not asking for any sort of commitment.  I didn't
> > think adding support for 3 would be so controversial.
> > On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
> >  wrote:
> >>
> >> The community of people doing python development and the community of
> people running Cassandra servers are not the same.  I am not fine riding
> the coat tails of libraries used in python development.  As others have
> stated we need to be following the lead of the OS vendors that people will
> be deploying Cassandra on top of.  And those will not be dropping Python 2
> at the end of the year.
> >>
> >> -Jeremiah
> >>
> >>> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad 
> wrote:
> >>>
> >>> Both can work.  I did a lot of the work on the port of the Python
> >>> driver's object mapper (formerly cqlengine) to Python 3.  It's
> >>> reasonably straightforward if you use the six library.
> >>>
> >>> Both pandas and numpy are dropping support for Python 2 at the end of
> >>> this year.  I'm fine with riding on their coattails.
>  On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman 
> wrote:
> 
>  Support for, but not the very script, right? Because, as gently
> pointed
>  out by several realists here, Python 2 is far from dead and arguably
>  still the majority usage. That's only just now beginning to change. I
>  think it will be more than 2 years before people begin asking what
>  Python 2 was.
> 
> 
> > On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > Supporting both as a next step is logical, removing support for 2 in
> the
> > next year or two seems reasonable enough. Gotta rip the band aid off
> at
> > some point.
> >
> >> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman 
> wrote:
> >>
> >> Hi,
> >>
> >> Deprecating in this context does not mean removing it or it being
> >> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> >> refers to future versions (>7), but there are none at this point. It
> >> appears Ubuntu has deviated from Debian in this sense, but Debian
> has
> >> not changed yet (likely Debian 10 will, but that's not out yet and
> has
> >> no announced release date).
> >>
> >> Thus, 2.x still remains the most used version for servers. And
> servers
> >> deployed at this point of time will use these versions for years.
> >>
> >>   - Micke
> >>
> >>
> >>> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
>  On 2018/06/01 07:40:04, Michael Burman 
> wrote:
>  IIRC, there's no major distribution yet that defaults to Python 3
> (I
>  think Ubuntu & Debian are still defaulting to Python 2 also).
> This will
>  happen eventually (maybe), but not yet. Discarding Python 2
> support
>  would mean more base-OS work for most people wanting to run
> Cassandra
>  and that's not a positive thing.
> 
> >>> Ubuntu since 16.04 defaults to Python 3:
> >>>
>  Python2 is not installed anymore by default on the server, cloud
> and
> >> the touch images, long live Python3! Python3 itself has been
> upgraded to
> >> the 3.5 series. -
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc=
> >>> RHEL 7.5 deprecates Python 2 (
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU=
> >> ).
> >>>
> >>>
> >>>
> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>
> >>
> 

RE: Urgent help needed : Cassandra benchmarking with Cassandra Stress tool

[Vinay Vongour]

Hi Murukesh

Here is my Cassandra.yaml file in GitHub gist
https://gist.github.com/vvinayreddy/bdaf6c54fa426256bfca69bfca32c0a6 

Thanks and Regards
Vinay Kumar Vongour

-Original Message-
From: Murukesh Mohanan  
Sent: Friday, June 01, 2018 1:10 PM
To: dev@cassandra.apache.org
Subject: Re: Urgent help needed : Cassandra benchmarking with Cassandra Stress 
tool

Attachments don't work well with mailing lists. You might want to post the yaml 
to a GitHub gist or some other public pastebin site and provide the link 
instead. Also, I think this topic is more suited to the user@ mailing list 
rather then this development-focused mailing list.
On Sat, Jun 2, 2018 at 1:20 Vinay Vongour 
wrote:

> Hi all
>
>
>
> Hope you all doing good
>
> I am working on benchmarking Cassandra using Cassandra stress tool and 
> results are not very impressive. I really some help to tune Cassandra 
> to get great results. Below is my environment and test results and 
> attached is Cassandra.yaml I am using for the test.
>
>
>
> I am running single node Cassandra virtual machine and 1 SSD disk in 
> VMWare ESXi 6.7 eventually I want to scale the database to 4 fours 
> once I get the expected results in the single node. Attached is the 
> Cassandra.yaml file is used for configuring database.
>
>
>
> The result I am getting is not very impressive operations/sec is very 
> low and the latency is very high for the reads here is the results it 
> got from the test.
>
> Load command: Cassandra-stress write n=40 -rate threads=96 
> -node
> $node1 -log file=load_400M.log
>
>
>
> *Results:*
>
> Op rate   :2,347 op/s  [READ: 2,347 op/s]
>
> Partition rate:2,347 pk/s  [READ: 2,347 pk/s]
>
> Row rate  :2,347 row/s [READ: 2,347 row/s]
>
> Latency mean  :  381.1 ms [READ: 381.1 ms]
>
> Latency median:  327.4 ms [READ: 327.4 ms]
>
> Latency 95th percentile   :  836.2 ms [READ: 836.2 ms]
>
> Latency 99th percentile   : 1138.8 ms [READ: 1,138.8 ms]
>
> Latency 99.9th percentile : 1695.5 ms [READ: 1,695.5 ms]
>
> Latency max   : 6253.7 ms [READ: 6,253.7 ms]
>
> Total partitions  :  1,000,000 [READ: 1,000,000]
>
> Total errors  :  0 [READ: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 00:07:06
>
>
>
> The write performance is really impressive with less latency and good 
> operation/sec and when we compare it with reads I feel something is wrong.
>
> Read command: Cassandra-stress read n=1  no-warmup cl=one 
> -mode native cql3 -schema keyspace=”keyspace1” -rate threads=96 -node 
> $node1 -log file=read_10M_test.log
>
>
>
> Results:
>
> Op rate   :   35,979 op/s  [WRITE: 35,979 op/s]
>
> Partition rate:   35,979 pk/s  [WRITE: 35,979 pk/s]
>
> Row rate  :   35,979 row/s [WRITE: 35,979 row/s]
>
> Latency mean  :3.3 ms [WRITE: 3.3 ms]
>
> Latency median:2.2 ms [WRITE: 2.2 ms]
>
> Latency 95th percentile   :8.2 ms [WRITE: 8.2 ms]
>
> Latency 99th percentile   :   17.6 ms [WRITE: 17.6 ms]
>
> Latency 99.9th percentile :   75.6 ms [WRITE: 75.6 ms]
>
> Latency max   : 1840.3 ms [WRITE: 1,840.3 ms]
>
> Total partitions  : 1,600,000,000 [WRITE: 1,600,000,000]
>
> Total errors  :  0 [WRITE: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 12:21:10
>
>
>
>
>
> When I compare the both read and write performance they are not at all 
> consistent and I feel there is some kind of bottleneck with the reads 
> during test.
>
>
>
> Could you please look into these details and let me know where I am 
> missing I have been struggling hard to get good results out of 
> Cassandra database but I couldn’t figure out where I am missing.
>
>
>
> Please let me know if you need additional information about the 
> Cassandra setup.
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org

-- 

Muru

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[J. D. Jordan]

All for using six and supporting both. Sorry, I read your initial email as 
wanting to drop support for 2 at the end of the year.

> On Jun 1, 2018, at 1:01 PM, Jonathan Haddad  wrote:
> 
> And that's why I said supporting both with six is the right path
> forward, later dropping support for 2.  I'm not advocating we drop 2
> support now, and I'm not asking for any sort of commitment.  I didn't
> think adding support for 3 would be so controversial.
> On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
>  wrote:
>> 
>> The community of people doing python development and the community of people 
>> running Cassandra servers are not the same.  I am not fine riding the coat 
>> tails of libraries used in python development.  As others have stated we 
>> need to be following the lead of the OS vendors that people will be 
>> deploying Cassandra on top of.  And those will not be dropping Python 2 at 
>> the end of the year.
>> 
>> -Jeremiah
>> 
>>> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad  wrote:
>>> 
>>> Both can work.  I did a lot of the work on the port of the Python
>>> driver's object mapper (formerly cqlengine) to Python 3.  It's
>>> reasonably straightforward if you use the six library.
>>> 
>>> Both pandas and numpy are dropping support for Python 2 at the end of
>>> this year.  I'm fine with riding on their coattails.
 On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman  
 wrote:
 
 Support for, but not the very script, right? Because, as gently pointed
 out by several realists here, Python 2 is far from dead and arguably
 still the majority usage. That's only just now beginning to change. I
 think it will be more than 2 years before people begin asking what
 Python 2 was.
 
 
> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> Supporting both as a next step is logical, removing support for 2 in the
> next year or two seems reasonable enough. Gotta rip the band aid off at
> some point.
> 
>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  
>> wrote:
>> 
>> Hi,
>> 
>> Deprecating in this context does not mean removing it or it being
>> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
>> refers to future versions (>7), but there are none at this point. It
>> appears Ubuntu has deviated from Debian in this sense, but Debian has
>> not changed yet (likely Debian 10 will, but that's not out yet and has
>> no announced release date).
>> 
>> Thus, 2.x still remains the most used version for servers. And servers
>> deployed at this point of time will use these versions for years.
>> 
>>   - Micke
>> 
>> 
>>> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
 On 2018/06/01 07:40:04, Michael Burman  wrote:
 IIRC, there's no major distribution yet that defaults to Python 3 (I
 think Ubuntu & Debian are still defaulting to Python 2 also). This will
 happen eventually (maybe), but not yet. Discarding Python 2 support
 would mean more base-OS work for most people wanting to run Cassandra
 and that's not a positive thing.
 
>>> Ubuntu since 16.04 defaults to Python 3:
>>> 
 Python2 is not installed anymore by default on the server, cloud and
>> the touch images, long live Python3! Python3 itself has been upgraded to
>> the 3.5 series. -
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc=
>>> RHEL 7.5 deprecates Python 2 (
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU=
>> ).
>>> 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
>> --
> Jon Haddad
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=ElPVVa0MdfruNq11vJS0JQo6LYDBQVJIVMFHQIEHnT4=
> twitter: rustyrazorblade
> 
 
>>> 
>>> 
>>> --
>>> Jon Haddad
>>> 

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Murukesh Mohanan]

If anything, riding on their coattails will mean we'll drop support a year
or two later, after we see their experiences - which will be nicely in line
with the Python upstream EOL date of 2020. (Of course, distro vendors will
provide support after that, say till 2025.) So perhaps 2020 should be a
reasonable deadline for Cassandra to drop support too? Assuming C* version
X EOLing in 2025 will be the last to have support for Python 2 is released
around 2020.
On Sat, Jun 2, 2018 at 2:01 Jonathan Haddad  wrote:

> And that's why I said supporting both with six is the right path
> forward, later dropping support for 2.  I'm not advocating we drop 2
> support now, and I'm not asking for any sort of commitment.  I didn't
> think adding support for 3 would be so controversial.
> On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
>  wrote:
> >
> > The community of people doing python development and the community of
> people running Cassandra servers are not the same.  I am not fine riding
> the coat tails of libraries used in python development.  As others have
> stated we need to be following the lead of the OS vendors that people will
> be deploying Cassandra on top of.  And those will not be dropping Python 2
> at the end of the year.
> >
> > -Jeremiah
> >
> > > On Jun 1, 2018, at 12:37 PM, Jonathan Haddad 
> wrote:
> > >
> > > Both can work.  I did a lot of the work on the port of the Python
> > > driver's object mapper (formerly cqlengine) to Python 3.  It's
> > > reasonably straightforward if you use the six library.
> > >
> > > Both pandas and numpy are dropping support for Python 2 at the end of
> > > this year.  I'm fine with riding on their coattails.
> > > On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman 
> wrote:
> > >>
> > >> Support for, but not the very script, right? Because, as gently
> pointed
> > >> out by several realists here, Python 2 is far from dead and arguably
> > >> still the majority usage. That's only just now beginning to change. I
> > >> think it will be more than 2 years before people begin asking what
> > >> Python 2 was.
> > >>
> > >>
> > >> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > >>> Supporting both as a next step is logical, removing support for 2 in
> the
> > >>> next year or two seems reasonable enough. Gotta rip the band aid off
> at
> > >>> some point.
> > >>>
> > >>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman 
> wrote:
> > >>>
> >  Hi,
> > 
> >  Deprecating in this context does not mean removing it or it being
> >  replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> >  refers to future versions (>7), but there are none at this point. It
> >  appears Ubuntu has deviated from Debian in this sense, but Debian
> has
> >  not changed yet (likely Debian 10 will, but that's not out yet and
> has
> >  no announced release date).
> > 
> >  Thus, 2.x still remains the most used version for servers. And
> servers
> >  deployed at this point of time will use these versions for years.
> > 
> > - Micke
> > 
> > 
> >  On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> > > On 2018/06/01 07:40:04, Michael Burman 
> wrote:
> > >> IIRC, there's no major distribution yet that defaults to Python 3
> (I
> > >> think Ubuntu & Debian are still defaulting to Python 2 also).
> This will
> > >> happen eventually (maybe), but not yet. Discarding Python 2
> support
> > >> would mean more base-OS work for most people wanting to run
> Cassandra
> > >> and that's not a positive thing.
> > >>
> > > Ubuntu since 16.04 defaults to Python 3:
> > >
> > >> Python2 is not installed anymore by default on the server, cloud
> and
> >  the touch images, long live Python3! Python3 itself has been
> upgraded to
> >  the 3.5 series. -
> > 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc=
> > > RHEL 7.5 deprecates Python 2 (
> > 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU=
> >  ).
> > >
> > >
> > >
> -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > 
> > 
> -
> >  To unsubscribe, e-mail: 

Re: Urgent help needed : Cassandra benchmarking with Cassandra Stress tool

[Murukesh Mohanan]

Attachments don't work well with mailing lists. You might want to post the
yaml to a GitHub gist or some other public pastebin site and provide the
link instead. Also, I think this topic is more suited to the user@ mailing
list rather then this development-focused mailing list.
On Sat, Jun 2, 2018 at 1:20 Vinay Vongour 
wrote:

> Hi all
>
>
>
> Hope you all doing good
>
> I am working on benchmarking Cassandra using Cassandra stress tool and
> results are not very impressive. I really some help to tune Cassandra to
> get great results. Below is my environment and test results and attached is
> Cassandra.yaml I am using for the test.
>
>
>
> I am running single node Cassandra virtual machine and 1 SSD disk in
> VMWare ESXi 6.7 eventually I want to scale the database to 4 fours once I
> get the expected results in the single node. Attached is the Cassandra.yaml
> file is used for configuring database.
>
>
>
> The result I am getting is not very impressive operations/sec is very low
> and the latency is very high for the reads here is the results it got from
> the test.
>
> Load command: Cassandra-stress write n=40 -rate threads=96 -node
> $node1 -log file=load_400M.log
>
>
>
> *Results:*
>
> Op rate   :2,347 op/s  [READ: 2,347 op/s]
>
> Partition rate:2,347 pk/s  [READ: 2,347 pk/s]
>
> Row rate  :2,347 row/s [READ: 2,347 row/s]
>
> Latency mean  :  381.1 ms [READ: 381.1 ms]
>
> Latency median:  327.4 ms [READ: 327.4 ms]
>
> Latency 95th percentile   :  836.2 ms [READ: 836.2 ms]
>
> Latency 99th percentile   : 1138.8 ms [READ: 1,138.8 ms]
>
> Latency 99.9th percentile : 1695.5 ms [READ: 1,695.5 ms]
>
> Latency max   : 6253.7 ms [READ: 6,253.7 ms]
>
> Total partitions  :  1,000,000 [READ: 1,000,000]
>
> Total errors  :  0 [READ: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 00:07:06
>
>
>
> The write performance is really impressive with less latency and good
> operation/sec and when we compare it with reads I feel something is wrong.
>
> Read command: Cassandra-stress read n=1  no-warmup cl=one -mode
> native cql3 -schema keyspace=”keyspace1” -rate threads=96 -node $node1 -log
> file=read_10M_test.log
>
>
>
> Results:
>
> Op rate   :   35,979 op/s  [WRITE: 35,979 op/s]
>
> Partition rate:   35,979 pk/s  [WRITE: 35,979 pk/s]
>
> Row rate  :   35,979 row/s [WRITE: 35,979 row/s]
>
> Latency mean  :3.3 ms [WRITE: 3.3 ms]
>
> Latency median:2.2 ms [WRITE: 2.2 ms]
>
> Latency 95th percentile   :8.2 ms [WRITE: 8.2 ms]
>
> Latency 99th percentile   :   17.6 ms [WRITE: 17.6 ms]
>
> Latency 99.9th percentile :   75.6 ms [WRITE: 75.6 ms]
>
> Latency max   : 1840.3 ms [WRITE: 1,840.3 ms]
>
> Total partitions  : 1,600,000,000 [WRITE: 1,600,000,000]
>
> Total errors  :  0 [WRITE: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 12:21:10
>
>
>
>
>
> When I compare the both read and write performance they are not at all
> consistent and I feel there is some kind of bottleneck with the reads
> during test.
>
>
>
> Could you please look into these details and let me know where I am
> missing I have been struggling hard to get good results out of Cassandra
> database but I couldn’t figure out where I am missing.
>
>
>
> Please let me know if you need additional information about the Cassandra
> setup.
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org

-- 

Muru

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Jonathan Haddad]

And that's why I said supporting both with six is the right path
forward, later dropping support for 2.  I'm not advocating we drop 2
support now, and I'm not asking for any sort of commitment.  I didn't
think adding support for 3 would be so controversial.
On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
 wrote:
>
> The community of people doing python development and the community of people 
> running Cassandra servers are not the same.  I am not fine riding the coat 
> tails of libraries used in python development.  As others have stated we need 
> to be following the lead of the OS vendors that people will be deploying 
> Cassandra on top of.  And those will not be dropping Python 2 at the end of 
> the year.
>
> -Jeremiah
>
> > On Jun 1, 2018, at 12:37 PM, Jonathan Haddad  wrote:
> >
> > Both can work.  I did a lot of the work on the port of the Python
> > driver's object mapper (formerly cqlengine) to Python 3.  It's
> > reasonably straightforward if you use the six library.
> >
> > Both pandas and numpy are dropping support for Python 2 at the end of
> > this year.  I'm fine with riding on their coattails.
> > On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman  
> > wrote:
> >>
> >> Support for, but not the very script, right? Because, as gently pointed
> >> out by several realists here, Python 2 is far from dead and arguably
> >> still the majority usage. That's only just now beginning to change. I
> >> think it will be more than 2 years before people begin asking what
> >> Python 2 was.
> >>
> >>
> >> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> >>> Supporting both as a next step is logical, removing support for 2 in the
> >>> next year or two seems reasonable enough. Gotta rip the band aid off at
> >>> some point.
> >>>
> >>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  wrote:
> >>>
>  Hi,
> 
>  Deprecating in this context does not mean removing it or it being
>  replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
>  refers to future versions (>7), but there are none at this point. It
>  appears Ubuntu has deviated from Debian in this sense, but Debian has
>  not changed yet (likely Debian 10 will, but that's not out yet and has
>  no announced release date).
> 
>  Thus, 2.x still remains the most used version for servers. And servers
>  deployed at this point of time will use these versions for years.
> 
> - Micke
> 
> 
>  On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> > On 2018/06/01 07:40:04, Michael Burman  wrote:
> >> IIRC, there's no major distribution yet that defaults to Python 3 (I
> >> think Ubuntu & Debian are still defaulting to Python 2 also). This will
> >> happen eventually (maybe), but not yet. Discarding Python 2 support
> >> would mean more base-OS work for most people wanting to run Cassandra
> >> and that's not a positive thing.
> >>
> > Ubuntu since 16.04 defaults to Python 3:
> >
> >> Python2 is not installed anymore by default on the server, cloud and
>  the touch images, long live Python3! Python3 itself has been upgraded to
>  the 3.5 series. -
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc=
> > RHEL 7.5 deprecates Python 2 (
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU=
>  ).
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> 
>  -
>  To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>  For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
>  --
> >>> Jon Haddad
> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=ElPVVa0MdfruNq11vJS0JQo6LYDBQVJIVMFHQIEHnT4=
> >>> twitter: rustyrazorblade
> >>>
> >>
> >
> >
> > --
> > Jon Haddad
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=ElPVVa0MdfruNq11vJS0JQo6LYDBQVJIVMFHQIEHnT4=
> > twitter: rustyrazorblade

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Jeremiah D Jordan]

The community of people doing python development and the community of people 
running Cassandra servers are not the same.  I am not fine riding the coat 
tails of libraries used in python development.  As others have stated we need 
to be following the lead of the OS vendors that people will be deploying 
Cassandra on top of.  And those will not be dropping Python 2 at the end of the 
year.

-Jeremiah

> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad  wrote:
> 
> Both can work.  I did a lot of the work on the port of the Python
> driver's object mapper (formerly cqlengine) to Python 3.  It's
> reasonably straightforward if you use the six library.
> 
> Both pandas and numpy are dropping support for Python 2 at the end of
> this year.  I'm fine with riding on their coattails.
> On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman  wrote:
>> 
>> Support for, but not the very script, right? Because, as gently pointed
>> out by several realists here, Python 2 is far from dead and arguably
>> still the majority usage. That's only just now beginning to change. I
>> think it will be more than 2 years before people begin asking what
>> Python 2 was.
>> 
>> 
>> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
>>> Supporting both as a next step is logical, removing support for 2 in the
>>> next year or two seems reasonable enough. Gotta rip the band aid off at
>>> some point.
>>> 
>>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  wrote:
>>> 
 Hi,
 
 Deprecating in this context does not mean removing it or it being
 replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
 refers to future versions (>7), but there are none at this point. It
 appears Ubuntu has deviated from Debian in this sense, but Debian has
 not changed yet (likely Debian 10 will, but that's not out yet and has
 no announced release date).
 
 Thus, 2.x still remains the most used version for servers. And servers
 deployed at this point of time will use these versions for years.
 
- Micke
 
 
 On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> On 2018/06/01 07:40:04, Michael Burman  wrote:
>> IIRC, there's no major distribution yet that defaults to Python 3 (I
>> think Ubuntu & Debian are still defaulting to Python 2 also). This will
>> happen eventually (maybe), but not yet. Discarding Python 2 support
>> would mean more base-OS work for most people wanting to run Cassandra
>> and that's not a positive thing.
>> 
> Ubuntu since 16.04 defaults to Python 3:
> 
>> Python2 is not installed anymore by default on the server, cloud and
 the touch images, long live Python3! Python3 itself has been upgraded to
 the 3.5 series. -
 https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc=
> RHEL 7.5 deprecates Python 2 (
 https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU=
 ).
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
 For additional commands, e-mail: dev-h...@cassandra.apache.org
 
 --
>>> Jon Haddad
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=ElPVVa0MdfruNq11vJS0JQo6LYDBQVJIVMFHQIEHnT4=
>>> twitter: rustyrazorblade
>>> 
>> 
> 
> 
> -- 
> Jon Haddad
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA=ElPVVa0MdfruNq11vJS0JQo6LYDBQVJIVMFHQIEHnT4=
> twitter: rustyrazorblade
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Jonathan Haddad]

Both can work.  I did a lot of the work on the port of the Python
driver's object mapper (formerly cqlengine) to Python 3.  It's
reasonably straightforward if you use the six library.

Both pandas and numpy are dropping support for Python 2 at the end of
this year.  I'm fine with riding on their coattails.
On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman  wrote:
>
> Support for, but not the very script, right? Because, as gently pointed
> out by several realists here, Python 2 is far from dead and arguably
> still the majority usage. That's only just now beginning to change. I
> think it will be more than 2 years before people begin asking what
> Python 2 was.
>
>
> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > Supporting both as a next step is logical, removing support for 2 in the
> > next year or two seems reasonable enough. Gotta rip the band aid off at
> > some point.
> >
> > On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  wrote:
> >
> >> Hi,
> >>
> >> Deprecating in this context does not mean removing it or it being
> >> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> >> refers to future versions (>7), but there are none at this point. It
> >> appears Ubuntu has deviated from Debian in this sense, but Debian has
> >> not changed yet (likely Debian 10 will, but that's not out yet and has
> >> no announced release date).
> >>
> >> Thus, 2.x still remains the most used version for servers. And servers
> >> deployed at this point of time will use these versions for years.
> >>
> >> - Micke
> >>
> >>
> >> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> >>> On 2018/06/01 07:40:04, Michael Burman  wrote:
>  IIRC, there's no major distribution yet that defaults to Python 3 (I
>  think Ubuntu & Debian are still defaulting to Python 2 also). This will
>  happen eventually (maybe), but not yet. Discarding Python 2 support
>  would mean more base-OS work for most people wanting to run Cassandra
>  and that's not a positive thing.
> 
> >>> Ubuntu since 16.04 defaults to Python 3:
> >>>
>  Python2 is not installed anymore by default on the server, cloud and
> >> the touch images, long live Python3! Python3 itself has been upgraded to
> >> the 3.5 series. -
> >> https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3
> >>> RHEL 7.5 deprecates Python 2 (
> >> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality
> >> ).
> >>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >> --
> > Jon Haddad
> > http://www.rustyrazorblade.com
> > twitter: rustyrazorblade
> >
>


-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Russell Bateman]

Support for, but not the very script, right? Because, as gently pointed 
out by several realists here, Python 2 is far from dead and arguably 
still the majority usage. That's only just now beginning to change. I 
think it will be more than 2 years before people begin asking what 
Python 2 was.



On 06/01/2018 10:10 AM, Jonathan Haddad wrote:

Supporting both as a next step is logical, removing support for 2 in the
next year or two seems reasonable enough. Gotta rip the band aid off at
some point.

On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  wrote:


Hi,

Deprecating in this context does not mean removing it or it being
replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
refers to future versions (>7), but there are none at this point. It
appears Ubuntu has deviated from Debian in this sense, but Debian has
not changed yet (likely Debian 10 will, but that's not out yet and has
no announced release date).

Thus, 2.x still remains the most used version for servers. And servers
deployed at this point of time will use these versions for years.

- Micke


On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:

On 2018/06/01 07:40:04, Michael Burman  wrote:

IIRC, there's no major distribution yet that defaults to Python 3 (I
think Ubuntu & Debian are still defaulting to Python 2 also). This will
happen eventually (maybe), but not yet. Discarding Python 2 support
would mean more base-OS work for most people wanting to run Cassandra
and that's not a positive thing.


Ubuntu since 16.04 defaults to Python 3:


Python2 is not installed anymore by default on the server, cloud and

the touch images, long live Python3! Python3 itself has been upgraded to
the 3.5 series. -
https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3

RHEL 7.5 deprecates Python 2 (

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality
).



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

--

Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

Urgent help needed : Cassandra benchmarking with Cassandra Stress tool

[Vinay Vongour]

Hi all

Hope you all doing good
I am working on benchmarking Cassandra using Cassandra stress tool and results 
are not very impressive. I really some help to tune Cassandra to get great 
results. Below is my environment and test results and attached is 
Cassandra.yaml I am using for the test.

I am running single node Cassandra virtual machine and 1 SSD disk in VMWare 
ESXi 6.7 eventually I want to scale the database to 4 fours once I get the 
expected results in the single node. Attached is the Cassandra.yaml file is 
used for configuring database.

The result I am getting is not very impressive operations/sec is very low and 
the latency is very high for the reads here is the results it got from the test.
Load command: Cassandra-stress write n=40 -rate threads=96 -node $node1 
-log file=load_400M.log

Results:
Op rate   :2,347 op/s  [READ: 2,347 op/s]
Partition rate:2,347 pk/s  [READ: 2,347 pk/s]
Row rate  :2,347 row/s [READ: 2,347 row/s]
Latency mean  :  381.1 ms [READ: 381.1 ms]
Latency median:  327.4 ms [READ: 327.4 ms]
Latency 95th percentile   :  836.2 ms [READ: 836.2 ms]
Latency 99th percentile   : 1138.8 ms [READ: 1,138.8 ms]
Latency 99.9th percentile : 1695.5 ms [READ: 1,695.5 ms]
Latency max   : 6253.7 ms [READ: 6,253.7 ms]
Total partitions  :  1,000,000 [READ: 1,000,000]
Total errors  :  0 [READ: 0]
Total GC count: 0
Total GC memory   : 0.000 KiB
Total GC time :0.0 seconds
Avg GC time   :NaN ms
StdDev GC time:0.0 ms
Total operation time  : 00:07:06

The write performance is really impressive with less latency and good 
operation/sec and when we compare it with reads I feel something is wrong.
Read command: Cassandra-stress read n=1  no-warmup cl=one -mode native 
cql3 -schema keyspace="keyspace1" -rate threads=96 -node $node1 -log 
file=read_10M_test.log

Results:
Op rate   :   35,979 op/s  [WRITE: 35,979 op/s]
Partition rate:   35,979 pk/s  [WRITE: 35,979 pk/s]
Row rate  :   35,979 row/s [WRITE: 35,979 row/s]
Latency mean  :3.3 ms [WRITE: 3.3 ms]
Latency median:2.2 ms [WRITE: 2.2 ms]
Latency 95th percentile   :8.2 ms [WRITE: 8.2 ms]
Latency 99th percentile   :   17.6 ms [WRITE: 17.6 ms]
Latency 99.9th percentile :   75.6 ms [WRITE: 75.6 ms]
Latency max   : 1840.3 ms [WRITE: 1,840.3 ms]
Total partitions  : 1,600,000,000 [WRITE: 1,600,000,000]
Total errors  :  0 [WRITE: 0]
Total GC count: 0
Total GC memory   : 0.000 KiB
Total GC time :0.0 seconds
Avg GC time   :NaN ms
StdDev GC time:0.0 ms
Total operation time  : 12:21:10


When I compare the both read and write performance they are not at all 
consistent and I feel there is some kind of bottleneck with the reads during 
test.

Could you please look into these details and let me know where I am missing I 
have been struggling hard to get good results out of Cassandra database but I 
couldn't figure out where I am missing.

Please let me know if you need additional information about the Cassandra setup.


Thanks and Regards
Vinay Kumar Vongour



Thanks and Regards
Vinay Kumar Vongour


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Jonathan Haddad]

Supporting both as a next step is logical, removing support for 2 in the
next year or two seems reasonable enough. Gotta rip the band aid off at
some point.

On Fri, Jun 1, 2018 at 2:34 AM Michael Burman  wrote:

> Hi,
>
> Deprecating in this context does not mean removing it or it being
> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> refers to future versions (>7), but there are none at this point. It
> appears Ubuntu has deviated from Debian in this sense, but Debian has
> not changed yet (likely Debian 10 will, but that's not out yet and has
> no announced release date).
>
> Thus, 2.x still remains the most used version for servers. And servers
> deployed at this point of time will use these versions for years.
>
>- Micke
>
>
> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> > On 2018/06/01 07:40:04, Michael Burman  wrote:
> >> IIRC, there's no major distribution yet that defaults to Python 3 (I
> >> think Ubuntu & Debian are still defaulting to Python 2 also). This will
> >> happen eventually (maybe), but not yet. Discarding Python 2 support
> >> would mean more base-OS work for most people wanting to run Cassandra
> >> and that's not a positive thing.
> >>
> > Ubuntu since 16.04 defaults to Python 3:
> >
> >> Python2 is not installed anymore by default on the server, cloud and
> the touch images, long live Python3! Python3 itself has been upgraded to
> the 3.5 series. -
> https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3
> > RHEL 7.5 deprecates Python 2 (
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality
> ).
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Michael Burman]

Hi,

Deprecating in this context does not mean removing it or it being 
replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It 
refers to future versions (>7), but there are none at this point. It 
appears Ubuntu has deviated from Debian in this sense, but Debian has 
not changed yet (likely Debian 10 will, but that's not out yet and has 
no announced release date).


Thus, 2.x still remains the most used version for servers. And servers 
deployed at this point of time will use these versions for years.


  - Micke


On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:

On 2018/06/01 07:40:04, Michael Burman  wrote:

IIRC, there's no major distribution yet that defaults to Python 3 (I
think Ubuntu & Debian are still defaulting to Python 2 also). This will
happen eventually (maybe), but not yet. Discarding Python 2 support
would mean more base-OS work for most people wanting to run Cassandra
and that's not a positive thing.


Ubuntu since 16.04 defaults to Python 3:


Python2 is not installed anymore by default on the server, cloud and the touch 
images, long live Python3! Python3 itself has been upgraded to the 3.5 series. 
- https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3

RHEL 7.5 deprecates Python 2 
(https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality).



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Murukesh Mohanan]

On 2018/06/01 07:40:04, Michael Burman  wrote: 
> 
> IIRC, there's no major distribution yet that defaults to Python 3 (I 
> think Ubuntu & Debian are still defaulting to Python 2 also). This will 
> happen eventually (maybe), but not yet. Discarding Python 2 support 
> would mean more base-OS work for most people wanting to run Cassandra 
> and that's not a positive thing.
> 

Ubuntu since 16.04 defaults to Python 3:

> Python2 is not installed anymore by default on the server, cloud and the 
> touch images, long live Python3! Python3 itself has been upgraded to the 3.5 
> series. - https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3  

RHEL 7.5 deprecates Python 2 
(https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality).



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Michael Burman]

Hi,

Should definitely be cross compatible with Python 2/3. Most of the 
systems (such as those running on RHEL7 or distros based on it like 
CentOS) are shipping with 2.7 only by default. And these systems are 
probably going to be used for a long time to run Cassandra.


IIRC, there's no major distribution yet that defaults to Python 3 (I 
think Ubuntu & Debian are still defaulting to Python 2 also). This will 
happen eventually (maybe), but not yet. Discarding Python 2 support 
would mean more base-OS work for most people wanting to run Cassandra 
and that's not a positive thing.


For future, 2 & 3 compatibility would mean that we support larger amount 
of distributions out of the box.


  - Micke


On 06/01/2018 05:44 AM, Patrick Bannister wrote:

I propose porting cqlsh and cqlshlib to Python 3. End-of-life for Python 2.7
 is currently planned for 1
January 2020. We should prepare to port the tool to a version of Python
that will be officially supported.

I'm seeking input on three questions:
- Should we port it to straight Python 3, or Python 2/3 cross compatible?
- How much more testing is needed?
- Can we wait until after 4.0 for this?

I have an implementation
 to go with my
proposal. In parallel with getting the dtest cqlsh_tests working again, I
ported cqlsh.py and cqlshlib to Python 3. It passes with almost all of the
dtests and the unittests, so it's in pretty good shape, although it's not
100% done (more on that below).

*Python 3 or 2/3 cross compatible?* There are plenty of examples of Python
libraries that are compatible with both Python 2 and Python 3 (notably the
Cassandra Python driver), so I think this is achievable. The question is,
do we want to pay the price of cross compatibility? If we write cqlsh to be
2/3 cross compatible, we'll carry a long term technical debt to maintain
that feature. The value of continuing to support Python 2 will diminish
over time. However, a cross compatible implementation may ease the
transition for some users, especially if there are users who have made
significant custom modifications to the Python 2.7 implementation of cqlsh,
so I think we must at least consider the question.

*What additional testing is needed before we could release it?* I used
coverage.py to check on the code coverage of our existing dtest cqlsh_tests
and cqlshlib unittests. There are several blind spots in our current
testing that should be addressed before we release a port of cqlsh. Details
of this are available on JIRA ticket CASSANDRA-10190
 in the attachment
coverage_notes.txt
.
Beyond that, I've made no efforts to test on platforms other than Ubuntu
and CentOS, so Windows testing is needed if we're making efforts to support
Windows. It would also be preferable for some real users to try out the
port before it replaces the Python 2.7 cqlsh in a release.

Besides this, there are a couple of test failures I'm still trying to
figure out, notably tests involving user defined map types (a task made
more interesting by Python's general lack of support for immutable map
types).

*Can we wait until after 4.0 for this?* I don't think it's reasonable to
try to release this with 4.0 given the current consensus around a feature
freeze in the next few months. My feeling is that our testers and
committers are already very busy with the currently planned changes for
4.0. I recommend planning toward a release to occur after 4.0. If we run up
against Python 2.7 EOL before we can cut the next release, we could
consider releasing a ported cqlsh independently, for installation through
distutils or pip.

Patrick Bannister




-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org