[jira] [Commented] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344220#comment-15344220
]
Giampaolo commented on CASSANDRA-10273:
---
Yes, [~ifesdjeen], but I would not like to "hijack" this JIRA too much. rather
I would prefer to open new one.
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> # pre-flight (startup tests) scrub
> # pre-flight (startup tests) sstable min version
> # {{ColumnFamilyStore.createColumnFamilyStore}}
> # {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11516) Make max number of streams configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo reassigned CASSANDRA-11516: - Assignee: Giampaolo > Make max number of streams configurable > --- > > Key: CASSANDRA-11516 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11516 > Project: Cassandra > Issue Type: New Feature >Reporter: Sebastian Estevez >Assignee: Giampaolo > Labels: lhf > > Today we default to num cores. In large boxes (many cores), this is > suboptimal as it can generate huge amounts of garbage that GC can't keep up > with. > Usually we tackle issues like this with the streaming throughput levers but > in this case the problem is CPU consumption by StreamReceiverTasks > specifically in the IntervalTree build -- > https://github.com/apache/cassandra/blob/cassandra-2.1.12/src/java/org/apache/cassandra/utils/IntervalTree.java#L257 > We need a max number of parallel streams lever to hanlde this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11516) Make max number of streams configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334171#comment-15334171
]
Giampaolo commented on CASSANDRA-11516:
---
Thanks, [~pauloricardomg] for the pointer. That was my first choice at the
beginning but the issue refers to {{StreamReceiveTask}}. I will go with the one
you gave me.
> Make max number of streams configurable
> ---
>
> Key: CASSANDRA-11516
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11516
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Sebastian Estevez
> Labels: lhf
>
> Today we default to num cores. In large boxes (many cores), this is
> suboptimal as it can generate huge amounts of garbage that GC can't keep up
> with.
> Usually we tackle issues like this with the streaming throughput levers but
> in this case the problem is CPU consumption by StreamReceiverTasks
> specifically in the IntervalTree build --
> https://github.com/apache/cassandra/blob/cassandra-2.1.12/src/java/org/apache/cassandra/utils/IntervalTree.java#L257
> We need a max number of parallel streams lever to hanlde this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11516) Make max number of streams configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333742#comment-15333742
]
Giampaolo commented on CASSANDRA-11516:
---
I'm studying how to solve this issue. A quick question: do you mean to put a
configuration for [this
line|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java#L62]
using {{newFixedThreadPool}} and defaulting to
{{FBUtilities#getAvailableProcessors}}?
> Make max number of streams configurable
> ---
>
> Key: CASSANDRA-11516
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11516
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Sebastian Estevez
> Labels: lhf
>
> Today we default to num cores. In large boxes (many cores), this is
> suboptimal as it can generate huge amounts of garbage that GC can't keep up
> with.
> Usually we tackle issues like this with the streaming throughput levers but
> in this case the problem is CPU consumption by StreamReceiverTasks
> specifically in the IntervalTree build --
> https://github.com/apache/cassandra/blob/cassandra-2.1.12/src/java/org/apache/cassandra/utils/IntervalTree.java#L257
> We need a max number of parallel streams lever to hanlde this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo updated CASSANDRA-10273:
--
Description:
ATM we scan each data directory four times. We could easily reduce that to at
least two, maybe to one.
# pre-flight (startup tests) scrub
# pre-flight (startup tests) sstable min version
# {{ColumnFamilyStore.createColumnFamilyStore}}
# {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
First two pre-flight tests could be combined to one and 3+4 could also be
combined, as both appear at pretty related code paths.
was:
ATM we scan each data directory four times. We could easily reduce that to at
least two, maybe to one.
1. pre-flight (startup tests) scrub
1. pre-flight (startup tests) sstable min version
1. {{ColumnFamilyStore.createColumnFamilyStore}}
1. {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
First two pre-flight tests could be combined to one and 3+4 could also be
combined, as both appear at pretty related code paths.
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> # pre-flight (startup tests) scrub
> # pre-flight (startup tests) sstable min version
> # {{ColumnFamilyStore.createColumnFamilyStore}}
> # {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329194#comment-15329194
]
Giampaolo commented on CASSANDRA-10273:
---
Sorry for the delay with this bug. I've tried to understand better the problem
but I have some observations to discuss. I studied a bit point 3 and 4 to
understand how to integrate them. For these points, "scan" means
{{SSTableLister.list}}
[https://github.com/apache/cassandra/blob/fed476f9c049128674841d1c46b868979352b1a5/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L412-L414]
{code:title=src/java/org/apache/cassandra/db/ColumnFamilyStore.java|borderStyle=solid}
Directories.SSTableLister sstableFiles =
directories.sstableLister(Directories.OnTxnErr.IGNORE).skipTemporary(true);
Collection sstables =
SSTableReader.openAll(sstableFiles.list().entrySet(), metadata);
{code}
and
[https://github.com/apache/cassandra/blob/fed476f9c049128674841d1c46b868979352b1a5/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L579-L581]
{code:title=src/java/org/apache/cassandra/db/ColumnFamilyStore.java|borderStyle=solid}
Directories directories = new Directories(metadata, initialDirectories);
Directories.SSTableLister lister =
directories.sstableLister(Directories.OnTxnErr.IGNORE).includeBackups(true);
List generations = new ArrayList();
for (Map.Entry entry : lister.list().entrySet())
{code}
The scans are done with different parameters and, due to internals of the
{{SSTableLister}}, after a scan an additional filter cannot be applied. In
facts, the {{SSTableLister}} is something like a one shot builder for
{{Map}} (please correct me if I'm doing a "flight
of fancy").
My poor conclusion is that the scans are a bit different and I did not see a
simple way to reuse existing classes to avoid one. Changes in the
{{SSTableLister}} are out of discussion.
But I'm here to ask to more experiences developers if I'm missing some points
and there's a smart way to accomplish this task I did not see.
As sub-product of my investigation, I saw that {{Directories}} class is
instantiated may times during startup with the same parameters, but this class
is like an immutable one. All fields are final and are passed via constructor.
Another little optimization could be to reduce the number of these instances
using a cache.
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> 1. pre-flight (startup tests) scrub
> 1. pre-flight (startup tests) sstable min version
> 1. {{ColumnFamilyStore.createColumnFamilyStore}}
> 1. {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322091#comment-15322091 ] Giampaolo commented on CASSANDRA-8700: -- I "vote" for Sphinx + reStructuredText too. Recently I did some documentation with it and was a smooth experience > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Minor > Attachments: drivers_list.md, installation.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294137#comment-15294137
]
Giampaolo commented on CASSANDRA-4:
---
Yes, thanks for your review. :)
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294023#comment-15294023
]
Giampaolo commented on CASSANDRA-4:
---
This sentence was taken directly from current documentation. I just removed the
sentence about limits on clustering column since it's not true. I think yours
is better, but what about removing the part on existing data?
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294019#comment-15294019
]
Giampaolo commented on CASSANDRA-4:
---
Yes, please. I didn't think about a unit test to enforce docs. It's a good idea.
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285274#comment-15285274
]
Giampaolo commented on CASSANDRA-4:
---
These are allowed conversions for clustering columns.
||Data type may be altered to:||Data type||
|_ascii_, _text_, _varchar_|_blob_|
|_ascii_, varchar|text|
|_ascii_, text|varchar|
Respect to
https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cql_data_types_c.html,
{{int}} -> {{varint}} case is not present anymore.
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo updated CASSANDRA-4:
--
Attachment: cassandra-4-trunk.patch
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo updated CASSANDRA-4:
--
Status: Patch Available (was: Open)
Patch is also available at
https://github.com/radicalbit/cassandra/tree/CASSANDRA-4-trunk.
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
> Attachments: cassandra-4-trunk.patch
>
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278712#comment-15278712
]
Giampaolo commented on CASSANDRA-4:
---
About this issue, I wrote a little
[program|https://github.com/giampaolotrapasso/cassandra-type-conversions] to
test all possible combinations of type conversion in {{ALTER TABLE}} statement.
I run this on 3.4, I will run again with the 3.5/3.6.
I've tried all basic types without considering when the column is a partition
key or a clustering column. This is the list of the allowed conversion
||From|To||
|_timestamp_|_bigint_|
|ascii, bigint, boolean, _date_, decimal, double, float, inet, int, _smallint_,
_text_, _time_, timestamp, timeuuid, _tinyint_, uuid, varchar, varint|blob|
|_int_|_date_|
|_bigint_|_time_|
|_ascii_, _varchar_|_text_|
|_bigint_|_timestamp_|
|_ascii_, _text_|_varchar_|
|_bigint_, _int_, _timestamp_|_varint_|
|timeuuid|uuid|
It differs a bit from what should be the most updated
[documentation|https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cql_data_types_c.html].
I have put in italic what are the additions respect to the mentioned url.
Going to do the same for clustering column, meanwhile any comment on this, if
any, is appreciated.
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11114) Document which type conversions are allowed
[
https://issues.apache.org/jira/browse/CASSANDRA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo reassigned CASSANDRA-4:
-
Assignee: Giampaolo
> Document which type conversions are allowed
> ---
>
> Key: CASSANDRA-4
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Documentation and Website
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: doc-impacting, lhf
>
> We allow only some type conversion through {{ALTER TABLE}} and type casts,
> the ones that don't break stuff, but we don't currently document which ones
> those are. We should add it to
> http://cassandra.apache.org/doc/cql3/CQL-3.0.html.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11546) Stress doesn't respect case-sensitive column names when building insert queries
[ https://issues.apache.org/jira/browse/CASSANDRA-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-11546: -- Reproduced In: 3.0.5, 2.2.5, 3.6 (was: 2.2.5, 3.0.5, 3.6) Status: Patch Available (was: In Progress) > Stress doesn't respect case-sensitive column names when building insert > queries > --- > > Key: CASSANDRA-11546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11546 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Joel Knighton >Assignee: Giampaolo >Priority: Trivial > Labels: lhf > Attachments: cassandra-11546-trunk-giampaolo-trapasso.patch, > example.yaml > > > When using a custom stress profile, if the schema uses case sensitive column > names, stress doesn't respect case sensitivity when building insert/update > statements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo reassigned CASSANDRA-10273:
-
Assignee: Giampaolo
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> 1. pre-flight (startup tests) scrub
> 1. pre-flight (startup tests) sstable min version
> 1. {{ColumnFamilyStore.createColumnFamilyStore}}
> 1. {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10273) Reduce number of data directory scans during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254745#comment-15254745
]
Giampaolo commented on CASSANDRA-10273:
---
I would like to take this, if [~alextkaiser] agrees.
> Reduce number of data directory scans during startup
>
>
> Key: CASSANDRA-10273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10273
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Priority: Minor
> Labels: lhf
>
> ATM we scan each data directory four times. We could easily reduce that to at
> least two, maybe to one.
> 1. pre-flight (startup tests) scrub
> 1. pre-flight (startup tests) sstable min version
> 1. {{ColumnFamilyStore.createColumnFamilyStore}}
> 1. {{ColumnFamilyStore.}} (if {{loadSSTables==true}})
> First two pre-flight tests could be combined to one and 3+4 could also be
> combined, as both appear at pretty related code paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11546) Stress doesn't respect case-sensitive column names when building insert queries
[
https://issues.apache.org/jira/browse/CASSANDRA-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254682#comment-15254682
]
Giampaolo edited comment on CASSANDRA-11546 at 4/22/16 9:04 PM:
Patch is available also at
[https://github.com/radicalbit/cassandra/tree/CASSANDRA-11546-trunk].
I've tested with {{stress user profile=example.yaml ops(insert=1,simple=9)
-node 127.0.0.1 -log}}.
I had not problem with {{ant test}}, while I could not run full dtest due to
limitation of my working machine.
Note: Maybe the static method {{quoteIdentifier(String identifier)}} could be
useful in some other place, but I think a wider refactoring is beyond the goal
of this issue.
was (Author: giampaolo):
Patch is available also at
[https://github.com/radicalbit/cassandra/tree/CASSANDRA-11546-trunk].
I've tested with `stress user profile=example.yaml ops(insert=1,simple=9) -node
127.0.0.1 -log` and solves the problem.
I had not problem with ant test, while I could not run full dtest on my limited
working machine.
Note: Maybe the static method `quoteIdentifier(String identifier)` could be
useful in some other place, but I think a wider refactoring is beyond the goal
of this issue.
> Stress doesn't respect case-sensitive column names when building insert
> queries
> ---
>
> Key: CASSANDRA-11546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11546
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Joel Knighton
>Assignee: Giampaolo
>Priority: Trivial
> Labels: lhf
> Attachments: cassandra-11546-trunk-giampaolo-trapasso.patch,
> example.yaml
>
>
> When using a custom stress profile, if the schema uses case sensitive column
> names, stress doesn't respect case sensitivity when building insert/update
> statements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11546) Stress doesn't respect case-sensitive column names when building insert queries
[ https://issues.apache.org/jira/browse/CASSANDRA-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-11546: -- Attachment: example.yaml cassandra-11546-trunk-giampaolo-trapasso.patch Patch is available also at [https://github.com/radicalbit/cassandra/tree/CASSANDRA-11546-trunk]. I've tested with `stress user profile=example.yaml ops(insert=1,simple=9) -node 127.0.0.1 -log` and solves the problem. I had not problem with ant test, while I could not run full dtest on my limited working machine. Note: Maybe the static method `quoteIdentifier(String identifier)` could be useful in some other place, but I think a wider refactoring is beyond the goal of this issue. > Stress doesn't respect case-sensitive column names when building insert > queries > --- > > Key: CASSANDRA-11546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11546 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Joel Knighton >Assignee: Giampaolo >Priority: Trivial > Labels: lhf > Attachments: cassandra-11546-trunk-giampaolo-trapasso.patch, > example.yaml > > > When using a custom stress profile, if the schema uses case sensitive column > names, stress doesn't respect case sensitivity when building insert/update > statements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11522) batch_size_fail_threshold_in_kb shouldn't only apply to batch
[
https://issues.apache.org/jira/browse/CASSANDRA-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242986#comment-15242986
]
Giampaolo commented on CASSANDRA-11522:
---
I forgot about {{max_mutation_size_kb}}, maybe because it's not a commented
property in the {{cassandra.yaml}} but somehow "hidden" into
{{commitlog_segment_size_in_mb}} description.
Anyway, I agree with you, a soft limit won't be of help. Thanks for the
clarification.
> batch_size_fail_threshold_in_kb shouldn't only apply to batch
> -
>
> Key: CASSANDRA-11522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11522
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> I can buy that C* is not good at dealing with large (in bytes) inserts and
> that it makes sense to provide a user configurable protection against inserts
> larger than a certain size, but it doesn't make sense to limit this to
> batches. It's absolutely possible to insert a single very large row and
> internally a batch with a single statement is exactly the same than a single
> similar insert, so rejecting the former and not the later is confusing and
> well, wrong.
> Note that I get that batches are more likely to get big and that's where the
> protection is most often useful, but limiting the option to batch is still
> less useful (it's a hole in the protection) and it's going to confuse users
> in thinking that batches to a single partition are different from single
> inserts.
> Of course that also mean that we should rename that option to
> {{write_size_fail_threshold_in_kb}}. Which means we probably want to add this
> new option and just deprecate {{batch_size_fail_threshold_in_kb}} for now
> (with removal in 4.0).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11522) batch_size_fail_threshold_in_kb shouldn't only apply to batch
[
https://issues.apache.org/jira/browse/CASSANDRA-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242682#comment-15242682
]
Giampaolo commented on CASSANDRA-11522:
---
If I understand correctly [~slebresne] concerns, there are two problems here:
* align single insert behaviour to single insert batch and
* warn against too large single partition inserts (as simple operation or
inside a batch).
While the first is solved by
[CASSANDRA-10876|https://issues.apache.org/jira/browse/CASSANDRA-10876] but
setting should be renamed, the second one remains open and could be discussed
as different issue.
> batch_size_fail_threshold_in_kb shouldn't only apply to batch
> -
>
> Key: CASSANDRA-11522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11522
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> I can buy that C* is not good at dealing with large (in bytes) inserts and
> that it makes sense to provide a user configurable protection against inserts
> larger than a certain size, but it doesn't make sense to limit this to
> batches. It's absolutely possible to insert a single very large row and
> internally a batch with a single statement is exactly the same than a single
> similar insert, so rejecting the former and not the later is confusing and
> well, wrong.
> Note that I get that batches are more likely to get big and that's where the
> protection is most often useful, but limiting the option to batch is still
> less useful (it's a hole in the protection) and it's going to confuse users
> in thinking that batches to a single partition are different from single
> inserts.
> Of course that also mean that we should rename that option to
> {{write_size_fail_threshold_in_kb}}. Which means we probably want to add this
> new option and just deprecate {{batch_size_fail_threshold_in_kb}} for now
> (with removal in 4.0).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11522) batch_size_fail_threshold_in_kb shouldn't only apply to batch
[
https://issues.apache.org/jira/browse/CASSANDRA-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giampaolo reassigned CASSANDRA-11522:
-
Assignee: Giampaolo
> batch_size_fail_threshold_in_kb shouldn't only apply to batch
> -
>
> Key: CASSANDRA-11522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11522
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> I can buy that C* is not good at dealing with large (in bytes) inserts and
> that it makes sense to provide a user configurable protection against inserts
> larger than a certain size, but it doesn't make sense to limit this to
> batches. It's absolutely possible to insert a single very large row and
> internally a batch with a single statement is exactly the same than a single
> similar insert, so rejecting the former and not the later is confusing and
> well, wrong.
> Note that I get that batches are more likely to get big and that's where the
> protection is most often useful, but limiting the option to batch is still
> less useful (it's a hole in the protection) and it's going to confuse users
> in thinking that batches to a single partition are different from single
> inserts.
> Of course that also mean that we should rename that option to
> {{write_size_fail_threshold_in_kb}}. Which means we probably want to add this
> new option and just deprecate {{batch_size_fail_threshold_in_kb}} for now
> (with removal in 4.0).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11522) batch_size_fail_threshold_in_kb shouldn't only apply to batch
[
https://issues.apache.org/jira/browse/CASSANDRA-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238932#comment-15238932
]
Giampaolo commented on CASSANDRA-11522:
---
Is it ok to work on trunk?
> batch_size_fail_threshold_in_kb shouldn't only apply to batch
> -
>
> Key: CASSANDRA-11522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11522
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
>
> I can buy that C* is not good at dealing with large (in bytes) inserts and
> that it makes sense to provide a user configurable protection against inserts
> larger than a certain size, but it doesn't make sense to limit this to
> batches. It's absolutely possible to insert a single very large row and
> internally a batch with a single statement is exactly the same than a single
> similar insert, so rejecting the former and not the later is confusing and
> well, wrong.
> Note that I get that batches are more likely to get big and that's where the
> protection is most often useful, but limiting the option to batch is still
> less useful (it's a hole in the protection) and it's going to confuse users
> in thinking that batches to a single partition are different from single
> inserts.
> Of course that also mean that we should rename that option to
> {{write_size_fail_threshold_in_kb}}. Which means we probably want to add this
> new option and just deprecate {{batch_size_fail_threshold_in_kb}} for now
> (with removal in 4.0).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11546) Stress doesn't respect case-sensitive column names when building insert queries
[ https://issues.apache.org/jira/browse/CASSANDRA-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238926#comment-15238926 ] Giampaolo commented on CASSANDRA-11546: --- Which branch should I use to make the fix? > Stress doesn't respect case-sensitive column names when building insert > queries > --- > > Key: CASSANDRA-11546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11546 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Joel Knighton >Assignee: Giampaolo >Priority: Trivial > Labels: lhf > > When using a custom stress profile, if the schema uses case sensitive column > names, stress doesn't respect case sensitivity when building insert/update > statements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11546) Stress doesn't respect case-sensitive column names when building insert queries
[ https://issues.apache.org/jira/browse/CASSANDRA-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo reassigned CASSANDRA-11546: - Assignee: Giampaolo > Stress doesn't respect case-sensitive column names when building insert > queries > --- > > Key: CASSANDRA-11546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11546 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Joel Knighton >Assignee: Giampaolo >Priority: Trivial > Labels: lhf > > When using a custom stress profile, if the schema uses case sensitive column > names, stress doesn't respect case sensitivity when building insert/update > statements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[ https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-11352: -- Attachment: cassandra-11352-trunk-giampaolo-trapasso@radicalbit-io.patch Patch for 11352 to be applied on trunk > Include units of metrics in the cassandra-stress tool > -- > > Key: CASSANDRA-11352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11352 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Rajath Subramanyam >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > cassandra-11352-trunk-giampaolo-trapasso@radicalbit-io.patch > > > cassandra-stress in the Results section can have units for the metrics as an > improvement to make the tool more usable. > Results: > op rate : 14668 [READ:7334, WRITE:7334] > partition rate: 14668 [READ:7334, WRITE:7334] > row rate : 14668 [READ:7334, WRITE:7334] > latency mean : 0.7 [READ:0.7, WRITE:0.7] > latency median: 0.6 [READ:0.6, WRITE:0.6] > latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8] > latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2] > latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0] > latency max : 448.7 [READ:162.3, WRITE:448.7] > Total partitions : 105612753 [READ:52805915, WRITE:52806838] > Total errors : 0 [READ:0, WRITE:0] > total gc count: 0 > total gc mb : 0 > total gc time (s) : 0 > avg gc time(ms) : NaN > stdev gc time(ms) : 0 > Total operation time : 02:00:00 > END -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[ https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-11352: -- Status: Patch Available (was: Open) > Include units of metrics in the cassandra-stress tool > -- > > Key: CASSANDRA-11352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11352 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Rajath Subramanyam >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > > cassandra-stress in the Results section can have units for the metrics as an > improvement to make the tool more usable. > Results: > op rate : 14668 [READ:7334, WRITE:7334] > partition rate: 14668 [READ:7334, WRITE:7334] > row rate : 14668 [READ:7334, WRITE:7334] > latency mean : 0.7 [READ:0.7, WRITE:0.7] > latency median: 0.6 [READ:0.6, WRITE:0.6] > latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8] > latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2] > latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0] > latency max : 448.7 [READ:162.3, WRITE:448.7] > Total partitions : 105612753 [READ:52805915, WRITE:52806838] > Total errors : 0 [READ:0, WRITE:0] > total gc count: 0 > total gc mb : 0 > total gc time (s) : 0 > avg gc time(ms) : NaN > stdev gc time(ms) : 0 > Total operation time : 02:00:00 > END -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[
https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238020#comment-15238020
]
Giampaolo commented on CASSANDRA-11352:
---
I've prepared a patch for this issue that is available as attachment here and
as branch (rebased and squashed at
[https://github.com/radicalbit/cassandra/tree/CASSANDRA-11352-trunk]). The
output of stress is now like this (keep in mind that in my locale, comma is the
decimal separator while dot is the three digits separator)
{code}
Results:
Op rate : 4.160op/s [WRITE:4.160op/s]
Partition rate: 4.160pk/s [WRITE:4.160pk/s]
Row rate : 4.160row/s [WRITE:4.160row/s]
Latency mean : 0,9ms [WRITE:0,9ms]
Latency median: 0,5ms [WRITE:0,5ms]
Latency 95th percentile : 2,2ms [WRITE:2,2ms]
Latency 99th percentile : 6,5ms [WRITE:6,5ms]
Latency 99.9th percentile : 28,3ms [WRITE:28,3ms]
Latency max :134,3ms [WRITE:134,3ms]
Total partitions :378.374 [WRITE:378.374]
Total errors : 0 [WRITE:0]
Total GC count: 89
Total GC memory : 3,493GiB
Total GC time : 1,2s
Avg GC time : 13,9ms
StdDev GC time: 8,3ms
Total operation time : 00:01:30
{code}
I've tried some other alignment but this seems the best possible compromise to
me.
Some rationals:
* as [https://issues.apache.org/jira/browse/CASSANDRA-9692], I kept units
beside numbers without spaces
* decimal numbers are aligned using decimal sign
* Total GC time is in seconds while other GC measures are in milliseconds. This
could be misleading, I'm waiting reviewer comment on this
* Respect to ticket, I took the liberty of adding three digits separator
About tests: stress tool is in dtests, I've tried to run them but after a whole
workday they weren't finished and besides the "F" letter, no useful output was
produced.
Moreover, during tests I was not able to connect with JMX, the error was:
{{the Connection refused to host: 127.0.0.4; nested exception is:
java.net.ConnectException: Connection refused}}.
How can I fix this for future tests?
> Include units of metrics in the cassandra-stress tool
> --
>
> Key: CASSANDRA-11352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11352
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Rajath Subramanyam
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
>
> cassandra-stress in the Results section can have units for the metrics as an
> improvement to make the tool more usable.
> Results:
> op rate : 14668 [READ:7334, WRITE:7334]
> partition rate: 14668 [READ:7334, WRITE:7334]
> row rate : 14668 [READ:7334, WRITE:7334]
> latency mean : 0.7 [READ:0.7, WRITE:0.7]
> latency median: 0.6 [READ:0.6, WRITE:0.6]
> latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8]
> latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2]
> latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0]
> latency max : 448.7 [READ:162.3, WRITE:448.7]
> Total partitions : 105612753 [READ:52805915, WRITE:52806838]
> Total errors : 0 [READ:0, WRITE:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 02:00:00
> END
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214847#comment-15214847 ] Giampaolo edited comment on CASSANDRA-9692 at 3/28/16 8:46 PM: --- Thanks for the fixes. I learned a lot of thing through the solution of this bug. I'll work hard to improve the quality of future patches. was (Author: giampaolo): Thanks for the fixes. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch, > dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214847#comment-15214847 ] Giampaolo commented on CASSANDRA-9692: -- Thanks for the fixes. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch, > dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[
https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208719#comment-15208719
]
Giampaolo commented on CASSANDRA-9692:
--
I added a patch for {{dtest}}. The strange thing was that I had a negative
compaction ratio like the one below, so I added also the minus sign to the regex
{code}
Compacted (3c496851-f111-11e5-bd49-13f05e23c5c9) 9 sstables to
[/tmp/dtest-8P7lxi/test/node1/data1/keyspace1/standard1-6881b590f11011e5bd4913f05e23c5c9/ma-28-big,]
to level=0. 43.792MiB to 43.868MiB (~100% of original) in 18,517ms =
-4.186KiB/s. 0 total partitions merged to 199,890. Partition merge counts
were {1:199890, }
{code}
To make things work, I added also a little change to the {{ccm}}. It's not
clear to me at the moment if ccm was to change or not. If this is the case, I
will do a PR on github.
You can find patches on github also:
[dtest|https://github.com/radicalbit/cassandra-dtest/commit/8a1017398ab55a4648fcc307a9be0644c02602dd]
and
[ccm|https://github.com/radicalbit/ccm/commit/bb08b6798f3fda39217f2daf710116a84a3ede84]
> Print sensible units for all log messages
> -
>
> Key: CASSANDRA-9692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9692
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
> Attachments:
> Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff,
> Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff,
> Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff,
> ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch,
> dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch
>
>
> Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts
> log analysis. I've introduced some improvements to the bits I touched for
> CASSANDRA-9681, but we should do this across the codebase. It's a small
> investment for a lot of long term clarity in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch, > dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197768#comment-15197768 ] Giampaolo commented on CASSANDRA-9692: -- I'm not familiar with Python, so this could be a good opportunity to learn it. I will take care of the dtests. Do not worry about giving me even the most trivial information, I prefer to get bored than wrong :). As you said, this is my very first ticket and I need to learn many things. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197452#comment-15197452 ] Giampaolo commented on CASSANDRA-9692: -- I agree with you. In the case you need it, I've created a new branch (https://github.com/radicalbit/cassandra/tree/9692-trunk-rev2]) that adds your changes to my patch afted I rebased the trunk. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: cassandra-9692-trunk-giampaolo.trapa...@radicalbit-io.diff > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Comment: was deleted (was: I'm not familiar with Python, so this could be a good opportunity to learn it. I will take care of the dtests. Do not worry about giving me even the most trivial information, I prefer to get bored than wrong :). As you said, this is my very first ticket and I need to learn many things.) > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[ https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo reassigned CASSANDRA-11352: - Assignee: Giampaolo > Include units of metrics in the cassandra-stress tool > -- > > Key: CASSANDRA-11352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11352 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Rajath Subramanyam >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 2.1.0 > > > cassandra-stress in the Results section can have units for the metrics as an > improvement to make the tool more usable. > Results: > op rate : 14668 [READ:7334, WRITE:7334] > partition rate: 14668 [READ:7334, WRITE:7334] > row rate : 14668 [READ:7334, WRITE:7334] > latency mean : 0.7 [READ:0.7, WRITE:0.7] > latency median: 0.6 [READ:0.6, WRITE:0.6] > latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8] > latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2] > latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0] > latency max : 448.7 [READ:162.3, WRITE:448.7] > Total partitions : 105612753 [READ:52805915, WRITE:52806838] > Total errors : 0 [READ:0, WRITE:0] > total gc count: 0 > total gc mb : 0 > total gc time (s) : 0 > avg gc time(ms) : NaN > stdev gc time(ms) : 0 > Total operation time : 02:00:00 > END -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197766#comment-15197766 ] Giampaolo commented on CASSANDRA-9692: -- I'm not familiar with Python, so this could be a good opportunity to learn it. I will take care of the dtests. Do not worry about giving me even the most trivial information, I prefer to get bored than wrong :). As you said, this is my very first ticket and I need to learn many things. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[ https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197737#comment-15197737 ] Giampaolo commented on CASSANDRA-11352: --- Second question. Should the change done on 2.1? Trunk seems quite the same to me. > Include units of metrics in the cassandra-stress tool > -- > > Key: CASSANDRA-11352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11352 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Rajath Subramanyam >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 2.1.0 > > > cassandra-stress in the Results section can have units for the metrics as an > improvement to make the tool more usable. > Results: > op rate : 14668 [READ:7334, WRITE:7334] > partition rate: 14668 [READ:7334, WRITE:7334] > row rate : 14668 [READ:7334, WRITE:7334] > latency mean : 0.7 [READ:0.7, WRITE:0.7] > latency median: 0.6 [READ:0.6, WRITE:0.6] > latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8] > latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2] > latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0] > latency max : 448.7 [READ:162.3, WRITE:448.7] > Total partitions : 105612753 [READ:52805915, WRITE:52806838] > Total errors : 0 [READ:0, WRITE:0] > total gc count: 0 > total gc mb : 0 > total gc time (s) : 0 > avg gc time(ms) : NaN > stdev gc time(ms) : 0 > Total operation time : 02:00:00 > END -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: (was: cassandra-9692-trunk-giampaolo.trapa...@radicalbit-io.diff) > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Comment: was deleted (was: I'm not familiar with Python, so this could be a good opportunity to learn it. I will take care of the dtests. Do not worry about giving me even the most trivial information, I prefer to get bored than wrong :). As you said, this is my very first ticket and I need to learn many things.) > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11352) Include units of metrics in the cassandra-stress tool
[
https://issues.apache.org/jira/browse/CASSANDRA-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197672#comment-15197672
]
Giampaolo commented on CASSANDRA-11352:
---
I would like to take this one. I just need more details.
On latency mean, is the following correct?
{code}
latency mean : 0.7ms [READ:0.7ms, WRITE:0.7ms]
{code}
and in the case that 0.7 are seconds
{code}
latency mean : 0.7s [READ:0.7s, WRITE:0.7s]
{code}
Does
{code}
Total partitions : 105612753 [READ:52805915, WRITE:52806838]
{code}
need a three digit separation?
{code}
Total partitions : 105.612.753 [READ:52.805.915, WRITE:52.806.838]
{code}
> Include units of metrics in the cassandra-stress tool
> --
>
> Key: CASSANDRA-11352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11352
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Rajath Subramanyam
>Priority: Minor
> Labels: lhf
> Fix For: 2.1.0
>
>
> cassandra-stress in the Results section can have units for the metrics as an
> improvement to make the tool more usable.
> Results:
> op rate : 14668 [READ:7334, WRITE:7334]
> partition rate: 14668 [READ:7334, WRITE:7334]
> row rate : 14668 [READ:7334, WRITE:7334]
> latency mean : 0.7 [READ:0.7, WRITE:0.7]
> latency median: 0.6 [READ:0.6, WRITE:0.6]
> latency 95th percentile : 0.8 [READ:0.8, WRITE:0.8]
> latency 99th percentile : 1.2 [READ:1.2, WRITE:1.2]
> latency 99.9th percentile : 8.8 [READ:8.9, WRITE:9.0]
> latency max : 448.7 [READ:162.3, WRITE:448.7]
> Total partitions : 105612753 [READ:52805915, WRITE:52806838]
> Total errors : 0 [READ:0, WRITE:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 02:00:00
> END
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197769#comment-15197769 ] Giampaolo commented on CASSANDRA-9692: -- I'm not familiar with Python, so this could be a good opportunity to learn it. I will take care of the dtests. Do not worry about giving me even the most trivial information, I prefer to get bored than wrong :). As you said, this is my very first ticket and I need to learn many things. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff Revision one of original patch. I've not removed old one since it may be still nice to have. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[
https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195117#comment-15195117
]
Giampaolo commented on CASSANDRA-9692:
--
Hi [~jkoett],
I've rebased the code, fixed the errors (sorry for the compiler ones), applied
your indication and moved {{prettyPrintRateInSeconds}} to {{FBUtilities}} . I
created a new branch
(https://github.com/radicalbit/cassandra/tree/9692-trunk-rev1) and uploaded a
new patch. I hope that now is CI ready.
> Print sensible units for all log messages
> -
>
> Key: CASSANDRA-9692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9692
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
> Attachments:
> Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff
>
>
> Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts
> log analysis. I've introduced some improvements to the bits I touched for
> CASSANDRA-9681, but we should do this across the codebase. It's a small
> investment for a lot of long term clarity in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff The patch as separate file for the records. > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152404#comment-15152404 ] Giampaolo commented on CASSANDRA-9692: -- Maybe these are trivial questions or the answer is left to the contributor, however since I'm new to this project, I prefer to ask > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Minor > Labels: lhf > Fix For: 3.x > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[
https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152402#comment-15152402
]
Giampaolo commented on CASSANDRA-9692:
--
I've finished this task and I'm double checking the code before submit the
patch. To avoid recycles of the patch I have four simple questions.
1. How about changing {{FBUtilities.prettyPrintMemory}} this way? I think that
is size is < 1024, makes sense to see, for example {{512 bytes}}, instead of
{{0.512}} KiB
{code:title=FBUtilities.java}
public static String prettyPrintMemory(long size)
{
if (size >= 1 << 30)
return String.format("%.3fGiB", size / (double) (1 << 30));
if (size >= 1 << 20)
return String.format("%.3fMiB", size / (double) (1 << 20));
if (size >= 1 << 10)
return String.format("%.3fKiB", size / (double) (1 << 10));
return String.format("%d bytes", size);
}
{code}
2. Sometimes, the "bytes" unit of measure is not present in log, like
[this|https://github.com/apache/cassandra/blob/29525005071dd7321c5d444cb7e898a18057ba67/src/java/org/apache/cassandra/db/compaction/writers/SplittingSizeTieredCompactionWriter.java#L98]
or
[this|https://github.com/apache/cassandra/blob/8036f04c08385fadc4c527fca5b3478171ae31dc/src/java/org/apache/cassandra/streaming/StreamReader.java#L102],
is it good to pretty print in these case also?
3. I've changed only logger stuff, however there are a lot of exceptions like
[this|https://github.com/apache/cassandra/blob/7374e9b5ab08c1f1e612bf72293ea14c959b0c3c/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L267].
I think that in this case also, pretty printing would be better, should I
change these also or does it require a separate task?
4. As question 3, but for other cases, like
[this|https://github.com/apache/cassandra/blob/9f322ac600d8cbd8d06427a0b11cea59991ba5ee/src/java/org/apache/cassandra/tools/BulkLoader.java#L252]
Let me know. Thanks in advance.
> Print sensible units for all log messages
> -
>
> Key: CASSANDRA-9692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9692
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
>
> Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts
> log analysis. I've introduced some improvements to the bits I touched for
> CASSANDRA-9681, but we should do this across the codebase. It's a small
> investment for a lot of long term clarity in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149318#comment-15149318 ] Giampaolo commented on CASSANDRA-9692: -- Ok Paulo, is it ok to you to assign me the task? (to this account, not the other that I wrongly used) > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Minor > Labels: lhf > Fix For: 3.x > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148672#comment-15148672 ] Giampaolo commented on CASSANDRA-9692: -- I would like to work on this as my first contribution to the project. I understand that's a long but simple task that could help me to look at all the codebase. Is my evaluation correct? > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Minor > Labels: lhf > Fix For: 3.x > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
