[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548079#comment-16548079
 ] 

ASF GitHub Bot commented on CASSANDRA-14457:


Github user clohfink closed the pull request at:

https://github.com/apache/cassandra/pull/226


> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506543#comment-16506543
 ] 

Jeff Jirsa commented on CASSANDRA-14457:


[~iamaleksey]  Re: {{CompactionMetrics}} - [~krummas] recently noted in passing 
that most of compaction has been slowly evolving over time and could probably 
use a nice, thorough, ground-up rewrite in the near future. May be worth a chat 
on the dev@ list about a potential redesign.


> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501794#comment-16501794
 ] 

Aleksey Yeschenko commented on CASSANDRA-14457:
---

Committed to trunk as 
[899f7c41935d92f15bc17a33f36443030616c8eb|https://github.com/apache/cassandra/commit/899f7c41935d92f15bc17a33f36443030616c8eb],
 thanks.

Made some minor tweaks on commit:
- Use the correct {{Optional.ofNullable()}} rather than {{Optional.of()}} in 
{{getKeyspace()}} and {{getTable()}}
- Fix {{asMap()}} to properly handle empty {{Optional}}s so it doesn't throw
- Made nodetool {{CompactionStats}} use {{CompactionInfo}} constants that we 
have now to access the map
- Hid dealing with {{Holder}} in {{CompactionMetrics}}

In an unrelated note: why, why on Earth is {{CompactionMetrics}} the place that 
is the authorative source of all running compactions? And all that stored in a 
static synchronised identity set? Has nobody been bothered by this since 2012?

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500434#comment-16500434
 ] 

Chris Lohfink commented on CASSANDRA-14457:
---

minor note: renamed compaction_id to task_id since not calling them compactions 
anymore

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500429#comment-16500429
 ] 

Chris Lohfink commented on CASSANDRA-14457:
---

Latest looks something like:

{code}
cqlsh> desc table system_views.sstable_tasks;

CREATE TABLE system_views.sstable_tasks (
compaction_id uuid,
keyspace_name text,
kind text,
progress bigint,
table_name text,
total bigint,
unit text,
PRIMARY KEY (keyspace_name, table_name, compaction_id)
) WITH CLUSTERING ORDER BY (table_name ASC, compaction_id ASC)
AND compaction = {'class': 'None'}
AND compression = {};

cqlsh> SELECT * FROM system_views.sstable_tasks;

 keyspace_name | table_name | compaction_id| kind   
| progress | total| unit
---++--++--+--+---
 basic |  wide3 | 066ba210-6811-11e8-ade8-f5df16641a9d | compaction 
|  2266347 | 33091208 | bytes

(1 rows)
{code}

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500102#comment-16500102
 ] 

Sam Tunnicliffe commented on CASSANDRA-14457:
-

Nice, this looks really good. [~cnlwsu] if you don't mind me chipping in, I 
have a couple of comments on naming.


Could we avoid the requirement to quote some column names by adopting the 
pattern used in system tables? {{keyspace_name/table_name}} would remove the 
need to quote and be consistent with existing tables.


Using {{undefined}} for tasks without a specific keyspace/table won't really 
fly as that's a perfectly valid identifier. For tasks without a specific target 
how about using values which are totally illegal for actual keyspace/table 
names like {{"all keyspaces"}} and {{"all tables"}}?


I understand that the naming of the non-primary key columns is designed to make 
the layout in cqlsh user friendly, but I think we could achieve the same thing 
with a bit more meaningful naming ({{progess_total}} for example, is a bit 
unintuitive). OTOMH, a set of column names like {{kind, progress, size, unit}} 
would produce a usable display and communicate what the columns represent a bit 
better.


Lastly, I'm a bit bothered by this being named the {{compactions}} table and 
described as a "{{List of current compactions}}" when there's a whole bunch of 
operations that are going to show up in this table which have little to do with 
compaction, except for the fact that they may be utilising some resources from 
the compaction pool. I'm not suggesting that they shouldn't be included here, 
they definitely should be if this table is to be used in the same way as 
nodetool compactionstats is currently, but seeing as this both user facing and 
something of a blank canvas I think we should try to get this sort of thing as 
right as we can up front.  My vote would be to rename this to something like 
{{sstable_tasks}} and open a separate jira to create an actual {{compactions}} 
table which would "contain" a strict subset of the data exposed here.


> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500052#comment-16500052
 ] 

Aleksey Yeschenko commented on CASSANDRA-14457:
---

Just noticed that we are doing some unnecessary conversions, followed by 
reversals here. Which isn't a big deal, but feels a little bit silly: 
{{CompactionManager.getCompactions()}} conversts {{CompactionInfo}} objects to 
maps of strings to strings, then we convert those strings back to proper types 
in {{CompactionsTable}}.

What would be better is if {{CompactionManager}} had a method that would return 
a collection or just an iterable of {{CompactionInfo}} objects that we could 
work with directly, without a redundant ser-deser cycle.

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486765#comment-16486765
 ] 

Chris Lohfink commented on CASSANDRA-14457:
---

updated in pull request and [unit 
tests|https://circleci.com/gh/clohfink/cassandra/150]

{code}
cqlsh> desc table system_views.compactions;

CREATE TABLE system_views.compactions (
compaction_id uuid,
"keyspace" text,
progress_current bigint,
progress_total bigint,
progress_unit text,
"table" text,
task_type text,
PRIMARY KEY ("keyspace", "table", compaction_id)
) WITH CLUSTERING ORDER BY ("table" ASC, compaction_id ASC)

cqlsh> select * from system_views.compactions;

 keyspace | table | compaction_id| progress_current | 
progress_total | progress_unit | task_type
--+---+--+--++---+
basic | wide3 | 644e4790-5e4b-11e8-ba7a-7f580808cfbc | 22913336 |   
79669692 | bytes | compaction
basic | wide3 | 67281b80-5e4b-11e8-ba7a-7f580808cfbc |  3819930 |   
 4639969 | bytes | compaction
{code}

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483823#comment-16483823
 ] 

Benjamin Lerer commented on CASSANDRA-14457:


I am also in favor of {{(keyspace, table, id)}} as it makes querying easier.

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482546#comment-16482546
 ] 

Aleksey Yeschenko commented on CASSANDRA-14457:
---

As for 3/5, I’m thinking (keyspace, table, id) - so that you can do a SELECT by 
keyspace, without the table or ALLOW FILTERING. They’d still be equally close 
together in cqlsh.

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482541#comment-16482541
 ] 

Aleksey Yeschenko commented on CASSANDRA-14457:
---

bq. Can we just use "undefined" for summary redistribution with changing it to 
be part of key?

We could use a sentinel like that, so long as it's something that isn't a legal 
keyspace/table name. Think 'all keyspaces' and 'all tables', with a space 
in-between. But I'm not sure we should even list it there, or that it should 
have ever been a compaction type in the first place.

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482532#comment-16482532
 ] 

Chris Lohfink commented on CASSANDRA-14457:
---

3/5: ((keyspace, table), id) would solve the issue I concatenated the 
keyspace/table together for (columns listed alphabetically in cqlsh so having 
them on opposite sides of row was hard to read. So I definitely will go with 
that.

Can we just use "undefined" for summary redistribution with changing it to be 
part of key?

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482413#comment-16482413
 ] 

Aleksey Yeschenko commented on CASSANDRA-14457:
---

This looks good, so I only have some bikeshedding to contribute:

1. The table doesn't really represent compaction statistics, so should probably 
not name it compaction_stats? I know that the nodetool cmd is named 
compactionstats, but that was a bad name too, imo. So perhaps 
{{compaction_state}} and {{CompactionStateTable}}, or {{compcation_status}} and 
{{CompactionStatusTable}}, or even {{active_compactions}} and 
{{ActiveCompactionsTable}}; whichever sounds nicer to your American ear.
2. Should we perhaps use {{int}} as the type for current and total columns, 
instead of {{text}}?
3. Maybe don't concat the names of the keyspace and the table? Why not have 
them in separate columns, for easier querying?
4. We tend to lowercase enums in table, usually. Can you slap a 
{{toLowerCase()}} on {{task_type}} please?
5. I would prefer to have ((keyspace, table), id) or ((keyspace), table, id) as 
PRIMARY KEY here, personally.

Upon a quick look, it seems like the only case where we don't have a 
keyspace/table attached to a compaction is summary redistribution, which is 
performed on all sstables. But it's not really a compaction, so perhaps we 
should exclude it from the dataset?

> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14457) Add a virtual table with current compactions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481486#comment-16481486
 ] 

ASF GitHub Bot commented on CASSANDRA-14457:


GitHub user clohfink opened a pull request:

https://github.com/apache/cassandra/pull/226

Add compactions virtual table for CASSANDRA-14457



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/clohfink/cassandra compaction_stats

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra/pull/226.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #226


commit cfd54fb802c15609c5eedf715d57cb173c2f28cb
Author: Chris Lohfink 
Date:   2018-05-19T06:27:28Z

Add compactions virtual table for CASSANDRA-14457




> Add a virtual table with current compactions
> 
>
> Key: CASSANDRA-14457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org