[jira] [Commented] (CASSANDRA-13937) Cassandra node's startup time increased after increase count of big tables

2017-10-10 Thread Andrey Lataev (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198850#comment-16198850
 ] 

Andrey Lataev commented on CASSANDRA-13937:
---


I try to replace LCS on STCS but without significant result.

>  Cassandra node's startup time increased after increase count of big tables
> ---
>
> Key: CASSANDRA-13937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13937
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: RHEL 7.3
> JDK HotSpot 1.8.0_121-b13
> cassandra-3.11 cluster with 43 nodes in 9 datacenters
> 8vCPU, 100 GB RAM
>Reporter: Andrey Lataev
> Attachments: cassandra.zip, debug.zip
>
>
> In startup time Cassandra spends a long time on read some big Columnfamilies.
> For example, in debug.log:
> {code:java}
> grep SSTableReader.java:506 /var/log/cassandra/debug.log
> <...> 
> DEBUG [SSTableBatchOpen:3] 2017-10-04 22:40:05,297 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171003-b341cc709c7511e7b1cfed1e90eb03dc/mc-45242-big
>  (19.280MiB)
> DEBUG [SSTableBatchOpen:5] 2017-10-04 22:42:14,188 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-49002-big
>  (10.607MiB)
> <...>
> DEBUG [SSTableBatchOpen:4] 2017-10-04 22:42:19,792 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-47907-big
>  (128.172MiB)
> DEBUG [SSTableBatchOpen:1] 2017-10-04 22:44:23,560 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/pk4smevauditbody/messagelogbody20170324-f918bfa0107b11e7adfc2d0b45a372ac/mc-4-big
>  (96.310MiB)
> <..>
> {code}
> SSTableReader.java:506 spent ~ 2 min per every big table in p00smevauditbody 
> keyspace.
> I was planned too keep similar tables for the full month...
> So it seems like Cassandra will need more then 1h on startup...
> Does it available to speed up SSTableBatchOpen ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13937) Cassandra node's startup time increased after increase count of big tables

2017-10-04 Thread Andrey Lataev (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192062#comment-16192062
 ] 

Andrey Lataev commented on CASSANDRA-13937:
---

 LZ4 compression used:
{code:java}
sys@cqlsh> DESC TABLE p00smevauditbody.messagelogbody20171004;

CREATE TABLE p00smevauditbody.messagelogbody20171004 (
d timestamp,
mid text,
mt text,
rec_msg text,
rec_sid text,
sd_msg text,
PRIMARY KEY (d, mid, mt)
) WITH CLUSTERING ORDER BY (mid ASC, mt ASC)
AND bloom_filter_fp_chance = 0.1
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 
'sstable_size_in_mb': '160'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}



>  Cassandra node's startup time increased after increase count of big tables
> ---
>
> Key: CASSANDRA-13937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13937
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: RHEL 7.3
> JDK HotSpot 1.8.0_121-b13
> cassandra-3.11 cluster with 43 nodes in 9 datacenters
> 8vCPU, 100 GB RAM
>Reporter: Andrey Lataev
> Attachments: cassandra.zip, debug.zip
>
>
> In startup time Cassandra spends a long time on read some big Columnfamilies.
> For example, in debug.log:
> {code:java}
> grep SSTableReader.java:506 /var/log/cassandra/debug.log
> <...> 
> DEBUG [SSTableBatchOpen:3] 2017-10-04 22:40:05,297 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171003-b341cc709c7511e7b1cfed1e90eb03dc/mc-45242-big
>  (19.280MiB)
> DEBUG [SSTableBatchOpen:5] 2017-10-04 22:42:14,188 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-49002-big
>  (10.607MiB)
> <...>
> DEBUG [SSTableBatchOpen:4] 2017-10-04 22:42:19,792 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-47907-big
>  (128.172MiB)
> DEBUG [SSTableBatchOpen:1] 2017-10-04 22:44:23,560 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/pk4smevauditbody/messagelogbody20170324-f918bfa0107b11e7adfc2d0b45a372ac/mc-4-big
>  (96.310MiB)
> <..>
> {code}
> SSTableReader.java:506 spent ~ 2 min per every big table in p00smevauditbody 
> keyspace.
> I was planned too keep similar tables for the full month...
> So it seems like Cassandra will need more then 1h on startup...
> Does it available to speed up SSTableBatchOpen ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13937) Cassandra node's startup time increased after increase count of big tables

2017-10-04 Thread Andrey Lataev (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192063#comment-16192063
 ] 

Andrey Lataev commented on CASSANDRA-13937:
---

 LZ4 compression used:
{code:java}
sys@cqlsh> DESC TABLE p00smevauditbody.messagelogbody20171004;

CREATE TABLE p00smevauditbody.messagelogbody20171004 (
d timestamp,
mid text,
mt text,
rec_msg text,
rec_sid text,
sd_msg text,
PRIMARY KEY (d, mid, mt)
) WITH CLUSTERING ORDER BY (mid ASC, mt ASC)
AND bloom_filter_fp_chance = 0.1
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 
'sstable_size_in_mb': '160'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}



>  Cassandra node's startup time increased after increase count of big tables
> ---
>
> Key: CASSANDRA-13937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13937
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: RHEL 7.3
> JDK HotSpot 1.8.0_121-b13
> cassandra-3.11 cluster with 43 nodes in 9 datacenters
> 8vCPU, 100 GB RAM
>Reporter: Andrey Lataev
> Attachments: cassandra.zip, debug.zip
>
>
> In startup time Cassandra spends a long time on read some big Columnfamilies.
> For example, in debug.log:
> {code:java}
> grep SSTableReader.java:506 /var/log/cassandra/debug.log
> <...> 
> DEBUG [SSTableBatchOpen:3] 2017-10-04 22:40:05,297 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171003-b341cc709c7511e7b1cfed1e90eb03dc/mc-45242-big
>  (19.280MiB)
> DEBUG [SSTableBatchOpen:5] 2017-10-04 22:42:14,188 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-49002-big
>  (10.607MiB)
> <...>
> DEBUG [SSTableBatchOpen:4] 2017-10-04 22:42:19,792 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/p00smevauditbody/messagelogbody20171004-f82225509d3e11e7b1cfed1e90eb03dc/mc-47907-big
>  (128.172MiB)
> DEBUG [SSTableBatchOpen:1] 2017-10-04 22:44:23,560 SSTableReader.java:506 - 
> Opening 
> /egov/data/cassandra/datafiles1/pk4smevauditbody/messagelogbody20170324-f918bfa0107b11e7adfc2d0b45a372ac/mc-4-big
>  (96.310MiB)
> <..>
> {code}
> SSTableReader.java:506 spent ~ 2 min per every big table in p00smevauditbody 
> keyspace.
> I was planned too keep similar tables for the full month...
> So it seems like Cassandra will need more then 1h on startup...
> Does it available to speed up SSTableBatchOpen ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org