Re: Cassandra Compaction Metrics - CompletedTasks vs TotalCompactionCompleted
If its occurring that often you can monitor nodetool compactionstats to see whats running > On Aug 10, 2018, at 11:35 AM, Dionne Cloudoupoulos > wrote: > > On 2017/10/31 16:56:29, Chris Lohfink wrote: >> The "CompletedTasks" metric is a measure of how many tasks ran on these two >> executors combined. >> The "TotalCompactionsCompleted" metric is a measure of how many compactions >> issued from the compaction manager ran (normal compactions, cache writes, >> scrub, 2i and MVs). So while they may be close, depending on whats >> happening on the system, theres no assurance that they will be within any >> bounds of each other. > >all this is very interesting, but I do not understand why > CompletedTasks grows at the rate of five thousand operations per hour in > my cloud. Have an idea where can I look? kalo dromo > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org >
Removing Extra Spaces and Row counts while using Capture Command
Hi All, I am using Cassandra Capture Command to perform a select query operation to write data from a column family into JSON format file for further processing. I am able to do that successfully, but I am seeing extra spaces and row count values after every few records. please suggest a to get rid of these unusual extra spaces and row count values. Regards, Bharath Kumar B
Cassandra caches
Hi All,
I had allocated 2 GB each for Key, Row, Counter & Chunk cache and performed
below steps. Please note it is test box not others users are connected to it.
Output 1 shows 0 hits and 0 requests - After clean startup of cassandra
Output 2 shows 0 hits and 1 requests - Executed a select query with returns 1
row (Json format)
Output 3 shows 1 hits and 2 requests - Reexecuted the same select query in
step2.
My questions are
1) Is there any way to preload all the rows to row cache without executing the
select statements.
2) Regarding Key cache I only selected col2 i.e value but requests increased by
4 i.e to 102 from 98.
Can any one explain on the above.
cqlsh:> desc table
CREATE TABLE table (
Col1 text PRIMARY KEY,
Col2 text. — (Json format)
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
Output1:
Hostname:/Users/> nodetool info
ID : 9b10a667-b668-44c4-8deb-2e0ad317f287
Gossip active : true
Thrift active : false
Native Transport active: true
Load : 488.54 MiB
Generation No : 1533931780
Uptime (seconds) : 20
Heap Memory (MB) : 572.35 / 12208.00
Off Heap Memory (MB) : 0.20
Data Center: datacenter1
Rack : rack1
Exceptions : 0
Key Cache : entries 26, size 2.3 KiB, capacity 2 GiB, 62 hits, 98
requests, 0.633 recent hit rate, 60 save period in seconds
Row Cache : entries 0, size 0 bytes, capacity 2 GiB, 0 hits, 0
requests, NaN recent hit rate, 60 save period in seconds
Counter Cache : entries 0, size 0 bytes, capacity 2 GiB, 0 hits, 0
requests, NaN recent hit rate, 60 save period in seconds
Chunk Cache: entries 19, size 1.19 MiB, capacity 1.97 GiB, 62
misses, 186 requests, 0.667 recent hit rate, 233.240 microseconds miss latency
Percent Repaired : 0.0%
Token : (invoke with -T/--tokens to see all 256 tokens)
rkommineni-mac.local:/Users/rkommineni >
Select col2 from table where col1=key;
Output2:
Hostname:/Users/> nodetool info
ID : 9b10a667-b668-44c4-8deb-2e0ad317f287
Gossip active : true
Thrift active : false
Native Transport active: true
Load : 488.54 MiB
Generation No : 1533931780
Uptime (seconds) : 68
Heap Memory (MB) : 627.20 / 12208.00
Off Heap Memory (MB) : 0.20
Data Center: datacenter1
Rack : rack1
Exceptions : 0
Key Cache : entries 28, size 2.5 KiB, capacity 2 GiB, 64 hits, 102
requests, 0.627 recent hit rate, 60 save period in seconds
Row Cache : entries 1, size 40.07 KiB, capacity 2 GiB, 0 hits, 1
requests, 0.000 recent hit rate, 60 save period in seconds
Counter Cache : entries 0, size 0 bytes, capacity 2 GiB, 0 hits, 0
requests, NaN recent hit rate, 60 save period in seconds
Chunk Cache: entries 24, size 1.5 MiB, capacity 1.97 GiB, 67
misses, 225 requests, 0.702 recent hit rate, 285.506 microseconds miss latency
Percent Repaired : 0.0%
Token : (invoke with -T/--tokens to see all 256 tokens)
Select col2 from table where col1=key; - Reran the same query
Output3:
Hostname:/Users/> nodetool info
ID : 9b10a667-b668-44c4-8deb-2e0ad317f287
Gossip active : true
Thrift active : false
Native Transport active: true
Load : 488.54 MiB
Generation No : 1533931780
Uptime (seconds) : 78
Heap Memory (MB) : 651.93 / 12208.00
Off Heap Memory (MB) : 0.20
Data Center: datacenter1
Rack : rack1
Exceptions : 0
Key Cache : entries 28, size 2.5 KiB, capacity 2 GiB, 64 hits, 102
requests, 0.627 recent hit rate, 60 save period in seconds
Row Cache : entries 1, size 40.07 KiB, capacity 2 GiB, 1 hits, 2
requests, 0.500 recent hit rate, 60 save period in seconds
Counter Cache : entries 0, size 0 bytes, capacity 2 GiB, 0 hits, 0
requests, NaN recent hit rate, 60 save period in seconds
Chunk Cache: entries 24, size 1.5 MiB, capacity 1.97 GiB, 67
misses, 225 requests, 0.702 recent hit rate, 208.327 microseconds miss latency
Percent Repaired : 0.0%
Token : (invoke with -T/--tokens to see
upgrade 2.1 to 3.0
Hello, We recently upgrade C* from 2.1 to 3.0. After the upgrade we are seeing increase in the total read bytes and read ops on the EBS volumes. It almost doubled on all the nodes. The number of writes are same. Thank you.
Re: Cassandra Compaction Metrics - CompletedTasks vs TotalCompactionCompleted
On 2017/10/31 16:56:29, Chris Lohfink wrote: > The "CompletedTasks" metric is a measure of how many tasks ran on these two > executors combined. > The "TotalCompactionsCompleted" metric is a measure of how many compactions > issued from the compaction manager ran (normal compactions, cache writes, > scrub, 2i and MVs). So while they may be close, depending on whats > happening on the system, theres no assurance that they will be within any > bounds of each other. all this is very interesting, but I do not understand why CompletedTasks grows at the rate of five thousand operations per hour in my cloud. Have an idea where can I look? kalo dromo - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Extending Cassandra on AWS from single Region to Multi-Region
Hey All, Any info on this topic.? Thank You, Regards, Srini On Wed, Aug 8, 2018, 9:46 PM srinivasarao daruna wrote: > Hi All, > > We have built Cassandra on AWS EC2 instances. Initially when creating > cluster we have not considered multi-region deployment and we have used AWS > EC2Snitch. > > We have used EBS Volumes to save our data and each of those disks were > filled around 350G. > We want to extend it to Multi Region and wanted to know the better > approach and recommendations to achieve this process. > > I agree that we have made a mistake by not using EC2MultiRegionSnitch, but > its past now and if anyone faced or implemented similar thing i would like > to get some guidance. > > Any help would be very much appreciated. > > Thank You, > Regards, > Srini >
Re: difference between CompletedTasks and TotalComactionCompleted
Hello, There has been a similar thread a few months back. https://lists.apache.org/thread.html/2cad65aba4fb035378591ef007ee68ccaf23a2c025aaaf7e7512d364@%3Cuser.cassandra.apache.org%3E On fre, 2018-08-10 at 14:18 +0200, Dionne Cloudoupoulos wrote: Hello! I'm trying to monitor cassandra through jmx and it turns out that I find two values that seem to be the same thing but not: org.apache.cassandra.metrics: type = Compaction, name = CompletedTasks.Value org.apache.cassandra.metrics: type = Compaction, name = TotalCompactionsCompleted.Count and I do not know what to use. I began to graph both and the second one remains stable, but the first increases from five thousand per hour in a clulster without traffic. So the questions are: which one should I graph? and why does the first increase if there is no traffic? Excuse me if you do not understand, I'm using a translator. kalo dromo, Dionne
difference between CompletedTasks and TotalComactionCompleted
Hello! I'm trying to monitor cassandra through jmx and it turns out that I find two values that seem to be the same thing but not: org.apache.cassandra.metrics: type = Compaction, name = CompletedTasks.Value org.apache.cassandra.metrics: type = Compaction, name = TotalCompactionsCompleted.Count and I do not know what to use. I began to graph both and the second one remains stable, but the first increases from five thousand per hour in a clulster without traffic. So the questions are: which one should I graph? and why does the first increase if there is no traffic? Excuse me if you do not understand, I'm using a translator. kalo dromo, Dionne
Re: CQL data type compatibility between ascii and text
Thanks Yoshi! That explains it a lot :) On Fri, 10 Aug 2018 18:30:25 +1000 Y K wrote Hi Thira, First, there's the 3.0 branch of versions and the 3.x branch of versions. http://cassandra.apache.org/doc/latest/development/patches.html#choosing-the-right-branches-to-work-on The 3.0.16 belongs to the 3.0 branch. The 3.9 and 3.11.2 belong to the 3.x. branch I believe the change was made by this: Remove alter type support https://issues.apache.org/jira/browse/CASSANDRA-12443 where it was "Fixed" in versions 3.0.11 in the 3.0 branch and in version 3.10 in 3.x branch. So 3.0.16 has the fix, 3.9 doesn't have it, but 3.11.2 has it. Best regards, Yoshi 2018年8月10日(金) 17:10 thiranjith : Hi, According to documentation at https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cql_data_types_c.html#cql_data_types_c__cql_data_type_compatibility we should not be able to change the column type from ascii to text. I have had a mix experience with conversion between data types on different versions of Cassandra. For example, given the following table definition: CREATE TABLE changelog ( sequence int, description ascii, createdby ascii, executedon timestamp, PRIMARY KEY (sequence, description) ) Attempting change the data type for column 'createdby' with following CQL alter table changelog alter createdby TYPE text; gives the behaviour outlined below depending on the version of Cassandra: With [cqlsh 5.0.1 | Cassandra 3.0.16 | CQL spec 3.4.0 | Native protocol v4] InvalidRequest: Error from server: code=2200 [Invalid query] message="Altering of types is not allowed" (Expected, per documentation) With [cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4] Query succeeds and change the column type to 'text' (as verified by running describe changelog and also inserting data with non-ascii chars into the column) With Cassandra 3.11.2 InvalidRequest: Error from server: code=2200 [Invalid query] message="Altering of types is not allowed" (Expected, per documentation) Can anyone please explain why it works on 3.9 and not on others? Thanks! Thira
Re: CQL data type compatibility between ascii and text
Hi Thira, First, there's the 3.0 branch of versions and the 3.x branch of versions. http://cassandra.apache.org/doc/latest/development/patches.html#choosing-the-right-branches-to-work-on The 3.0.16 belongs to the 3.0 branch. The 3.9 and 3.11.2 belong to the 3.x. branch I believe the change was made by this: Remove alter type support https://issues.apache.org/jira/browse/CASSANDRA-12443 where it was "Fixed" in versions 3.0.11 in the 3.0 branch and in version 3.10 in 3.x branch. So 3.0.16 has the fix, 3.9 doesn't have it, but 3.11.2 has it. Best regards, Yoshi 2018年8月10日(金) 17:10 thiranjith : > Hi, > > According to documentation at > https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cql_data_types_c.html#cql_data_types_c__cql_data_type_compatibility > we > should not be able to change the column type from ascii to text. > > I have had a mix experience with conversion between data types on > different versions of Cassandra. > > For example, given the following table definition: > > > *CREATE TABLE changelog (* > > *sequence int, * > > *description ascii,* > > *createdby ascii,* > > *executedon timestamp,* > > *PRIMARY KEY (sequence, description)* > *)* > > Attempting change the data type for column 'createdby' with following CQL > *alter table changelog alter createdby TYPE text;* > > gives the behaviour outlined below depending on the version of Cassandra: > > >- With [cqlsh 5.0.1 | Cassandra 3.0.16 | CQL spec 3.4.0 | Native >protocol v4] >- InvalidRequest: Error from server: code=2200 [Invalid query] > message="Altering of types is not allowed" (Expected, per documentation) > - With [cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native >protocol v4] >- Query succeeds and change the column type to 'text' (as verified by > running describe changelog and also inserting data with non-ascii chars > into the column) > - With Cassandra 3.11.2 >- InvalidRequest: Error from server: code=2200 [Invalid query] > message="Altering of types is not allowed" (Expected, per documentation) > > Can anyone please explain why it works on 3.9 and not on others? > > Thanks! > Thira > > >
CQL data type compatibility between ascii and text
Hi, According to documentation at https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cql_data_types_c.html#cql_data_types_c__cql_data_type_compatibility we should not be able to change the column type from ascii to text. I have had a mix experience with conversion between data types on different versions of Cassandra. For example, given the following table definition: CREATE TABLE changelog ( sequence int, description ascii, createdby ascii, executedon timestamp, PRIMARY KEY (sequence, description) ) Attempting change the data type for column 'createdby' with following CQL alter table changelog alter createdby TYPE text; gives the behaviour outlined below depending on the version of Cassandra: With [cqlsh 5.0.1 | Cassandra 3.0.16 | CQL spec 3.4.0 | Native protocol v4] InvalidRequest: Error from server: code=2200 [Invalid query] message="Altering of types is not allowed" (Expected, per documentation) With [cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4] Query succeeds and change the column type to 'text' (as verified by running describe changelog and also inserting data with non-ascii chars into the column) With Cassandra 3.11.2 InvalidRequest: Error from server: code=2200 [Invalid query] message="Altering of types is not allowed" (Expected, per documentation) Can anyone please explain why it works on 3.9 and not on others? Thanks! Thira
Re: Huge daily outbound network traffic
I don't have any external process or planed repair in that time period. In case of network, I can see outbound network on Cassandra node network interface but couldn't find any way to check the VPC network to make sure it is not going out of network. Maybe the only way is analysing VPC Flow Log. B. On Tue, Aug 7, 2018 at 11:23 PM, Rahul Singh wrote: > Are you sure you don’t have an outside process that is doing an export , > Spark job, non AWS managed backup process ? > > Is this network out from Cassandra or from the network? > > > Rahul > On Aug 7, 2018, 4:09 AM -0400, Behnam B.Marandi , wrote: > > Hi, > I have a 3 node Cassandra cluster (version 3.11.1) on m4.xlarge EC2 > instances with separate EBS volumes for root (gp2), data (gp2) and > commitlog (io1). > I get daily outbound traffic at a certain time everyday. As you can see in > the attached screenshot, whiile my normal networkl oad hardly meets 200MB, > this outbound (orange) spikes up to 2GB while inbound (purple) is less than > 800MB. > There is no repair or backup process giong on in that time window, so I am > wondering where to look. Any idea? > > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > >
