Hi,
I have a pretty urgent issue with 1.0.9 cluster
in opscenter i saw a compation that had a progress of 0% for a long
time, looking at the cassandra log on the relevant node I see REPEATED
messages of IllegalArgumentException in CompactionExecutor
INFO [CompactionExecutor:3335]
Looking at http://blog.mikiobraun.de/2010/08/cassandra-gc-tuning.html
and server logs, I think my situation is this
The default cassandra settings has the highest peak heap usage. The
problem with this is that it raises the possibility that during the
CMS cycle, a collection of the young
On Thu, Jun 14, 2012 at 8:26 AM, Piavlo lolitus...@gmail.com wrote:
I started looking for similar messages on other nodes saw a SINGLE
IllegalArgumentException on
ValidationExecutor on the same node and 2 other nodes (this is a 6 node
cluster) which happened
at almost the same time , in all
Le 13 juin 2012 à 20:52, aaron morton a écrit :
You meant -Dcassandra.load_ring_state=false right ?
yes, sorry.
Maybe I could open a jira about my issue ? Maybe there was a config mess on
my part at some point, ie the unsynchronized date on my machines, but I
think it would be nice if
Hi All,
Help needed on the following front.
In my Cassandra node logs I can see the following error:
CustomTThreadPoolServer.java (line 201) Thrift error occurred during processing
of message.
org.apache.thrift.protocol.TProtocolException: Missing version in
readMessageBegin, old client?
Hector : 1.0.0.1
Cassandra: 1.0.3
From: Tiwari, Dushyant (ISGT)
Sent: Thursday, June 14, 2012 2:16 PM
To: user@cassandra.apache.org
Subject: Cassandra error while processing message
Hi All,
Help needed on the following front.
In my Cassandra node logs I can see the following error:
Hi !
What is meaning of
chunk_length_kb: sets the compression chunk size in kilobytes.
?
It means that uncompressed sstable data is compressed to approximately
chunk_length_kb and every read needs to read approximately chunk_length_kb and
decompress it to read any value from compressed range
Hi Sylvain,
Yes this UserCompletions CF uses composite comparator and I do use
sstable compression.
What's the procedure to check if the compressed sstable is corrupted or not?
If it's corrupted what can I do to fix the issue with minimal cluster
load impact?
Is there way to delete all
I have a columnfamily like:
CREATE COLUMN FAMILY Watchdog
WITH key_validation_class =
'CompositeType(LexicalUUIDType,LexicalUUIDType)'
AND comparator = UTF8Type
AND column_metadata = [
{column_name: error_code, validation_class: UTF8Type, index_type:
KEYS}
That does looks fishy.
Would you mind opening a ticket on jira (
https://issues.apache.org/jira/browse/CASSANDRA) directly for that. It's
easier for us to track it there.
Thanks,
Sylvain
On Wed, Jun 13, 2012 at 8:05 PM, Ganza, Ivan iga...@globeandmail.comwrote:
Greetings,
** **
We have
It means that uncompressed sstable data is compressed to approximately
chunk_length_kb and every read needs to read approximately chunk_length_kb and
decompress it to read any value from compressed range ?
Or it means approximately chunk_length_kb of sstable data is compressed and
stored on
INFO [ScheduledTasks:1] 2012-06-14 07:49:54,355 MessagingService.java (line
615) 15 MUTATION message dropped in last 5000ms
It is at INFO level so I'm inclined to think not but is seems like whenever
messages are dropped there may be some issue?
On Thu, Jun 14, 2012 at 12:00 PM, Piavlo lolitus...@gmail.com wrote:
What's the procedure to check if the compressed sstable is corrupted or not?
Since you use compression, in theory that can't be disk bitrot since
in that case
you would have got some checksum error instead. The fact that it
Hi again,
After some further investigation now I'm in a situation that there are
3 nodes (of 6 nodes cluster) and all of them are falling with
ValidationExecutor during compaction which is is trigger by
repair -pr PRODUCTION UserCompletions against any node in the cluster
- repair get's
Is there way to make cassandra throw away the offending column?
Running scrub should allow to get read of the row containing the
problematic column. Unfortunately it will discard the whole row, not
just the column.
However, since scrub takes a snapshot anyway (and should tell you
which sstable
Greetings,
Thank you - issue is created here:
https://issues.apache.org/jira/browse/CASSANDRA-4340
-Ivan/
---
[cid:image001.jpg@01CD4A16.2AA22DE0]
Ivan Ganza | Senior Developer | Information Technology
c: 647.701.6084 | e: iga...@globeandmail.com
From: Sylvain Lebresne
I have a columnfamily like:
CREATE COLUMN FAMILY Watchdog
WITH key_validation_class =
'CompositeType(LexicalUUIDType,LexicalUUIDType)'
AND comparator = UTF8Type
AND column_metadata = [
{column_name: error_code, validation_class: UTF8Type, index_type:
KEYS}
Hi everyone,
We have problem with our Cassandra cluster, and that is that sometimes it
takes several seconds to open a new Thrift connection to the server. We've
had this issue when we ran on windows, and we have this issue now that we
run on Ubuntu. We've had it with our old networking setup,
On 2012-06-14, at 10:38 AM, Henrik Schröder wrote:
Hi everyone,
We have problem with our Cassandra cluster, and that is that sometimes it
takes several seconds to open a new Thrift connection to the server. We've
had this issue when we ran on windows, and we have this issue now that we
Ok i've run scrub on the 3 nodes and the problematic row
Error validating row
DecoratedKey(92668395684826132216160944211592988451,
353339332d3134363533393931)
The full message is
WARN [CompactionExecutor:2700] 2012-06-14 14:26:42,041
CompactionManager.java (line 582) Non-fatal error
Derek,
Thanks for that!
Yes, I am aware of that technique. I am currently using something
very similar on an sql database. I think one of the great benefits with
Cassandra is that you can invent these on the fly. I also think there
is great benefit to keep all of the columns in the same row.
Hi Mina,
The delay is not constant, in the absolute majority of cases, connecting is
almost instant, but occasionally, connecting to a server takes a few
seconds.
We can't even reproduce it reliably, we can see in our server logs that
sometimes, maybe a few times a day, maybe once every few
Hello.
For sure. Here they are:
http://www.slideshare.net/vittim1/practical-cassandra
Slides are in english.
I've presented this presentation some time ago at JEEConf and once more
yesterday in local developers club.
There should be video recording (russian) available somewhen, but it's
not
As a random guess, you might want to check your open file descriptor limit
on the C* servers. Use cat /proc/pid/limits, where pid is the pid of
the Cassandra process; it's the most reliable way to check this.
On Thu, Jun 14, 2012 at 10:43 AM, Henrik Schröder skro...@gmail.com wrote:
Hi Mina,
I have playing around with composite CFs, I have one declared:
create columnfamily
at_event_ac_c
(
ac_event_id int,
ac_creation timestamp,
ac_action text,
ac_addr text,
ac_advisory_id text,
ac_c text,
...
ev_sev text,
...
ev_total text,
ev_url text,
I was able to figure out that 353339332d3134363533393931 is the row key
while no idea what is 92668395684826132216160944211592988451 part?
sstable2json also fails with validation error on this row key
now since I have lost data for this row - how do I find out that was
the root cause?
JNA is installed. swappiness was 0. vfs_cache_pressure was 100. 2 questions
on this..
1. Is there a way to find out if mlockall really worked other than just the
mlockall successful log message?
2. Does cassandra only mlock the jvm heap or also the mmaped memory?
I disabled mmap completely, and
Hi,
Wonder why this cap is in place? We are experimenting on some CF names with
UUIDs and hit this issue.
--
Regards,
Tharindu
blog: http://mackiemathew.com/
One of the column names on the row with key 353339332d3134363533393931
failed to validate with the validator for the column.
If you really are after what column is problematic, and are able to
build and run cassandra, you can add debugging info to Column.java
protected void
Upgrade java (version 1.6.21 have memleaks) to latest 1.6.32. Its
abnormally that on 80Gigs you have 15Gigs of index
vfs_cache_pressure - used for inodes and dentrys
Also to check that you have memleaks use drop_cache sysctl
2012/6/14 Gurpreet Singh gurpreet.si...@gmail.com:
JNA is
2012/6/14 Gurpreet Singh gurpreet.si...@gmail.com:
JNA is installed. swappiness was 0. vfs_cache_pressure was 100. 2 questions
on this..
1. Is there a way to find out if mlockall really worked other than just the
mlockall successful log message?
yes you must see something like this (from our
Soory i mistaken,here is right string
INFO [main] 2012-06-14 02:03:14,520 CLibrary.java (line 109) JNA
mlockall successful
2012/6/15 ruslan usifov ruslan.usi...@gmail.com:
2012/6/14 Gurpreet Singh gurpreet.si...@gmail.com:
JNA is installed. swappiness was 0. vfs_cache_pressure was 100. 2
32 matches
Mail list logo
