date:20111223


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175581#comment-13175581
 ] 

Eric Evans commented on CASSANDRA-3634:
---

v1-0001-CASSANDRA-3634-generated-thrift-code.txt and 
v1-0002-change-bind-parms-from-string-to-bytes.txt convert string bind params 
to binary for purposes of performance testing.

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2011-12-23 Thread Eric Evans (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans updated CASSANDRA-3634:
--

Attachment: stress-change-bind-parms-to-BB.patch

stress-change-bind-parms-to-BB.patch updates stress to use binary query 
parameters for prepared statements.

This patch only updates the operations used in testing, (it would need more 
work before committing).

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: stress-change-bind-parms-to-BB.patch, 
 v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175588#comment-13175588
 ] 

Eric Evans commented on CASSANDRA-3634:
---

Here is the performance comparison.  I stuck to the same tests I performed 
earlier (those earlier results can be found  
[here|http://www.acunu.com/blogs/eric-evans/cql-benchmarking]).  The patches to 
support binary query parameters for Cassandra and {{stress}} are attached to 
this issue, and the raw results can be found [here| 
http://people.apache.org/~eevans/3634].

_Note: Percentages listed are in relation to RPC performance._

h3. Inserts, 20M rows x 5 columns

!http://people.apache.org/~eevans/3634/insert_20mx5_noidx_t50_20111223.png|width=700!

|| ||Average OP rate||Average Latency||
|RPC|23,681/s|1.1ms|
|CQL|21,128/s (-11%)|1.3ms (+11%)|
|CQL w/ Prepared statements|23,911/s|1.1ms|
|CQL w/ Prepared statements (binary parms)|24,919/s (+5%)|1.2ms (+5%)|


h3. Inserts, 10M rows x 5 columns, KEYS index

!http://people.apache.org/~eevans/3634/insert_10mx5_keysidx_t50_20111223.png|width=700!

|| ||Average OP rate||Average Latency||
|RPC|10,054/s|5ms|
|CQL|9,326/s (-7%)|5.4ms (+8%)|
|CQL w/ Prepared statements|10,413/s (+3%)|4.8ms (-3%)|
|CQL w/ Prepared statements (binary parms)|10,299/s (+2%)|5ms|


h3. Counter increments, 10M rows x 5 columns

!http://people.apache.org/~eevans/3634/count_10mx5_noidx_t50_20111223.png|width=700!

|| ||Average OP rate||Average Latency||
|RPC|22,075/s|1.2ms|
|CQL|20,645/s (-6%)|1.2ms (+2%)|
|CQL w/ Prepared statements|24,286/s (+9%)|1.2ms (-1%)|
|CQL w/ Prepared statements (binary parms)|23,359/s (+5%)|1.2ms|


h3. Reads, 20M rows x 5 columns

!http://people.apache.org/~eevans/3634/read_20mx5_noidx_t50_20111223.png|width=700!

|| ||Average OP rate||Average Latency||
|RPC|22,285/s|2.1ms|
|CQL|20,080/s (-10%)|2.3ms (+9%)|
|CQL w/ Prepared statements|22,374/s|2.1ms (-1%)|
|CQL w/ Prepared statements (binary parms)|22,176/s|2.1ms|


 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: stress-change-bind-parms-to-BB.patch, 
 v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Vijay (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-3623:
-

Attachment: 0002-tests-for-MMaped-Compression-segmented-file-v2.patch
0001-MMaped-Compression-segmented-file-v2.patch

Attached patch has optimization on memcpy which the earlier one didnt.

Performance:
Current trunk: 400+ms Avg
Removing CRC (CASSANDRA-3611): 200+ms Avg
With this patch: 100+ms Avg



 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Update of Cassandra2474 by JonathanEllis

2011-12-23 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The Cassandra2474 page has been changed by JonathanEllis:
http://wiki.apache.org/cassandra/Cassandra2474?action=diffrev1=2rev2=3

Comment:
add Alpha, Beta, and Discussion Summary sections

  
  TableOfContents(100)
  
+ == Goals ==
+ 
+ Primary: provide a CQL syntax for updating and querying composite column 
families.
+ 
+ Secondary goal: proposed syntax should be implementable by the Hive driver 
with the minimum of changes from mainline Hive.  In particular, changes to the 
Hive parser are too difficult to maintain long-term and are Right Out.  We 
would prefer to avoid changes to the Hive metastore but this is doable if 
necessary.
+ 
+ Tertiary goal: it would be nice to also support supercolumns
+ 
+ == Non-goals ==
+ 
+ Supporting arbitrarily-and-non-uniformly nested document data is a 
non-goal.  https://issues.apache.org/jira/browse/CASSANDRA-3647 is created to 
follow up on this related problem.
+ 
  == Alpha ==
  
- Discussion starts 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13046834page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13046834|here]]
+ The short-lived first proposal envisioned adding the prefix from which to 
select a resultset to the table name in the FROM clause.  Discussion starts 
Discussion starts 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13046834page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13046834|here]]
  
- === Goals ===
+ {{{
+ SELECT x, y FROM foo:bar WHERE parent='columnA'
+ }}}
  
-  * FIXME: add goals
-  * FIXME: add goals
-  * FIXME: add goals
+ {{{
+ select a, b FROM foo:bar:columnA where subparent='x'
+ }}}
+ 
+ === Discussion Summary ===
+ 
+ Jonathan was thinking in terms of supercolumns for this early proposal.  It's 
not clear how to generalize this to composites where the subcolumns are not 
explicitly named in the CompositeType definition.
+ 
+ This proposal would require a Hive metastore change, but the nail in the 
coffin is that this means you cannot use WHERE clauses with the parent parts 
of the column.  So, no range queries (necessary for map/reduce) or even slices 
within the same row.
  
  == Beta ==
  
- Discussion starts 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13095626page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13095626|here]]
+ This proposal suggests the use of a keyword or hint to indicate that a query 
is transposed. Discussion starts 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13046937page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13046937|here]]
  
- === Goals ===
+ The first part of the discussion is where to put the transposition marker:
  
-  * FIXME: add goals
-  * FIXME: add goals
-  * FIXME: add goals
+ {{{
+ select /*+TRANSPOSED*/ key, column, subcolumn, value from foo;
+ }}}
+ 
+ {{{
+ select key, column, subcolumn, value from foo TRANSPOSED;
+ }}}
+ 
+ {{{
+ select transposed(key, column, subcolumn, value) from foo;
+ }}}
+ 
+ Settling on table:transposed because that requires no Hive changes:
+ 
+ {{{
+ select key, column, subcolumn, value from foo:transposed;
+ }}}
+ 
+ The second part, starting 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13095626page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13095626|here]],
 digs into how to deal with destructuring the composite column name:
+ 
+ {{{
+ SELECT name AS (tweet_id, username), value AS body
+ FROM timeline:transposed
+ WHERE tweet_id = '95a789a' AND user_id = 'cscotta'
+ }}}
+ 
+ {{{
+ SELECT component1 AS tweet_id, component2 AS username, component3 location, 
value AS body
+ FROM timeline:transposed
+ WHERE user_id = '95a789a'
+ }}}
+ 
+ {{{
+ UPDATE tweets:transposed SET COMPOUND NAME ('2e1c3308', 'cscotta') = 'My 
motocycle...' WHERE KEY = key;
+ }}}
+ 
+ {{{
+ UPDATE tweets:transposed SET value = 'my motorcycle' WHERE KEY= key AND 
column = COMPOUND_NAME('2e1c3308', 'cscotta');
+ }}}
+ 
+ === Discussion Summary ===
+ 
+ There was general agreement that FROM foo:transposed is a reasonable 
syntax, however, neither the componentX syntax (where X is in range(1, number 
of components in the compositetype) nor the name AS (x, y) syntax met with 
approval: the name AS syntax requires patching the Hive parser, and the 
componentX syntax is ugly and repetitive to use.  The UPDATE syntaxes were 
also unsatisfactory.
  
  == Gamma ==
  
+ This proposal switches gears to dealing with transposition using DDL instead 
of 
+ 
  Discussion starts 
[[https://issues.apache.org/jira/browse/CASSANDRA-2474?focusedCommentId=13171304page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13171304|here]]
  
- === Goals ===
- 
-  * FIXME: add goals
-  * FIXME: add

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175595#comment-13175595
 ] 

Vijay commented on CASSANDRA-3623:
--

Hot Methods before the patch (trunk):
Excl. User CPUName

   sec.  %
1480.474 100.00   Total
756.717  51.11   crc32
387.767  26.19   static@0x54999 (snappy-1.0.4.1-libsnappyjava.so)
 54.814   3.70   
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(java.lang.String,
 org.apache.cassandra.io.compress.CompressionMetadata, boolean)
 46.676   3.15   
org.apache.cassandra.io.util.RandomAccessReader.init(java.io.File, int, 
boolean)
 45.697   3.09   Copy::pd_disjoint_words(HeapWord*, HeapWord*, unsigned long)
 39.417   2.66   memcpy
 36.931   2.49   static@0xd8e9 (libpthread-2.5.so)
 23.272   1.57   CompactibleFreeListSpace::block_size(const HeapWord*) const
 22.766   1.54   SpinPause
 12.593   0.85   BlockOffsetArrayNonContigSpace::block_start_unsafe(const 
void*) const
  9.304   0.63   CardTableModRefBSForCTRS::card_will_be_scanned(signed char)
  8.468   0.57   CardTableModRefBS::non_clean_card_iterate_work(MemRegion, 
MemRegionClosure*, bool)
  8.051   0.54   
ParallelTaskTerminator::offer_termination(TerminatorTerminator*)
  5.400   0.36   madvise
  4.619   0.31   CardTableModRefBS::process_chunk_boundaries(Space*, 
DirtyCardToOopClosure*, MemRegion, MemRegion, signed char**, unsigned long, 
unsigned long)
  1.584   0.11   CardTableModRefBS::dirty_card_range_after_reset(MemRegion, 
bool, int)
  1.551   0.10   SweepClosure::do_blk_careful(HeapWord*)


Hot Methods After the patch:
sec.  %
537.681 100.00   Total
529.719  98.52   static@0x54999 (snappy-1.0.4.1-libsnappyjava.so)
4.168   0.78   memcpy
0.143   0.03   Unknown
0.121   0.02   send
0.121   0.02   sun.misc.Unsafe.park(boolean, long)
0.110   0.02   sun.misc.Unsafe.unpark(java.lang.Object)
0.088   0.02   Interpreter
0.077   0.01   org.apache.cassandra.utils.EstimatedHistogram.max()
0.077   0.01   recv
0.066   0.01   SpinPause
0.055   0.01   org.apache.cassandra.utils.EstimatedHistogram.mean()
0.044   0.01   java.lang.Object.wait(long)
0.044   0.01   org.apache.cassandra.utils.EstimatedHistogram.min()
0.044   0.01   __pthread_cond_signal
0.044   0.01   vtable stub
0.033   0.01   java.lang.Object.notify()
0.033   0.01   
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
0.033   0.01   
org.apache.cassandra.io.compress.CompressedMappedFileDataInput.read()
0.033   0.01   PhaseLive::compute(unsigned)
0.033   0.01   poll
0.022   0.00   Arena::contains(const void*) const
0.022   0.00   CompactibleFreeListSpace::free() const
0.022   0.00   I2C/C2I adapters
0.022   0.00   IndexSetIterator::advance_and_next()
0.022   0.00   java.lang.Class.forName0(java.lang.String, boolean, 
java.lang.ClassLoader)
0.022   0.00   java.lang.Long.getChars(long, int, char[])
0.022   0.00   java.nio.Bits.swap(int)



Before this patch response times:
Epoch   Rds/s   RdLat   Wrts/s  WrtLat %user   %sys  %idle  
 %iowait %steal  md0r/s  w/s rMB/s   wMB/s   NetRxKb NetTxKb Percentiles
 ReadWrite   Compacts
1324587443  15  186.305 00.000   27.85  0.0271.83   
0.24  0.053.890.000.120.0041  45  99th 
545.791 ms 95th 454.826 ms 99th 0.00 ms95th 0.00 msPen/0
1324587455  15  1142.712   00.000   39.55  0.1357.61
   2.50  0.21118.30  0.302.200.0034  36  99th 
8409.007 ms95th 8409.007 ms99th 0.00 ms95th 0.00 msPen/0
1324587467  10  171.808 00.000   23.83  0.0476.05   
0.04   0.054.800.000.140.00127 33  99th 
454.826 ms 95th 315.852 ms 99th 0.00 ms95th 0.00 msPen/0
1324587478  10  182.775 00.000   20.43  0.0479.47   
0.01  0.051.600.400.040.0030  37  99th 
379.022 ms 95th 379.022 ms 99th 0.00 ms95th 0.00 msPen/0
1324587490  13  190.893 00.000   27.58  0.0372.20   
0.14  0.063.200.500.090.0039  42  99th 
545.791 ms 95th 379.022 ms 99th 0.00 ms95th 0.00 msPen/0
1324587503  28  358.719 00.000   52.24  0.0846.20   
1.40  0.09159.40  0.003.160.00196 71  99th 
3379.391 ms95th 943.127 ms 99th 0.00 ms95th 0.00 msPen/0
1324587517  13  194.281 00.000   16.68  0.0283.23   
0.04  0.022.400.300.070.0038  41  99th 
785.939 ms 95th 545.791 ms 99th 0.00 ms95th 0.00 msPen/0
1324587535  36  662.410 00.000   58.34  0.0841.42   
0.06  0.103.600.200.110.00173 81  99th 
3379.391 ms

[jira] [Commented] (CASSANDRA-3507) Proposal: separate cqlsh from CQL drivers

2011-12-23 Thread Jeremy Hanna (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175596#comment-13175596
 ] 

Jeremy Hanna commented on CASSANDRA-3507:
-

Makes sense.  I hadn't realized so much had gone into the python based shell.  
I also hadn't realized it could be made into an executable for windows.

 Proposal: separate cqlsh from CQL drivers
 -

 Key: CASSANDRA-3507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3507
 Project: Cassandra
  Issue Type: Improvement
  Components: Packaging, Tools
Affects Versions: 1.0.3
 Environment: Debian-based systems
Reporter: paul cannon
Assignee: paul cannon
Priority: Minor
  Labels: cql, cqlsh
 Fix For: 1.1


 Whereas:
 * It has been shown to be very desirable to decouple the release cycles of 
 Cassandra from the various client CQL drivers, and
 * It is also desirable to include a good interactive CQL client with releases 
 of Cassandra, and
 * It is not desirable for Cassandra releases to depend on 3rd-party software 
 which is neither bundled with Cassandra nor readily available for every 
 target platform, but
 * Any good interactive CQL client will require a CQL driver;
 Therefore, be it resolved that:
 * cqlsh will not use an official or supported CQL driver, but will include 
 its own private CQL driver, not intended for use by anything else, and
 * the Cassandra project will still recommend installing and using a proper 
 CQL driver for client software.
 To ease maintenance, the private CQL driver included with cqlsh may very well 
 be created by copying the python CQL driver from one directory into 
 another, but the user shouldn't rely on this. Maybe we even ought to take 
 some minor steps to discourage its use for other purposes.
 Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175598#comment-13175598
 ] 

Vijay commented on CASSANDRA-3623:
--

The above test was done on 12 node cluster but the response time and the hot 
methods where collected from one random node in the cluster. 
This test was executed on AWS M2.4xl's with heap settings of 12/2.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Vijay (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175595#comment-13175595
 ] 

Vijay edited comment on CASSANDRA-3623 at 12/23/11 10:30 PM:
-

Hot Methods before the patch (trunk, without any patch):
Excl. User CPUName

   sec.  %
1480.474 100.00   Total
756.717  51.11   crc32
387.767  26.19   static@0x54999 (snappy-1.0.4.1-libsnappyjava.so)
 54.814   3.70   
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(java.lang.String,
 org.apache.cassandra.io.compress.CompressionMetadata, boolean)
 46.676   3.15   
org.apache.cassandra.io.util.RandomAccessReader.init(java.io.File, int, 
boolean)
 45.697   3.09   Copy::pd_disjoint_words(HeapWord*, HeapWord*, unsigned long)
 39.417   2.66   memcpy
 36.931   2.49   static@0xd8e9 (libpthread-2.5.so)
 23.272   1.57   CompactibleFreeListSpace::block_size(const HeapWord*) const
 22.766   1.54   SpinPause
 12.593   0.85   BlockOffsetArrayNonContigSpace::block_start_unsafe(const 
void*) const
  9.304   0.63   CardTableModRefBSForCTRS::card_will_be_scanned(signed char)
  8.468   0.57   CardTableModRefBS::non_clean_card_iterate_work(MemRegion, 
MemRegionClosure*, bool)
  8.051   0.54   
ParallelTaskTerminator::offer_termination(TerminatorTerminator*)
  5.400   0.36   madvise
  4.619   0.31   CardTableModRefBS::process_chunk_boundaries(Space*, 
DirtyCardToOopClosure*, MemRegion, MemRegion, signed char**, unsigned long, 
unsigned long)
  1.584   0.11   CardTableModRefBS::dirty_card_range_after_reset(MemRegion, 
bool, int)
  1.551   0.10   SweepClosure::do_blk_careful(HeapWord*)


Hot Methods After the patch:
sec.  %
537.681 100.00   Total
529.719  98.52   static@0x54999 (snappy-1.0.4.1-libsnappyjava.so)
4.168   0.78   memcpy
0.143   0.03   Unknown
0.121   0.02   send
0.121   0.02   sun.misc.Unsafe.park(boolean, long)
0.110   0.02   sun.misc.Unsafe.unpark(java.lang.Object)
0.088   0.02   Interpreter
0.077   0.01   org.apache.cassandra.utils.EstimatedHistogram.max()
0.077   0.01   recv
0.066   0.01   SpinPause
0.055   0.01   org.apache.cassandra.utils.EstimatedHistogram.mean()
0.044   0.01   java.lang.Object.wait(long)
0.044   0.01   org.apache.cassandra.utils.EstimatedHistogram.min()
0.044   0.01   __pthread_cond_signal
0.044   0.01   vtable stub
0.033   0.01   java.lang.Object.notify()
0.033   0.01   
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
0.033   0.01   
org.apache.cassandra.io.compress.CompressedMappedFileDataInput.read()
0.033   0.01   PhaseLive::compute(unsigned)
0.033   0.01   poll
0.022   0.00   Arena::contains(const void*) const
0.022   0.00   CompactibleFreeListSpace::free() const
0.022   0.00   I2C/C2I adapters
0.022   0.00   IndexSetIterator::advance_and_next()
0.022   0.00   java.lang.Class.forName0(java.lang.String, boolean, 
java.lang.ClassLoader)
0.022   0.00   java.lang.Long.getChars(long, int, char[])
0.022   0.00   java.nio.Bits.swap(int)



Before this patch response times (With crc chance set to 0):
Epoch   Rds/s   RdLat   Wrts/s  WrtLat %user   %sys  %idle  
 %iowait %steal  md0r/s  w/s rMB/s   wMB/s   NetRxKb NetTxKb Percentiles
 ReadWrite   Compacts
1324587443  15  186.305 00.000   27.85  0.0271.83   
0.24  0.053.890.000.120.0041  45  99th 
545.791 ms 95th 454.826 ms 99th 0.00 ms95th 0.00 msPen/0
1324587455  15  1142.712   00.000   39.55  0.1357.61
   2.50  0.21118.30  0.302.200.0034  36  99th 
8409.007 ms95th 8409.007 ms99th 0.00 ms95th 0.00 msPen/0
1324587467  10  171.808 00.000   23.83  0.0476.05   
0.04   0.054.800.000.140.00127 33  99th 
454.826 ms 95th 315.852 ms 99th 0.00 ms95th 0.00 msPen/0
1324587478  10  182.775 00.000   20.43  0.0479.47   
0.01  0.051.600.400.040.0030  37  99th 
379.022 ms 95th 379.022 ms 99th 0.00 ms95th 0.00 msPen/0
1324587490  13  190.893 00.000   27.58  0.0372.20   
0.14  0.063.200.500.090.0039  42  99th 
545.791 ms 95th 379.022 ms 99th 0.00 ms95th 0.00 msPen/0
1324587503  28  358.719 00.000   52.24  0.0846.20   
1.40  0.09159.40  0.003.160.00196 71  99th 
3379.391 ms95th 943.127 ms 99th 0.00 ms95th 0.00 msPen/0
1324587517  13  194.281 00.000   16.68  0.0283.23   
0.04  0.022.400.300.070.0038  41  99th 
785.939 ms 95th 545.791 ms 99th 0.00 ms95th 0.00 msPen/0
1324587535  36  662.410 00.000   58.34  0.08

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Pavel Yaskevich (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175601#comment-13175601
 ] 

Pavel Yaskevich commented on CASSANDRA-3623:


Can you please compare your version with trunk without crc32 because it doesn't 
seem to be fare match, would be nice to see the same statistics about hot 
methods and response time. The thing that I hate about MappedByteBuffer is if 
you duplicate it like you do in reBuffer() - will make unmap impossible until 
the every last duplicate is GC'ed, this implies that we won't be able to 
release old SSTables...

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175606#comment-13175606
 ] 

Vijay commented on CASSANDRA-3623:
--

I did it Again, i confused everyone with my test data :)
Hot methods shown above is the only data which is from the trunk rest are 
without CRC (hot methods without CRC and without this patch is as follows).


Excl. User CPU   Name

  sec.  %
629.460 100.00   Total
336.913  53.52   static@0x54999 (snappy-1.0.4.1-libsnappyjava.so)
50.074   7.96   
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(java.lang.String,
 org.apache.cassandra.io.compress.CompressionMetadata, boolean)
43.057   6.84   
org.apache.cassandra.io.util.RandomAccessReader.init(java.io.File, int, 
boolean)
35.623   5.66   memcpy
33.555   5.33   static@0xd8e9 (libpthread-2.5.so)
30.673   4.87   Copy::pd_disjoint_words(HeapWord*, HeapWord*, unsigned long)
26.384   4.19   CompactibleFreeListSpace::block_size(const HeapWord*) const
15.199   2.41   SpinPause
11.966   1.90   BlockOffsetArrayNonContigSpace::block_start_unsafe(const void*) 
const
 8.479   1.35   CardTableModRefBSForCTRS::card_will_be_scanned(signed char)
 8.007   1.27   CardTableModRefBS::non_clean_card_iterate_work(MemRegion, 
MemRegionClosure*, bool)
 5.169   0.82   madvise
 5.059   0.80   ParallelTaskTerminator::offer_termination(TerminatorTerminator*)
 4.146   0.66   CardTableModRefBS::process_chunk_boundaries(Space*, 
DirtyCardToOopClosure*, MemRegion, MemRegion, signed char**, unsigned long, 
unsigned long)
 2.431   0.39   CardTableModRefBS::dirty_card_range_after_reset(MemRegion, 
bool, int)
 1.375   0.22   SweepClosure::do_blk_careful(HeapWord*)
 0.825   0.13   Par_PushOrMarkClosure::do_oop(oopDesc*)
 0.616   0.10   GenericTaskQueueoopDesc*, 131072::pop_local(oopDesc*)
 0.561   0.09   instanceKlass::oop_oop_iterate_nv(oopDesc*, 
Par_PushOrMarkClosure*)
 0.473   0.08   CardTableModRefBS::process_stride(Space*, MemRegion, int, int, 
DirtyCardToOopClosure*, MemRegionClosure*, bool, signed char**, unsigned long, 
unsigned long)
 0.374   0.06   Par_MarkFromRootsClosure::scan_oops_in_oop(HeapWord*)
 0.319   0.05   BitMap::par_at_put(unsigned long, bool)
 0.308   0.05   MemRegion::intersection(MemRegion) const
 0.275   0.04   munmap
 0.220   0.03   CardTableModRefBS::dirty_card_iterate(MemRegion, 
MemRegionClosure*)


Hope this makes sense.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175607#comment-13175607
 ] 

Vijay commented on CASSANDRA-3623:
--

BTW: i can remove the duplicate() i didnt realize the implications, If you 
think rest is fine.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3374) CQL can't create column with compression or that use leveled compaction

2011-12-23 Thread paul cannon (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175619#comment-13175619
 ] 

paul cannon commented on CASSANDRA-3374:


+1

 CQL can't create column with compression or that use leveled compaction
 ---

 Key: CASSANDRA-3374
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3374
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.0.0
Reporter: Sylvain Lebresne
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: cql
 Fix For: 1.0.7

 Attachments: CASSANDRA-3374.patch


 Looking at CreateColumnFamilyStatement.java, it doesn't seem CQL can create 
 compressed column families, nor define a compaction strategy.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2011-12-23 Thread Rick Shaw (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175623#comment-13175623
 ] 

Rick Shaw commented on CASSANDRA-3634:
--

+1

Looks like Strings wins in terms of performance. It offers the most 
flexibility in transformation as well. I think we have a winner.

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: stress-change-bind-parms-to-BB.patch, 
 v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Pavel Yaskevich (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175626#comment-13175626
 ] 

Pavel Yaskevich commented on CASSANDRA-3623:


The problem is that you can't remove duplicate() because the same segment can 
be requested concurrently by different reads and we don't want to limit 
concurrency with synchronisation over segment use.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175627#comment-13175627
 ] 

Eric Evans commented on CASSANDRA-3634:
---

At Brandon's suggestion, I'm rerunning the insert test with some higher column 
counts.  That should make any per-term performance costs/savings more obvious.  
I'll post those results when I have them.

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: stress-change-bind-parms-to-BB.patch, 
 v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Pavel Yaskevich (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175628#comment-13175628
 ] 

Pavel Yaskevich commented on CASSANDRA-3623:


Hot reads show the if we remove overhead of the CRAR and RAR initialization we 
would get the numbers very close to mmap'ed I/O, also as you can see that 
snappy takes ~1.6x time with mmap'ed I/O.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175630#comment-13175630
 ] 

Vijay commented on CASSANDRA-3623:
--

Regarding duplicates i was thinking of Creating duplicates in CMSF and having a 
helper function to track it.

Regarding Hot Reads: (I tried before and you have to access the FD and caching 
the initialized object didn't help), We do get something like 50% better 
latencies by doing MMap'ed without copying the data. Snappy is 1.6% more 
because there isn't any thing else holding up or any other over head. 

Currently with this patch we dont have to copy any uncompressed data but the 
CRAR will copy because we dont handle the DirectBB to snappy and that's made 
possible by using MMapped IO.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment

2011-12-23 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175631#comment-13175631
]

Pavel Yaskevich commented on CASSANDRA-3623:

bq. We do get something like 50% better latencies by doing MMap'ed without
copying the data.

But hot methods show the oposite, the main thing that hurts performance in the
normal read case is not memcopy but reader class initialization overhead.

bq. Snappy is 1.6% more because there isn't any thing else holding up or any
other over head.

I don't get what do you mean here, can you please elaborate? Slower snappy
execution on my opinion could be caused by the additional expenses related to
data mapping to the user-space in the conditions of the migrating page cache
(situation when dataset does not fit in the page cache), mmap'ed I/O in that
case makes kernel do more work comparing to syscalls (normal I/O).

bq. Currently with this patch we dont have to copy any uncompressed data but
the CRAR will copy because we dont handle the DirectBB to snappy and that's
made possible by using MMapped IO.

Did you mean compressed instead of uncompressed here?

use MMapedBuffer in CompressedSegmentedFile.getSegment
--

Key: CASSANDRA-3623
URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Labels: compression
Fix For: 1.1

Attachments: 0001-MMaped-Compression-segmented-file-v2.patch,
0001-MMaped-Compression-segmented-file.patch,
0002-tests-for-MMaped-Compression-segmented-file-v2.patch

CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to
use the MMap and hence a higher CPU on the nodes and higher latencies on
reads.
This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
// TODO refactor this to separate concept of buffer to avoid lots of read()
syscalls and compression buffer
but i think a separate class for the Buffer will be better.

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175638#comment-13175638
 ] 

Vijay commented on CASSANDRA-3623:
--

Pavel, it doesnt show the opposite it actually shows the time spent is 98% in 
the snappy library and only 2% in the remaining part of the code. Where as in 
the earlier case we spend 58% of the time in Snappy and rest in the other part 
of the code. Snappy/decompression is definitely the bottleneck... all i am 
saying is that now we are more efficient and thats the only bottleneck.

Did you mean compressed instead of uncompressed here?
Yes i ment compressed.

Plz try a test before and after the patch you will see what i am talking about, 
I did run the cluster (before and after there isnt any other variable in play 
here) test it for a long time and after this patch shows constat performance 
and doesn't vary a lot (response times after the patch).

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175639#comment-13175639
 ] 

Vijay commented on CASSANDRA-3623:
--

constant performance = not a lot of difference from 95th percentile and 
Average. Before patch there was a huge swing between those. Data is shown above.

Plz note i am not selling this patch ;) I am trying to find a better 
performance for our use case which needs compression... I am completely open 
for other options.

 use MMapedBuffer in CompressedSegmentedFile.getSegment
 --

 Key: CASSANDRA-3623
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
  Labels: compression
 Fix For: 1.1

 Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 
 0001-MMaped-Compression-segmented-file.patch, 
 0002-tests-for-MMaped-Compression-segmented-file-v2.patch


 CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to 
 use the MMap and hence a higher CPU on the nodes and higher latencies on 
 reads. 
 This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
 // TODO refactor this to separate concept of buffer to avoid lots of read() 
 syscalls and compression buffer
 but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2011-12-23 Thread Jonathan Ellis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175646#comment-13175646
 ] 

Jonathan Ellis commented on CASSANDRA-3634:
---

Is the server om a separate machine from the client here?

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1

 Attachments: stress-change-bind-parms-to-BB.patch, 
 v1-0001-CASSANDRA-3634-generated-thrift-code.txt, 
 v1-0002-change-bind-parms-from-string-to-bytes.txt


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3603) CounterColumn and CounterContext use a log4j logger instead of using slf4j like the rest of the code base

2011-12-23 Thread Peter Schuller (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175647#comment-13175647
 ] 

Peter Schuller commented on CASSANDRA-3603:
---

My apologies. Looks like I accidentally nuked projectCodeStyle.xml in the wc 
without realizing it.

 CounterColumn and CounterContext use a log4j logger instead of using slf4j 
 like the rest of the code base
 -

 Key: CASSANDRA-3603
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3603
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Fix For: 1.0.7

 Attachments: CASSANDRA-3603-trunk.txt


 (Will submit patch but not now, no time.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3641) inconsistent/corrupt counters w/ broken shards never converge

[
https://issues.apache.org/jira/browse/CASSANDRA-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Peter Schuller updated CASSANDRA-3641:
--

Attachment: CASSANDRA-3641-trunk-nojmx.txt

New version attached. Rebased to current trunk, and no JMX. Otherwise identical.

inconsistent/corrupt counters w/ broken shards never converge
-

Key: CASSANDRA-3641
URL: https://issues.apache.org/jira/browse/CASSANDRA-3641
Project: Cassandra
Issue Type: Bug
Reporter: Peter Schuller
Assignee: Peter Schuller
Attachments: 3641-0.8-internal-not-for-inclusion.txt, 3641-trunk.txt,
CASSANDRA-3641-trunk-nojmx.txt

We ran into a case (which MIGHT be related to CASSANDRA-3070) whereby we had
counters that were corrupt (hopefully due to CASSANDRA-3178). The corruption
was that there would exist shards with the *same* node_id, *same* clock id,
but *different* counts.
The counter column diffing and reconciliation code assumes that this never
happens, and ignores the count. The problem with this is that if there is an
inconsistency, the result of a reconciliation will depend on the order of the
shards.
In our case for example, we would see the value of the counter randomly
fluctuating on a CL.ALL read, but we would get consistent (whatever the node
had) on CL.ONE (submitted to one of the nodes in the replica set for the key).
In addition, read repair would not work despite digest mismatches because the
diffing algorithm also did not care about the counts when determining the
differences to send.
I'm attaching patches that fixes this. The first patch is against our 0.8
branch, which is not terribly useful to people, but I include it because it
is the well-tested version that we have used on the production cluster which
was subject to this corruption.
The other patch is against trunk, and contains the same change.
What the patch does is:
* On diffing, treat as DISJOINT if there is a count discrepancy.
* On reconciliation, look at the count and *deterministically* pick the
higher one, and:
** log the fact that we detected a corrupt counter
** increment a JMX observable counter for monitoring purposes
A cluster which is subject to such corruption and has this patch, will fix
itself with and AES + compact (or just repeated compactions assuming the
replicate-on-compact is able to deliver correctly).

[jira] [Created] (CASSANDRA-3670) provide red flags JMX instrumentation

2011-12-23 Thread Peter Schuller (Created) (JIRA)

provide red flags JMX instrumentation
---

 Key: CASSANDRA-3670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor


As discussed in CASSANDRA-3641, it would be nice to expose through JMX certain 
information which is almost without exception indicative of something being 
wrong with the node or cluster.

In the CASSANDRA-3641 case, it was the detection of corrupt counter shards. 
Other examples include:

* Number of times the selection of files to compact was adjusted due to disk 
space heuristics
* Number of times compaction has failed
* Any I/O error reading from or writing to disk (the work here is collecting, 
not exposing, so maybe not in an initial version)
* Any data skipped due to checksum mismatches (when checksumming is being 
used); e.g., number of skips.
* Any arbitrary exception at least in certain code paths (compaction, scrub, 
cleanup for starters)

Probably other things.

The motivation is that if we have clear and obvious indications that something 
truly is wrong, it seems suboptimal to just leave that information in the log 
somewhere, for someone to discover later when something else broke as a result 
and a human investigates. You might argue that one should use non-trivial log 
analysis to detect these things, but I highly doubt a lot of people do this and 
it seems very wasteful to require that in comparison to just providing the 
MBean.

It is important to note that the *lack* of a certain problem being advertised 
in this MBean is not supposed to be indicative of a *lack* of a problem. 
Rather, the point is that to the extent we can easily do so, it is nice to have 
a clear method of communicating to monitoring systems where there *is* a clear 
indication of something being wrong.

The main part of this ticket is not to cover everything under the sun, but 
rather to reach agreement on adding an MBean where these types of indicators 
can be collected. Individual counters can then be added over time as one thinks 
of them.

I propose:

* Create an org.apache.cassandra.db.RedFlags MBean
* Populate with a few things to begin with.

I'll submit the patch if there is agreement.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3670) provide red flags JMX instrumentation


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Schuller updated CASSANDRA-3670:
--

Reviewer: slebresne

 provide red flags JMX instrumentation
 ---

 Key: CASSANDRA-3670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor

 As discussed in CASSANDRA-3641, it would be nice to expose through JMX 
 certain information which is almost without exception indicative of something 
 being wrong with the node or cluster.
 In the CASSANDRA-3641 case, it was the detection of corrupt counter shards. 
 Other examples include:
 * Number of times the selection of files to compact was adjusted due to disk 
 space heuristics
 * Number of times compaction has failed
 * Any I/O error reading from or writing to disk (the work here is collecting, 
 not exposing, so maybe not in an initial version)
 * Any data skipped due to checksum mismatches (when checksumming is being 
 used); e.g., number of skips.
 * Any arbitrary exception at least in certain code paths (compaction, scrub, 
 cleanup for starters)
 Probably other things.
 The motivation is that if we have clear and obvious indications that 
 something truly is wrong, it seems suboptimal to just leave that information 
 in the log somewhere, for someone to discover later when something else broke 
 as a result and a human investigates. You might argue that one should use 
 non-trivial log analysis to detect these things, but I highly doubt a lot of 
 people do this and it seems very wasteful to require that in comparison to 
 just providing the MBean.
 It is important to note that the *lack* of a certain problem being advertised 
 in this MBean is not supposed to be indicative of a *lack* of a problem. 
 Rather, the point is that to the extent we can easily do so, it is nice to 
 have a clear method of communicating to monitoring systems where there *is* a 
 clear indication of something being wrong.
 The main part of this ticket is not to cover everything under the sun, but 
 rather to reach agreement on adding an MBean where these types of indicators 
 can be collected. Individual counters can then be added over time as one 
 thinks of them.
 I propose:
 * Create an org.apache.cassandra.db.RedFlags MBean
 * Populate with a few things to begin with.
 I'll submit the patch if there is agreement.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3483) Support bringing up a new datacenter to existing cluster without repair

[
https://issues.apache.org/jira/browse/CASSANDRA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Peter Schuller updated CASSANDRA-3483:
--

Attachment: CASSANDRA-3483-trunk-noredesign.txt

Attaching version rebased to trunk but not yet re-factored.

Support bringing up a new datacenter to existing cluster without repair
---

Key: CASSANDRA-3483
URL: https://issues.apache.org/jira/browse/CASSANDRA-3483
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Chris Goffinet
Assignee: Peter Schuller
Attachments: CASSANDRA-3483-0.8-prelim.txt, CASSANDRA-3483-1.0.txt,
CASSANDRA-3483-trunk-noredesign.txt

Was talking to Brandon in irc, and we ran into a case where we want to bring
up a new DC to an existing cluster. He suggested from jbellis the way to do
it currently was set strategy options of dc2:0, then add the nodes. After the
nodes are up, change the RF of dc2, and run repair.
I'd like to avoid a repair as it runs AES and is a bit more intense than how
bootstrap works currently by just streaming ranges from the SSTables. Would
it be possible to improve this functionality (adding a new DC to existing
cluster) than the proposed method? We'd be happy to do a patch if we got some
input on the best way to go about it.

[jira] [Commented] (CASSANDRA-3670) provide red flags JMX instrumentation

2011-12-23 Thread Brandon Williams (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175656#comment-13175656
 ] 

Brandon Williams commented on CASSANDRA-3670:
-

I almost feel bad to mention this here, but since the fixver is unset I'll do 
it :)

It seems like converting a lot of our one-off metrics to 
https://github.com/codahale/metrics would provide much more flexibility in the 
future, as well as giving us better metrics to gauge this sort of thing by.

 provide red flags JMX instrumentation
 ---

 Key: CASSANDRA-3670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor

 As discussed in CASSANDRA-3641, it would be nice to expose through JMX 
 certain information which is almost without exception indicative of something 
 being wrong with the node or cluster.
 In the CASSANDRA-3641 case, it was the detection of corrupt counter shards. 
 Other examples include:
 * Number of times the selection of files to compact was adjusted due to disk 
 space heuristics
 * Number of times compaction has failed
 * Any I/O error reading from or writing to disk (the work here is collecting, 
 not exposing, so maybe not in an initial version)
 * Any data skipped due to checksum mismatches (when checksumming is being 
 used); e.g., number of skips.
 * Any arbitrary exception at least in certain code paths (compaction, scrub, 
 cleanup for starters)
 Probably other things.
 The motivation is that if we have clear and obvious indications that 
 something truly is wrong, it seems suboptimal to just leave that information 
 in the log somewhere, for someone to discover later when something else broke 
 as a result and a human investigates. You might argue that one should use 
 non-trivial log analysis to detect these things, but I highly doubt a lot of 
 people do this and it seems very wasteful to require that in comparison to 
 just providing the MBean.
 It is important to note that the *lack* of a certain problem being advertised 
 in this MBean is not supposed to be indicative of a *lack* of a problem. 
 Rather, the point is that to the extent we can easily do so, it is nice to 
 have a clear method of communicating to monitoring systems where there *is* a 
 clear indication of something being wrong.
 The main part of this ticket is not to cover everything under the sun, but 
 rather to reach agreement on adding an MBean where these types of indicators 
 can be collected. Individual counters can then be added over time as one 
 thinks of them.
 I propose:
 * Create an org.apache.cassandra.db.RedFlags MBean
 * Populate with a few things to begin with.
 I'll submit the patch if there is agreement.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3670) provide red flags JMX instrumentation

2011-12-23 Thread Peter Schuller (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175660#comment-13175660
]

Peter Schuller commented on CASSANDRA-3670:
---

I have not used it, and only had a quick look. But provided that it does the
job and has no significant downside, I'd be very +1 just from the mere fact
alone that it natively supports exposing metrics through HTTP and JSON while
still retaining JMX visibility, and from the fact that you avoid the
ThingMBean+Thing acrobatics. The histogram support seems convenient.

The RedFlags stuff could be a good pilot case. If it causes problems, it
doesn't break anything that people are used to working already.

provide red flags JMX instrumentation
---

Key: CASSANDRA-3670
URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor

As discussed in CASSANDRA-3641, it would be nice to expose through JMX
certain information which is almost without exception indicative of something
being wrong with the node or cluster.
In the CASSANDRA-3641 case, it was the detection of corrupt counter shards.
Other examples include:
* Number of times the selection of files to compact was adjusted due to disk
space heuristics
* Number of times compaction has failed
* Any I/O error reading from or writing to disk (the work here is collecting,
not exposing, so maybe not in an initial version)
* Any data skipped due to checksum mismatches (when checksumming is being
used); e.g., number of skips.
* Any arbitrary exception at least in certain code paths (compaction, scrub,
cleanup for starters)
Probably other things.
The motivation is that if we have clear and obvious indications that
something truly is wrong, it seems suboptimal to just leave that information
in the log somewhere, for someone to discover later when something else broke
as a result and a human investigates. You might argue that one should use
non-trivial log analysis to detect these things, but I highly doubt a lot of
people do this and it seems very wasteful to require that in comparison to
just providing the MBean.
It is important to note that the *lack* of a certain problem being advertised
in this MBean is not supposed to be indicative of a *lack* of a problem.
Rather, the point is that to the extent we can easily do so, it is nice to
have a clear method of communicating to monitoring systems where there *is* a
clear indication of something being wrong.
The main part of this ticket is not to cover everything under the sun, but
rather to reach agreement on adding an MBean where these types of indicators
can be collected. Individual counters can then be added over time as one
thinks of them.
I propose:
* Create an org.apache.cassandra.db.RedFlags MBean
* Populate with a few things to begin with.
I'll submit the patch if there is agreement.

[jira] [Commented] (CASSANDRA-3670) provide red flags JMX instrumentation

2011-12-23 Thread Peter Schuller (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175661#comment-13175661
]

Peter Schuller commented on CASSANDRA-3670:
---

Also, the whole JMX bit is actually a pretty annoying little detail for many
situations. There seems to exist no implementation outside of the JVM, and
writing a trivial monitor along the lines of:

{code}
warnings=$(curl http://localhost:XXX/bla/bla/redflags | egrep -v ': 0$' | wc
-l)
{code}

Becomes a chore. From what I can tell everyone keeps using that magic .jar that
no one knows where it comes from that e.g. cassandra-munin-plugins uses. It's a
real hassle to be constantly launching a JVM just for metrics extraction.

Now granted, if you are fully JMX enabled in your infrastructure there is no
issue, but I really think something like this goes a long way towards making
Cassandra more operator-friendly - particularly to individuals and/or small
organizations that want to monitor in some simple way and do not want to spend
time on JMX issues.

provide red flags JMX instrumentation
---

Key: CASSANDRA-3670
URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor

[jira] [Commented] (CASSANDRA-3670) provide red flags JMX instrumentation

2011-12-23 Thread Peter Schuller (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175662#comment-13175662
 ] 

Peter Schuller commented on CASSANDRA-3670:
---

(For the record I'm not suggesting actually writing a monitor exactly like 
that; I'm not a fan of ad-hoc shell scripting for such things due to the 
potential for silent failures. But choose any arbitrary productive language and 
a HTTP+JSON interface is trivial to use in a clean way.)

 provide red flags JMX instrumentation
 ---

 Key: CASSANDRA-3670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3670
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor

 As discussed in CASSANDRA-3641, it would be nice to expose through JMX 
 certain information which is almost without exception indicative of something 
 being wrong with the node or cluster.
 In the CASSANDRA-3641 case, it was the detection of corrupt counter shards. 
 Other examples include:
 * Number of times the selection of files to compact was adjusted due to disk 
 space heuristics
 * Number of times compaction has failed
 * Any I/O error reading from or writing to disk (the work here is collecting, 
 not exposing, so maybe not in an initial version)
 * Any data skipped due to checksum mismatches (when checksumming is being 
 used); e.g., number of skips.
 * Any arbitrary exception at least in certain code paths (compaction, scrub, 
 cleanup for starters)
 Probably other things.
 The motivation is that if we have clear and obvious indications that 
 something truly is wrong, it seems suboptimal to just leave that information 
 in the log somewhere, for someone to discover later when something else broke 
 as a result and a human investigates. You might argue that one should use 
 non-trivial log analysis to detect these things, but I highly doubt a lot of 
 people do this and it seems very wasteful to require that in comparison to 
 just providing the MBean.
 It is important to note that the *lack* of a certain problem being advertised 
 in this MBean is not supposed to be indicative of a *lack* of a problem. 
 Rather, the point is that to the extent we can easily do so, it is nice to 
 have a clear method of communicating to monitoring systems where there *is* a 
 clear indication of something being wrong.
 The main part of this ticket is not to cover everything under the sun, but 
 rather to reach agreement on adding an MBean where these types of indicators 
 can be collected. Individual counters can then be added over time as one 
 thinks of them.
 I propose:
 * Create an org.apache.cassandra.db.RedFlags MBean
 * Populate with a few things to begin with.
 I'll submit the patch if there is agreement.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3671) provide JMX counters for unavailables/timeouts for reads and writes

2011-12-23 Thread Peter Schuller (Created) (JIRA)

provide JMX counters for unavailables/timeouts for reads and writes
---

 Key: CASSANDRA-3671
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3671
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor


Attaching patch against trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3671) provide JMX counters for unavailables/timeouts for reads and writes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Schuller updated CASSANDRA-3671:
--

Attachment: CASSANDRA-3671-trunk.txt

 provide JMX counters for unavailables/timeouts for reads and writes
 ---

 Key: CASSANDRA-3671
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3671
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: CASSANDRA-3671-trunk.txt


 Attaching patch against trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3671) provide JMX counters for unavailables/timeouts for reads and writes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Schuller updated CASSANDRA-3671:
--

Attachment: CASSANDRA-3671-trunk-v2.txt

Accidentally attached old version of patch. v2 attached which doesn't fail to 
re-throw in one case.

 provide JMX counters for unavailables/timeouts for reads and writes
 ---

 Key: CASSANDRA-3671
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3671
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: CASSANDRA-3671-trunk-v2.txt, CASSANDRA-3671-trunk.txt


 Attaching patch against trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters