date:20150106

[jira] [Updated] (CASSANDRA-8547) Make RangeTombstone.Tracker.isDeleted() faster

2015-01-06 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8547:

Reviewer: Sylvain Lebresne  (was: Benedict)

 Make RangeTombstone.Tracker.isDeleted() faster
 --

 Key: CASSANDRA-8547
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8547
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: 2.0.11
Reporter: Dominic Letz
Assignee: Dominic Letz
 Fix For: 2.1.3

 Attachments: Selection_044.png, cassandra-2.0.11-8547.txt, 
 cassandra-2.1-8547.txt, rangetombstone.tracker.txt


 During compaction and repairs with many tombstones an exorbitant amount of 
 time is spend in RangeTombstone.Tracker.isDeleted().
 The amount of time spend there can be so big that compactions and repairs 
 look stalled and the time remaining time estimated frozen at the same value 
 for days.
 Using visualvm I've been sample profiling the code during execution and both 
 in Compaction as well as during repairs found this. (point in time backtraces 
 attached)
 Looking at the code the problem is obviously the linear scanning:
 {code}
 public boolean isDeleted(Column column)
 {
 for (RangeTombstone tombstone : ranges)
 {
 if (comparator.compare(column.name(), tombstone.min) = 0
  comparator.compare(column.name(), tombstone.max) = 0
  tombstone.maxTimestamp() = column.timestamp())
 {
 return true;
 }
 }
 return false;
 }
 {code}
 I would like to propose to change this and instead use a sorted list (e.g. 
 RangeTombstoneList) here instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8547) Make RangeTombstone.Tracker.isDeleted() faster

2015-01-06 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265891#comment-14265891
 ] 

Benedict commented on CASSANDRA-8547:
-

This looks like something [~slebresne] should take a look at.

 Make RangeTombstone.Tracker.isDeleted() faster
 --

 Key: CASSANDRA-8547
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8547
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: 2.0.11
Reporter: Dominic Letz
Assignee: Dominic Letz
 Fix For: 2.1.3

 Attachments: Selection_044.png, cassandra-2.0.11-8547.txt, 
 cassandra-2.1-8547.txt, rangetombstone.tracker.txt


 During compaction and repairs with many tombstones an exorbitant amount of 
 time is spend in RangeTombstone.Tracker.isDeleted().
 The amount of time spend there can be so big that compactions and repairs 
 look stalled and the time remaining time estimated frozen at the same value 
 for days.
 Using visualvm I've been sample profiling the code during execution and both 
 in Compaction as well as during repairs found this. (point in time backtraces 
 attached)
 Looking at the code the problem is obviously the linear scanning:
 {code}
 public boolean isDeleted(Column column)
 {
 for (RangeTombstone tombstone : ranges)
 {
 if (comparator.compare(column.name(), tombstone.min) = 0
  comparator.compare(column.name(), tombstone.max) = 0
  tombstone.maxTimestamp() = column.timestamp())
 {
 return true;
 }
 }
 return false;
 }
 {code}
 I would like to propose to change this and instead use a sorted list (e.g. 
 RangeTombstoneList) here instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8552) Large compactions run out of off-heap RAM

2015-01-06 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265898#comment-14265898
 ] 

Benedict commented on CASSANDRA-8552:
-

[~philipthompson] could we try to reproduce ourselves, since Brent has 
navigated around the problem for the moment? It would be great to figure this 
out and fix for 2.1.3

 Large compactions run out of off-heap RAM
 -

 Key: CASSANDRA-8552
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8552
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 14.4 
 AWS EC2
 12 m1.xlarge nodes [4 cores, 16GB RAM, 1TB storage (251GB Used)]
 Java build 1.7.0_55-b13 and build 1.8.0_25-b17
Reporter: Brent Haines
Assignee: Benedict
Priority: Blocker
 Fix For: 2.1.3

 Attachments: Screen Shot 2015-01-02 at 9.36.11 PM.png, fhandles.log, 
 freelog.log, lsof.txt, meminfo.txt, sysctl.txt, system.log


 We have a large table of storing, effectively event logs and a pair of 
 denormalized tables for indexing.
 When updating from 2.0 to 2.1 we saw performance improvements, but some 
 random and silent crashes during nightly repairs. We lost a node (totally 
 corrupted) and replaced it. That node has never stabilized -- it simply can't 
 finish the compactions. 
 Smaller compactions finish. Larger compactions, like these two never finish - 
 {code}
 pending tasks: 48
compaction type   keyspace table completed total   
  unit   progress
 Compaction   data   stories   16532973358   75977993784   
 bytes 21.76%
 Compaction   data   stories_by_text   10593780658   38555048812   
 bytes 27.48%
 Active compaction remaining time :   0h10m51s
 {code}
 We are not getting exceptions and are not running out of heap space. The 
 Ubuntu OOM killer is reaping the process after all of the memory is consumed. 
 We watch memory in the opscenter console and it will grow. If we turn off the 
 OOM killer for the process, it will run until everything else is killed 
 instead and then the kernel panics.
 We have the following settings configured: 
 2G Heap
 512M New
 {code}
 memtable_heap_space_in_mb: 1024
 memtable_offheap_space_in_mb: 1024
 memtable_allocation_type: heap_buffers
 commitlog_total_space_in_mb: 2048
 concurrent_compactors: 1
 compaction_throughput_mb_per_sec: 128
 {code}
 The compaction strategy is leveled (these are read-intensive tables that are 
 rarely updated)
 I have tried every setting, every option and I have the system where the MTBF 
 is about an hour now, but we never finish compacting because there are some 
 large compactions pending. None of the GC tools or settings help because it 
 is not a GC problem. It is an off-heap memory problem.
 We are getting these messages in our syslog 
 {code}
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219527] BUG: Bad page map in 
 process java  pte:0320 pmd:2d6fa5067
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219545] addr:7fb820be3000 
 vm_flags:0870 anon_vma:  (null) mapping:  (null) 
 index:7fb820be3
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219556] CPU: 3 PID: 27344 
 Comm: java Tainted: GB3.13.0-24-generic #47-Ubuntu
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219559]  880028510e40 
 88020d43da98 81715ac4 7fb820be3000
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219565]  88020d43dae0 
 81174183 0320 0007fb820be3
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219568]  8802d6fa5f18 
 0320 7fb820be3000 7fb820be4000
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219572] Call Trace:
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219584]  [81715ac4] 
 dump_stack+0x45/0x56
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219591]  [81174183] 
 print_bad_pte+0x1a3/0x250
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219594]  [81175439] 
 vm_normal_page+0x69/0x80
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219598]  [8117580b] 
 unmap_page_range+0x3bb/0x7f0
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219602]  [81175cc1] 
 unmap_single_vma+0x81/0xf0
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219605]  [81176d39] 
 unmap_vmas+0x49/0x90
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219610]  [8117feec] 
 exit_mmap+0x9c/0x170
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219617]  [8110fcf3] 
 ? __delayacct_add_tsk+0x153/0x170
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219621]  [8106482c] 
 mmput+0x5c/0x120
 Jan  2 07:06:00 ip-10-0-2-226 kernel: [49801151.219625]  [81069bbc] 
 do_exit+0x26c/0xa50
 Jan  2 07:06:00 ip-10-0-2-226 kernel:

[jira] [Commented] (CASSANDRA-6246) EPaxos

2015-01-06 Thread sankalp kohli (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265909#comment-14265909
 ] 

sankalp kohli commented on CASSANDRA-6246:
--

I am a little confused as in how you will use epoch to make sure instances are 
executed on all replicas when incrementing? 

 EPaxos
 --

 Key: CASSANDRA-6246
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6246
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Blake Eggleston
Priority: Minor

 One reason we haven't optimized our Paxos implementation with Multi-paxos is 
 that Multi-paxos requires leader election and hence, a period of 
 unavailability when the leader dies.
 EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, 
 (2) is particularly useful across multiple datacenters, and (3) allows any 
 node to act as coordinator: 
 http://sigops.org/sosp/sosp13/papers/p358-moraru.pdf
 However, there is substantial additional complexity involved if we choose to 
 implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8546) RangeTombstoneList becoming bottleneck on tombstone heavy tasks

2015-01-06 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8546:

Assignee: (was: Benedict)

 RangeTombstoneList becoming bottleneck on tombstone heavy tasks
 ---

 Key: CASSANDRA-8546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8546
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: 2.0.11 / 2.1
Reporter: Dominic Letz
 Fix For: 2.1.3

 Attachments: cassandra-2.0.11-8546.txt, cassandra-2.1-8546.txt, 
 rangetombstonelist_compaction.png, rangetombstonelist_mutation.png, 
 rangetombstonelist_read.png, tombstone_test.tgz


 I would like to propose a change of the data structure used in the 
 RangeTombstoneList to store and insert tombstone ranges to something with at 
 least O(log N) insert in the middle and at near O(1) and start AND end. Here 
 is why:
 When having tombstone heavy work-loads the current implementation of 
 RangeTombstoneList becomes a bottleneck with slice queries.
 Scanning the number of tombstones up to the default maximum (100k) can take 
 up to 3 minutes of how addInternal() scales on insertion of middle and start 
 elements.
 The attached test shows that with 50k deletes from both sides of a range.
 INSERT 1...11
 flush()
 DELETE 1...5
 DELETE 11...6
 While one direction performs ok (~400ms on my notebook):
 {code}
 SELECT * FROM timeseries WHERE name = 'a' ORDER BY timestamp DESC LIMIT 1
 {code}
 The other direction underperforms (~7seconds on my notebook)
 {code}
 SELECT * FROM timeseries WHERE name = 'a' ORDER BY timestamp ASC LIMIT 1
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)

2015-01-06 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265911#comment-14265911
 ] 

Benedict commented on CASSANDRA-8325:
-

[~graham sanderson] thanks for the New Year cheer and putting together another 
great patch.

My slight concern with this approach is that it pollutes the instruction cache, 
so we do need to run some performance tests. These codepaths _are_ pretty 
sensitive to these effects.

Ultimately, I think a follow up ticket might be sensible that performs all of 
this work natively, since we should not need such code duplication. 

 Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
 -

 Key: CASSANDRA-8325
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325
 Project: Cassandra
  Issue Type: Bug
 Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit 
 Server VM
Reporter: Leonid Shalupov
 Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt, 
 untested_8325.patch


 See attached error file after JVM crash
 {quote}
 FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu 
 Jan 16 22:34:59 UTC 2014 
 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
 {quote}
 {quote}
  % java -version
 openjdk version 1.7.0_71
 OpenJDK Runtime Environment (build 1.7.0_71-b14)
 OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode)
 {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety

2015-01-06 Thread Benedict (JIRA)

Benedict created CASSANDRA-8568:
---

 Summary: Impose new API on data tracker modifications that makes 
correct usage obvious and imposes safety
 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict


DataTracker has become a bit of a quagmire, and not at all obvious to interface 
with, with many subtly different modifiers. I suspect it is still subtly 
broken, especially around error recovery.

I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
those situations where a try/finally block isn't possible) objects that have 
transactional behaviour, and with few simple declarative methods that can be 
composed simply to provide all of the functionality we currently need.

See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8506) Improve management of DataTracker, esp. compacting

2015-01-06 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-8506.
-
Resolution: Duplicate

 Improve management of DataTracker, esp. compacting
 --

 Key: CASSANDRA-8506
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8506
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
 Fix For: 3.0


 Building on CASSANDRA-7705, we can use debuggable ref counting to manage the 
 marking and unmarking of compaction state, so that we can quickly track down 
 errors.
 We should also simplify the logic wrt rewriters, by ignoring the descriptor 
 type, perhaps for all sets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

2015-01-06 Thread Robert Stupp (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257721#comment-14257721
 ] 

Robert Stupp edited comment on CASSANDRA-7438 at 1/6/15 10:08 AM:
--

I had the opportunity to test OHC on a big machine.
First: it works - very happy about that :)

Some things I want to notice:
* high number of segments do not have any really measurable influence (default 
of 2* # of cores is fine)
* throughput heavily depends on serialization (hash entry size) - Java8 gave 
about 10% to 15% improvement in some tests (either on {{Unsafe.copyMemory}} or 
something related like JNI barrier)
* the number of entries per bucket stays pretty low with the default load 
factor of .75 - vast majority has 0 or 1 entries, some 2 or 3 and few up to 8

Issue (not solvable yet):
It works great for hash entries to approx. 64kB with good to great throughput. 
Above that barrier it first works good but after some time the system spends a 
huge amount of CPU time (~95%) in {{malloc()}} / {{free()}} (with jemalloc, 
Unsafe.allocate is not worth discussing at all on Linux).
I tried to add some „memory buffer cache“ that caches free’d hash entries for 
reuse. But it turned out that in the end it would be too complex if done right. 
The current implementation is still in the code, but must be explicitly enabled 
with a system property. Workloads with small entries and high number of threads 
easily trigger Linux OOM protection (that kills the process). Please note that 
it works with large hash entries - but throughput drops dramatically to just a 
few thousand writes per second.

Some numbers (value sizes have gaussian distribution). Had to do these tests in 
a hurry because I had to give back the machine. Code used during these tests is 
tagged as {{0.1-SNAP-Bench}} in git. Throughput is limited by {{malloc()}} / 
{{free()}} and most tests did only use 50% of available CPU capacity (on 
_c3.8xlarge_ - 32 cores, Intel Xeon E5-2680v2 @2.8GHz, 64GB).
* -1k..200k value size, 32 threads, 1M keys, 90% read ratio, 32GB: 22k 
writes/sec, 200k reads/sec, ~8k evictions/sec, write: 8ms (99perc), read: 
3ms(99perc)-
* -1k..64k value size, 500 threads, 1M keys, 90% read ratio, 32GB: 55k 
writes/sec, 499k reads/sec, ~2k evictions/sec, write: .1ms (99perc), read: 
.03ms(99perc)-
* -1k..64k value size, 500 threads, 1M keys, 50% read ratio, 32GB: 195k 
writes/sec, 195k reads/sec, ~9k evictions/sec, write: .2ms (99perc), read: 
.1ms(99perc)-
* -1k..64k value size, 500 threads, 1M keys, 10% read ratio, 32GB: 185k 
writes/sec, 20k reads/sec, ~7k evictions/sec, write: 4ms (99perc), read: 
.07ms(99perc)-
* -1k..16k value size, 500 threads, 5M keys, 90% read ratio, 32GB: 110k 
writes/sec, 1M reads/sec, 30k evictions/sec, write: .04ms (99perc), read: 
.01ms(99perc)-
* -1k..16k value size, 500 threads, 5M keys, 50% read ratio, 32GB: 420k 
writes/sec, 420k reads/sec, 125k evictions/sec, write: .06ms (99perc), read: 
.01ms(99perc)-
* -1k..16k value size, 500 threads, 5M keys, 10% read ratio, 32GB: 435k 
writes/sec, 48k reads/sec, 130k evictions/sec, write: .06ms (99perc), read: 
.01ms(99perc)-
* -1k..4k value size, 500 threads, 20M keys, 90% read ratio, 32GB: 140k 
writes/sec, 1.25M reads/sec, 50k evictions/sec, write: .02ms (99perc), read: 
.005ms(99perc)-
* -1k..4k value size, 500 threads, 20M keys, 50% read ratio, 32GB: 530k 
writes/sec, 530k reads/sec, 220k evictions/sec, write: .04ms (99perc), read: 
.005ms(99perc)-
* -1k..4k value size, 500 threads, 20M keys, 10% read ratio, 32GB: 665k 
writes/sec, 74k reads/sec, 250k evcictions/sec, write: .04ms (99perc), read: 
.005ms(99perc)-

Command line to execute the benchmark:
{code}
java -jar ohc-benchmark/target/ohc-benchmark-0.1-SNAPSHOT.jar -rkd 
'uniform(1..2000)' -wkd 'uniform(1..2000)' -vs 'gaussian(1024..4096,2)' 
-r .1 -cap 320 -d 86400 -t 500 -dr 8

-r = read rate
-d = duration
-t = # of threads
-dr = # of driver threads that feed the worker threads
-rkd = read key distribution
-wkd = write key distribution
-vs = value size
-cap = capacity
{code}

Sample bucket histogram from 20M test:
{code}
[0..0]: 8118604
[1..1]: 5892298
[2..2]: 2138308
[3..3]: 518089
[4..4]: 94441
[5..5]: 13672
[6..6]: 1599
[7..7]: 189
[8..9]: 16
{code}

After trapping into that memory management issue with varying allocation sized 
of some few kB to several MB, I think that it’s still worth to work on an own 
off-heap memory management. Maybe some block-based approach (fixed or 
variable). But that’s out of the scope of this ticket.

EDIT: The problem with high system-CPU usage only persists on systems with 
multiple CPUs. Cross check with the second CPU socket disabled - calling the 
benchmark with {{taskset 0x3ff java -jar ...}}  does not show 95% system CPU 
usage.

EDIT2: Marked benchmark values as invalid (see my comment on

[jira] [Updated] (CASSANDRA-8194) Reading from Auth table should not be in the request path

2015-01-06 Thread sankalp kohli (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli updated CASSANDRA-8194:
-
Assignee: Vishy Kasar

 Reading from Auth table should not be in the request path
 -

 Key: CASSANDRA-8194
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8194
 Project: Cassandra
  Issue Type: Improvement
Reporter: Vishy Kasar
Assignee: Vishy Kasar
Priority: Minor
 Fix For: 2.0.12, 3.0

 Attachments: 8194-V2.patch, 8194.patch, CacheTest2.java


 We use PasswordAuthenticator and PasswordAuthorizer. The system_auth has a RF 
 of 10 per DC over 2 DCs. The permissions_validity_in_ms is 5 minutes. 
 We still have few thousand requests failing each day with the trace below. 
 The reason for this is read cache request realizing that cached entry has 
 expired and doing a blocking request to refresh cache. 
 We should have cache refreshed periodically only in the back ground. The user 
 request should simply look at the cache and not try to refresh it. 
 com.google.common.util.concurrent.UncheckedExecutionException: 
 java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
 received only 0 responses.
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
   at 
 com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
   at 
 org.apache.cassandra.service.ClientState.authorize(ClientState.java:292)
   at 
 org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:172)
   at 
 org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:165)
   at 
 org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:149)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:75)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:102)
   at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:113)
   at 
 org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1735)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4162)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4150)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:206)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 Caused by: java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
 received only 0 responses.
   at org.apache.cassandra.auth.Auth.selectUser(Auth.java:256)
   at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:84)
   at 
 org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
   at 
 org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:68)
   at org.apache.cassandra.service.ClientState$1.load(ClientState.java:278)
   at org.apache.cassandra.service.ClientState$1.load(ClientState.java:275)
   at 
 com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
   at 
 com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
   at 
 com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
   ... 19 more
 Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation 
 timed out - received only 0 responses.
   at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
   at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:943)
   at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:828)
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:140)
   at org.apache.cassandra.auth.Auth.selectUser(Auth.java:245)
   ... 28 more
 ERROR [Thrift:17232] 2014-10-24 05:06:51,004 CustomTThreadPoolServer.java 
 (line 224) Error occurred during processing of message.
 com.google.common.util.concurrent.UncheckedExecutionException:

[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)

2015-01-06 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265852#comment-14265852
 ] 

Sylvain Lebresne commented on CASSANDRA-8325:
-

For the sake of clarification, being from DataStax doesn't entitled anyone to 
make calls regarding Apache Cassandra (having a good track record of 
contributions, which usually means you're a committer, is probably what 
entitled you to make calls, though decisions are ideally agreed upon 
collectively).

 Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
 -

 Key: CASSANDRA-8325
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325
 Project: Cassandra
  Issue Type: Bug
 Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit 
 Server VM
Reporter: Leonid Shalupov
 Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt, 
 untested_8325.patch


 See attached error file after JVM crash
 {quote}
 FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu 
 Jan 16 22:34:59 UTC 2014 
 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
 {quote}
 {quote}
  % java -version
 openjdk version 1.7.0_71
 OpenJDK Runtime Environment (build 1.7.0_71-b14)
 OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode)
 {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8567) Functions like length() and trim() on cqlsh table fields

2015-01-06 Thread Sylvain Lebresne (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-8567.
-
Resolution: Won't Fix

We might add some {{length}} and {{trim}} method at some point, though even if 
we don't, UDF in 3.0 will allow you to define them. But what you're really 
asking here is to be able to use functions in {{ORDER BY}}, and that is not 
going to happen. The reason is that we have no better strategy server side than 
to read everything and sort it in memory afterwards before returning to the 
client. Which 1) is no faster than letting you the client do the sorting client 
side and 2) doesn't work with paging at all and thus makes it likely to OOM as 
soon as the amount of data to order is not small.

For those reason, we prefer letting clients do the sorting themselves 
post-query if they wants to (but we understand that for cqlsh it's not as nice 
as you'd like. Though if you really need quick and dirty post-query sorting, 
piping the output of cqlsh into sort (the unix utility) isn't terribly hard). 
Alternatively, if you need results in that sort order often, you should store 
the data in a table whose clustering is {{length(trim(field)}}.

 Functions like length() and trim() on cqlsh table fields
 

 Key: CASSANDRA-8567
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8567
 Project: Cassandra
  Issue Type: Wish
  Components: Core
Reporter: Rekha Joshi

 It would be nice to be able to order by length of field values.
 A function like length(field) and trim(field) on cqlsh
 To enable do something like say - select * from  table where 
 field1=value order by length(trim(field2)) desc;
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

2015-01-06 Thread Robert Stupp (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265947#comment-14265947
 ] 

Robert Stupp commented on CASSANDRA-7438:
-

The latest (just checked in) benchmark implementation gives much better 
results. Using 
{{com.codahale.metrics.Timer#time(java.util.concurrent.CallableT)}} 
eliminates use of {{System.nanoTime()}} or 
{{ThreadMXBean.getCurrentThreadCpuTime()}} - it can directly use its internal 
clock.
The benchmark {{java -jar ohc-benchmark/target/ohc-benchmark-0.2-SNAPSHOT.jar 
-rkd 'gaussian(1..2000,2)' -wkd 'gaussian(1..2000,2)' -vs 
'gaussian(1024..4096,2)' -r .9 -cap 16 -d 30 -t 30}} improved from 800k 
reads to 3.3M reads per second w/ 8 cores). So yes - benchmark was measuring 
its own mad code. Due to that I edited my previous comment with the benchmark 
results since those are invalid now.

I've added a (yet simple) JMH benchmark as a separate module. This one can 
cause high system CPU usage - at operation rates of 2M per second or more (8 
cores). I think these rates are really fine.

Note: these rates cannot be achieved in production since then you'll obviously 
have to pay for (de)serialization, too.

So we want to address these topics as follow-up:
* own off-heap allocator
* C* ability to access off-heap cached rows
* C* ability to serialize hot keys directly from off-heap (might be a minor win 
since it's triggered not that often)
* per-table knob to control whether to add to row-cache on writes -- I strongly 
believe that this is a useful feature (maybe LHF) on workloads where read and 
written data work on different (row} keys.
* investigate if counter-cache can benefit
* investigate if key-cache can benefit

bq. You could start with it outside and publish to maven central and if there 
an issue getting patches applied quickly we can always fork it in C*.
OK

bq. pluggable row cache
Then I'll start with that - just make row-cache pluggable and the 
implementation configurable.

Note: JNA has a synchronized block that's executed at every call - version 
4.2.0 fixes this (don't know when it will be released).

 Serializing Row cache alternative (Fully off heap)
 --

 Key: CASSANDRA-7438
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux
Reporter: Vijay
Assignee: Vijay
  Labels: performance
 Fix For: 3.0

 Attachments: 0001-CASSANDRA-7438.patch, tests.zip


 Currently SerializingCache is partially off heap, keys are still stored in 
 JVM heap as BB, 
 * There is a higher GC costs for a reasonably big cache.
 * Some users have used the row cache efficiently in production for better 
 results, but this requires careful tunning.
 * Overhead in Memory for the cache entries are relatively high.
 So the proposal for this ticket is to move the LRU cache logic completely off 
 heap and use JNI to interact with cache. We might want to ensure that the new 
 implementation match the existing API's (ICache), and the implementation 
 needs to have safe memory access, low overhead in memory and less memcpy's 
 (As much as possible).
 We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

2015-01-06 Thread Robert Stupp (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7438:

Reviewer: Ariel Weisberg  (was: Robert Stupp)
Assignee: Robert Stupp  (was: Vijay)

 Serializing Row cache alternative (Fully off heap)
 --

 Key: CASSANDRA-7438
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux
Reporter: Vijay
Assignee: Robert Stupp
  Labels: performance
 Fix For: 3.0

 Attachments: 0001-CASSANDRA-7438.patch, tests.zip


 Currently SerializingCache is partially off heap, keys are still stored in 
 JVM heap as BB, 
 * There is a higher GC costs for a reasonably big cache.
 * Some users have used the row cache efficiently in production for better 
 results, but this requires careful tunning.
 * Overhead in Memory for the cache entries are relatively high.
 So the proposal for this ticket is to move the LRU cache logic completely off 
 heap and use JNI to interact with cache. We might want to ensure that the new 
 implementation match the existing API's (ICache), and the implementation 
 needs to have safe memory access, low overhead in memory and less memcpy's 
 (As much as possible).
 We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8163) Re-introduce DESCRIBE permission

2015-01-06 Thread Ben Laplanche (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265958#comment-14265958
 ] 

Ben Laplanche commented on CASSANDRA-8163:
--

The ability for users to see the table names for other users keyspaces is 
blocking us from using a cluster in a multi-tenant setup, which for some 
clients and the nature of their work can be prohibitive in using Cassandra. 

 Re-introduce DESCRIBE permission
 

 Key: CASSANDRA-8163
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8163
 Project: Cassandra
  Issue Type: Improvement
Reporter: Vishy Kasar
Priority: Minor

 We have a cluster like this:
 project1_keyspace
 table101
 table102
 project2_keyspace
 table201
 table202
 We have set up following users and grants:
 project1_user has all access to project1_keyspace 
 project2_user has all access to project2_keyspace
 However project1_user can still do a 'describe schema' and get the schema for 
 project2_keyspace as well. We do not want project1_user to have any knowledge 
 for project2 in any way (cqlsh/java-driver etc) .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out

2015-01-06 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265965#comment-14265965
 ] 

Benedict commented on CASSANDRA-8558:
-

This is nothing to do with either change as far as I can tell. Somewhere 
inbetween those two changes something else was presumably broken, that is 
unrelated to either of them. It's possible that it was broken before either, in 
fact. Somewhere amongst them we changed drop behaviour to introduce flushing of 
dirty tables, and this flushing causes the problem. In fact we probably have a 
much worse problem than this appears.

I can illicit this behaviour with a simple call to nodetool flush. The deletion 
records are flushed to disk, and appear in their respective sstables, but are 
not being returned by the IndexedSliceReader that queries them. A new test to 
find this behaviour would be simpler: 

CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};
CREATE  TABLE space1.table1(a int, b int, c text,primary key(a,b));
INSERT INTO space1.table1(a,b,c) VALUES(1,1,'1');
// nodetool flush
DELETE FROM space1.table3 where a=1 and b=1;
// nodetool flush
SELECT * FROM space1.table3 where a=1 and b=1;

This is much more fundamentally broken than this ticket suggests, but I'm 
probably not the best person to investigate, since it looks to be a problem 
with IndexedSliceReader. Hopefully a git bisect with the updated test will 
blame a suitable candidate next time around :)

(assuming it isn't somehow still my fault)

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Benedict
 Fix For: 2.1.3


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out

2015-01-06 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8558:

Assignee: Philip Thompson  (was: Benedict)

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Philip Thompson
 Fix For: 2.1.3


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8558) deleted row still can be selected out

2015-01-06 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265965#comment-14265965
 ] 

Benedict edited comment on CASSANDRA-8558 at 1/6/15 12:12 PM:
--

This is nothing to do with either change as far as I can tell. Somewhere 
inbetween those two changes something else was presumably broken, that is 
unrelated to either of them. It's possible that it was broken before either, in 
fact. Somewhere amongst them we changed drop behaviour to introduce flushing of 
dirty tables, and this flushing causes the problem. In fact we probably have a 
much worse problem than this appears.

I can illicit this behaviour with a simple call to nodetool flush. The deletion 
records are flushed to disk, and appear in their respective sstables, but are 
not being returned by the IndexedSliceReader that queries them. A new test to 
find this behaviour would be simpler: 

CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};
CREATE  TABLE space1.table1(a int, b int, c text,primary key(a,b));
INSERT INTO space1.table1(a,b,c) VALUES(1,1,'1');
// nodetool flush
DELETE FROM space1.table1 where a=1 and b=1;
// nodetool flush
SELECT * FROM space1.table1 where a=1 and b=1;

This is much more fundamentally broken than this ticket suggests, but I'm 
probably not the best person to investigate, since it looks to be a problem 
with IndexedSliceReader. Hopefully a git bisect with the updated test will 
blame a suitable candidate next time around :)

(assuming it isn't somehow still my fault)


was (Author: benedict):
This is nothing to do with either change as far as I can tell. Somewhere 
inbetween those two changes something else was presumably broken, that is 
unrelated to either of them. It's possible that it was broken before either, in 
fact. Somewhere amongst them we changed drop behaviour to introduce flushing of 
dirty tables, and this flushing causes the problem. In fact we probably have a 
much worse problem than this appears.

I can illicit this behaviour with a simple call to nodetool flush. The deletion 
records are flushed to disk, and appear in their respective sstables, but are 
not being returned by the IndexedSliceReader that queries them. A new test to 
find this behaviour would be simpler: 

CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};
CREATE  TABLE space1.table1(a int, b int, c text,primary key(a,b));
INSERT INTO space1.table1(a,b,c) VALUES(1,1,'1');
// nodetool flush
DELETE FROM space1.table3 where a=1 and b=1;
// nodetool flush
SELECT * FROM space1.table3 where a=1 and b=1;

This is much more fundamentally broken than this ticket suggests, but I'm 
probably not the best person to investigate, since it looks to be a problem 
with IndexedSliceReader. Hopefully a git bisect with the updated test will 
blame a suitable candidate next time around :)

(assuming it isn't somehow still my fault)

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Philip Thompson
 Fix For: 2.1.3


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8194) Reading from Auth table should not be in the request path

2015-01-06 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266056#comment-14266056
 ] 

Sam Tunnicliffe commented on CASSANDRA-8194:


While there is a window during which a stale set of permissions is used, under 
normal operation I don't think this *should* present too many practical 
problems. 

Refresh is triggered by the first lookup after permisions_validity_in_ms, so 
we'll continue to use the stale set between that point and when that refresh 
actually completes. Outside of tests though, clients have no 
visibility/expectation about the precise load or expiry timings, so this 
shouldn't usually matter. My concern would be performing every 
IAuthorizer.authorize call on a single thread using StorageService.tasks 
instead of distributing them across client request threads could cause a 
backlog and allow the window to grow unacceptably (plus, these tasks will also 
be contending with other users of the shared executor). 

The point about the proliferation of threads and executors is valid, but maybe 
there's a case for a dedicated executor here. We could make it a TPE with a 
default poolsize of 1 but allow that to be increased via a system property if 
necessary.

What may be more of an issue is that we'll continue to serve the stale perms as 
long as the refresh fails completely due to IAuthorizer.authorize throwing some 
exception. This shouldn't really happen with CassandraAuthorizer, but other 
IAuthorizer impls could well encounter errors when fetching perms. To guard 
against that, we can force an invalidation if the ListenableFutureTask 
encounters an exception. That would pretty much maintain current behaviour, 
with the client receiving an error response while the refresh fails (actually, 
the authorize calls after an error would serve stale perms until the exception 
is thrown  caught, but all subsequent calls would fail as per current 
behaviour).

I've attached a v3 with this second change, what are your thoughts on reverting 
back to a dedicated executor for cache refresh?

Also, as I mentioned, tests do have concrete expectations about expiry of 
permissions and so this breaks auth_test.py:TestAuth.permissions_caching_test. 
I've pushed a fix [here|https://github.com/beobal/cassandra-dtest/tree/8194] 
and I'll open a PR shortly.

 Reading from Auth table should not be in the request path
 -

 Key: CASSANDRA-8194
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8194
 Project: Cassandra
  Issue Type: Improvement
Reporter: Vishy Kasar
Assignee: Vishy Kasar
Priority: Minor
 Fix For: 2.0.12, 3.0

 Attachments: 8194-V2.patch, 8194.patch, CacheTest2.java


 We use PasswordAuthenticator and PasswordAuthorizer. The system_auth has a RF 
 of 10 per DC over 2 DCs. The permissions_validity_in_ms is 5 minutes. 
 We still have few thousand requests failing each day with the trace below. 
 The reason for this is read cache request realizing that cached entry has 
 expired and doing a blocking request to refresh cache. 
 We should have cache refreshed periodically only in the back ground. The user 
 request should simply look at the cache and not try to refresh it. 
 com.google.common.util.concurrent.UncheckedExecutionException: 
 java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
 received only 0 responses.
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
   at 
 com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
   at 
 org.apache.cassandra.service.ClientState.authorize(ClientState.java:292)
   at 
 org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:172)
   at 
 org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:165)
   at 
 org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:149)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:75)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:102)
   at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:113)
   at 
 org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1735)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4162)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4150)
   at

[jira] [Commented] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations

2015-01-06 Thread Rajanarayanan Thottuvaikkatumana (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266033#comment-14266033
 ] 

Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-7124:
-

[~yukim], Did you get a chance to look at the changes made? Any comments? Thanks

 Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
 

 Key: CASSANDRA-7124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
  Labels: lhf
 Fix For: 3.0

 Attachments: 7124-wip.txt, cassandra-trunk-compact-7124.txt, 
 cassandra-trunk-decommission-7124.txt


 If {{nodetool cleanup}} or some other long-running operation takes too long 
 to complete, you'll see an error like the one in CASSANDRA-2126, so you can't 
 tell if the operation completed successfully or not.  CASSANDRA-4767 fixed 
 this for repairs with JMX notifications.  We should do something similar for 
 nodetool cleanup, compact, decommission, move, relocate, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8194) Reading from Auth table should not be in the request path

2015-01-06 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-8194:
---
Attachment: 8194-V3.txt

 Reading from Auth table should not be in the request path
 -

 Key: CASSANDRA-8194
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8194
 Project: Cassandra
  Issue Type: Improvement
Reporter: Vishy Kasar
Assignee: Vishy Kasar
Priority: Minor
 Fix For: 2.0.12, 3.0

 Attachments: 8194-V2.patch, 8194-V3.txt, 8194.patch, CacheTest2.java


 We use PasswordAuthenticator and PasswordAuthorizer. The system_auth has a RF 
 of 10 per DC over 2 DCs. The permissions_validity_in_ms is 5 minutes. 
 We still have few thousand requests failing each day with the trace below. 
 The reason for this is read cache request realizing that cached entry has 
 expired and doing a blocking request to refresh cache. 
 We should have cache refreshed periodically only in the back ground. The user 
 request should simply look at the cache and not try to refresh it. 
 com.google.common.util.concurrent.UncheckedExecutionException: 
 java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
 received only 0 responses.
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
   at 
 com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
   at 
 org.apache.cassandra.service.ClientState.authorize(ClientState.java:292)
   at 
 org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:172)
   at 
 org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:165)
   at 
 org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:149)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:75)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:102)
   at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:113)
   at 
 org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1735)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4162)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4150)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:206)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 Caused by: java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
 received only 0 responses.
   at org.apache.cassandra.auth.Auth.selectUser(Auth.java:256)
   at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:84)
   at 
 org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
   at 
 org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:68)
   at org.apache.cassandra.service.ClientState$1.load(ClientState.java:278)
   at org.apache.cassandra.service.ClientState$1.load(ClientState.java:275)
   at 
 com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
   at 
 com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
   at 
 com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
   ... 19 more
 Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation 
 timed out - received only 0 responses.
   at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
   at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:943)
   at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:828)
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:140)
   at org.apache.cassandra.auth.Auth.selectUser(Auth.java:245)
   ... 28 more
 ERROR [Thrift:17232] 2014-10-24 05:06:51,004 CustomTThreadPoolServer.java 
 (line 224) Error occurred during processing of message.

[jira] [Updated] (CASSANDRA-6983) DirectoriesTest fails when run as root

2015-01-06 Thread Alan Boudreault (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault updated CASSANDRA-6983:
---
Tester: Alan Boudreault

 DirectoriesTest fails when run as root
 --

 Key: CASSANDRA-6983
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6983
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor
 Fix For: 2.0.12


 When you run the DirectoriesTest as a normal user, it passes because it fails 
 to create the 'bad' directory:
 {noformat}
 [junit] - Standard Error -
 [junit] ERROR 16:16:18,111 Failed to create 
 /tmp/cassandra4119802552776680052unittest/ks/bad directory
 [junit]  WARN 16:16:18,112 Blacklisting 
 /tmp/cassandra4119802552776680052unittest/ks/bad for writes
 [junit] -  ---
 {noformat}
 But when you run the test as root, it succeeds in making the directory, 
 causing an assertion failure that it's unwritable:
 {noformat}
 [junit] Testcase: 
 testDiskFailurePolicy_best_effort(org.apache.cassandra.db.DirectoriesTest):   
 FAILED
 [junit] 
 [junit] junit.framework.AssertionFailedError: 
 [junit] at 
 org.apache.cassandra.db.DirectoriesTest.testDiskFailurePolicy_best_effort(DirectoriesTest.java:199)
 {noformat}
 It seems to me that we shouldn't be relying on failing the make the 
 directory.  If we're just going to test a nonexistent dir, why try to make 
 one at all?  And if that is supposed to succeed, then we have a problem with 
 either the test or blacklisting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6983) DirectoriesTest fails when run as root

2015-01-06 Thread Alan Boudreault (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alan Boudreault updated CASSANDRA-6983:
---
Attachment: 6983-v1.patch

As a non-root user perspective, the test is ok. We have to call the
Directories.create() if we want to get the internal cassandra disk policy
mechanism triggered. If we remove that fake directory creation, the test will
fail since the it is not marked as an unwritable BlacklistedDirectory.

The real issue with that test using the root user is that we cannot set the
directory unwritable using Java. Internally, Java will simply re-set the write
permission for the operation since the root user is allowed to do everything.
Please correct me if I'm wrong here about that Java behavior.

At this point, we have two options to fix this test:

1. Do not call Directories.create(), but rather set the directory unwritable
manually via the BlacklistedDirectories class. I don't especially like this
option since it doesn't really trigger the best_effort internal mechanism.

2. Simulate a fake directory creation by throwing an exception. This will
trigger the internal handleFSError and the best_effort mechanism. This will
work for the root user since we are not playing anymore with the filesystem.
I've attached a tentative patch with this solution.

Let me know what you think or if you have any suggestion.

DirectoriesTest fails when run as root
--

Key: CASSANDRA-6983
URL: https://issues.apache.org/jira/browse/CASSANDRA-6983
Project: Cassandra
Issue Type: Bug
Components: Tests
Reporter: Brandon Williams
Assignee: Ryan McGuire
Priority: Minor
Fix For: 2.0.12

Attachments: 6983-v1.patch

When you run the DirectoriesTest as a normal user, it passes because it fails
to create the 'bad' directory:
{noformat}
[junit] - Standard Error -
[junit] ERROR 16:16:18,111 Failed to create
/tmp/cassandra4119802552776680052unittest/ks/bad directory
[junit] WARN 16:16:18,112 Blacklisting
/tmp/cassandra4119802552776680052unittest/ks/bad for writes
[junit] - ---
{noformat}
But when you run the test as root, it succeeds in making the directory,
causing an assertion failure that it's unwritable:
{noformat}
[junit] Testcase:
testDiskFailurePolicy_best_effort(org.apache.cassandra.db.DirectoriesTest):
FAILED
[junit]
[junit] junit.framework.AssertionFailedError:
[junit] at
org.apache.cassandra.db.DirectoriesTest.testDiskFailurePolicy_best_effort(DirectoriesTest.java:199)
{noformat}
It seems to me that we shouldn't be relying on failing the make the
directory. If we're just going to test a nonexistent dir, why try to make
one at all? And if that is supposed to succeed, then we have a problem with
either the test or blacklisting.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Use lexical comparison for java revisions

2015-01-06 Thread brandonwilliams

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 1af8ea5ef - e83304ddf
  refs/heads/trunk dc0102a8e - 0f024a619


Use lexical comparison for java revisions

Patch by Michael Shuler, reviewed by brandonwilliams for CASSANDRA-8315


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e83304dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e83304dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e83304dd

Branch: refs/heads/cassandra-2.1
Commit: e83304ddfee1377417b334d393c61671895d8ed5
Parents: 1af8ea5
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 6 08:24:48 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 6 08:24:48 2015 -0600

--
 conf/cassandra-env.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e83304dd/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 191fb7e..755f962 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -99,7 +99,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 exit 1;
 fi
 
-if [ $JVM_VERSION \ 1.8 ]  [ $JVM_PATCH_VERSION -lt 25 ] ; then
+if [ $JVM_VERSION \ 1.8 ]  [ $JVM_PATCH_VERSION \ 25 ] ; then
 echo Cassandra 2.0 and later require Java 7u25 or later.
 exit 1;
 fi

[2/3] cassandra git commit: Use lexical comparison for java revisions

2015-01-06 Thread brandonwilliams

Use lexical comparison for java revisions

Patch by Michael Shuler, reviewed by brandonwilliams for CASSANDRA-8315


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e83304dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e83304dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e83304dd

Branch: refs/heads/trunk
Commit: e83304ddfee1377417b334d393c61671895d8ed5
Parents: 1af8ea5
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 6 08:24:48 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 6 08:24:48 2015 -0600

--
 conf/cassandra-env.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e83304dd/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 191fb7e..755f962 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -99,7 +99,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 exit 1;
 fi
 
-if [ $JVM_VERSION \ 1.8 ]  [ $JVM_PATCH_VERSION -lt 25 ] ; then
+if [ $JVM_VERSION \ 1.8 ]  [ $JVM_PATCH_VERSION \ 25 ] ; then
 echo Cassandra 2.0 and later require Java 7u25 or later.
 exit 1;
 fi

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-01-06 Thread brandonwilliams

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f024a61
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f024a61
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f024a61

Branch: refs/heads/trunk
Commit: 0f024a6192f185b93439ae53a69ce113b363
Parents: dc0102a e83304d
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 6 08:25:56 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 6 08:25:56 2015 -0600

--
 conf/cassandra-env.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f024a61/conf/cassandra-env.sh
--

[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries

2015-01-06 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266101#comment-14266101
 ] 

Sam Tunnicliffe commented on CASSANDRA-8303:


It seems to me that this could be a reasonably natural fit as an extension to 
the existing authz system. If we do go down that route there are a couple of 
things to consider. 

What we're really talking about here are restrictions on the permissions a 
particular user may have been granted. For instance, you could grant SELECT on 
ks1.cf1 to user foo, but add a restriction such that multipartition queries are 
not allowed. My first question then is are there other restrictions that would 
be useful but which don't fit this definition?

For restrictions which do fit that definition, it seems sensible to use a 
similar resource hierarchy as permissions. i.e. a restriction can be applied at 
the table level, keyspace level (which trickles down to apply to all tables in 
the ks), or at the root level. The alternative implied by the syntax in the 
previous comments suggests that a restriction should always be applied at the 
root level (i.e. for all keyspaces  tables). 

How inheritance is handled in that hierarchy of resources should be from the 
bottom upwards, working from the most specific permission/restriction.For 
instance, in the case where user x is granted SELECT without any restriction on 
ks1 and the same user is also granted SELECT with the MULTI_PARTITION 
restriction on ks1.cf1 it seems clear that MP queries from user x should only 
be disallowed on the one table, not all tables in ks1. Also, the unrestricted 
grant should not trump the more specific, restricted one and allow MP queries 
on that table.
In the reverse scenario, where the restriction is at the keyspace level, but 
there is also a non-restricted grant for the table, the behaviour should also 
be reversed, user x can make MP queries on the 1 table, but on no others in the 
keyspace.

As noted, managing all this at the user level could be overly burdensome but 
CASSANDRA-7653 should relieve most of that extra administrative load. However, 
it would involve some additional complexity in handling resolution of 
permissions and restrictions.

During the authz process when we resolve permissions for a user on a particular 
resource, the most specific permission whether granted directly or inherited 
through role membership should apply, along with any of its restrictions. If 
there are multiple inherited or direct grants for the exact same resource (i.e. 
at the same level in the resource hierarchy) then we would merge them, unioning 
any restrictions, e.g

{noformat}
role x is granted SELECT on ks1 with MULTI_PARTITION restriction
role y is granted SELECT on ks1.cf1, unrestricted
role y should be allowed to SELECT from any table in ks1, but perform MP 
queries on ks1.cf1 only 

role z is granted both role x and role y
role z should be allowed to SELECT from any table in ks1, but perform MP 
queries on ks1.cf1 only 

role a is granted SELECT on ks1, unrestricted
role z is granted roles x, y, a
role z should be allowed to SELECT from any table in ks1, but perform MP 
queries on ks1.cf1 only 
{noformat}

So, I think this could fit nicely into authz for 3.0 but we may also want to 
think about adding options to cassandra.yaml to enforce the same restrictions 
without requiring authz ( authn). Obviously, that would be much more of a 
blunt instrument - enforcing a given restriction for all queries when enabled, 
though it would mean that it could be turned on on a per-node/per-DC basis. I 
don't see a problem with supporting both cql and yaml based approaches so long 
as we define an order of precedence.

 Provide strict mode for CQL Queries
 -

 Key: CASSANDRA-8303
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303
 Project: Cassandra
  Issue Type: Improvement
Reporter: Anupam Arora
 Fix For: 3.0


 Please provide a strict mode option in cassandra that will kick out any CQL 
 queries that are expensive, e.g. any query with ALLOWS FILTERING, 
 multi-partition queries, secondary index queries, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6983) DirectoriesTest fails when run as root

2015-01-06 Thread Brandon Williams (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6983:

Reviewer: Yuki Morishita
Assignee: Alan Boudreault  (was: Ryan McGuire)

 DirectoriesTest fails when run as root
 --

 Key: CASSANDRA-6983
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6983
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Brandon Williams
Assignee: Alan Boudreault
Priority: Minor
 Fix For: 2.0.12

 Attachments: 6983-v1.patch


 When you run the DirectoriesTest as a normal user, it passes because it fails 
 to create the 'bad' directory:
 {noformat}
 [junit] - Standard Error -
 [junit] ERROR 16:16:18,111 Failed to create 
 /tmp/cassandra4119802552776680052unittest/ks/bad directory
 [junit]  WARN 16:16:18,112 Blacklisting 
 /tmp/cassandra4119802552776680052unittest/ks/bad for writes
 [junit] -  ---
 {noformat}
 But when you run the test as root, it succeeds in making the directory, 
 causing an assertion failure that it's unwritable:
 {noformat}
 [junit] Testcase: 
 testDiskFailurePolicy_best_effort(org.apache.cassandra.db.DirectoriesTest):   
 FAILED
 [junit] 
 [junit] junit.framework.AssertionFailedError: 
 [junit] at 
 org.apache.cassandra.db.DirectoriesTest.testDiskFailurePolicy_best_effort(DirectoriesTest.java:199)
 {noformat}
 It seems to me that we shouldn't be relying on failing the make the 
 directory.  If we're just going to test a nonexistent dir, why try to make 
 one at all?  And if that is supposed to succeed, then we have a problem with 
 either the test or blacklisting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out

2015-01-06 Thread Brandon Williams (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8558:

Priority: Blocker  (was: Major)

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Philip Thompson
Priority: Blocker
 Fix For: 2.1.3


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7653) Add role based access control to Cassandra

2015-01-06 Thread Mike Adamson (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266182#comment-14266182
]

Mike Adamson commented on CASSANDRA-7653:
-

I have a couple of initial thoughts on this.
# Is the IAuthenticator.constructInitialSaslToken method really necessary? The
only usage of this is from login methods that are only going to use the plain
text sasl implementation offered by the PasswordAuthenticator so they could
build the initial token themselves.
# Is there any way of not having the Option enum? This fixes the options that
an Authenticator can support and doesn't allow any 3rd party to have different
options but still work with the CQL grammar. Could we have some similar to the
replication strategies? Or perhaps keep the Option enum but have an option of
EXTENSION (or other name) that would allow the passing in of a json set of
extension options.

Add role based access control to Cassandra
--

Key: CASSANDRA-7653
URL: https://issues.apache.org/jira/browse/CASSANDRA-7653
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Mike Adamson
Assignee: Sam Tunnicliffe
Fix For: 3.0

Attachments: 7653.patch, CQLSmokeTest.java, cql_smoke_test.py

The current authentication model supports granting permissions to individual
users. While this is OK for small or medium organizations wanting to
implement authorization, it does not work well in large organizations because
of the overhead of having to maintain the permissions for each user.
Introducing roles into the authentication model would allow sets of
permissions to be controlled in one place as a role and then the role granted
to users. Roles should also be able to be granted to other roles to allow
hierarchical sets of permissions to be built up.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8448) Comparison method violates its general contract in AbstractEndpointSnitch

2015-01-06 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266191#comment-14266191
 ] 

Brandon Williams commented on CASSANDRA-8448:
-

ISTM we already do that:

{noformat}
ArrayListDouble sortedScores = new 
ArrayList(subsnitchOrderedScores);
Collections.sort(sortedScores);

IteratorDouble sortedScoreIterator = sortedScores.iterator();
for (Double subsnitchScore : subsnitchOrderedScores)
{
if (subsnitchScore  (sortedScoreIterator.next() * (1.0 + 
BADNESS_THRESHOLD)))
{
sortByProximityWithScore(address, addresses);
return;
}
}
{noformat}

So I'm not sure what to do (and have no clear repro steps so I can't even try.) 
A simple workaround would be disabling badness, though.

 Comparison method violates its general contract in AbstractEndpointSnitch
 ---

 Key: CASSANDRA-8448
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8448
 Project: Cassandra
  Issue Type: Bug
Reporter: J.B. Langston
Assignee: Brandon Williams

 Seen in both 1.2 and 2.0.  The error is occurring here: 
 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/locator/AbstractEndpointSnitch.java#L49
 {code}
 ERROR [Thrift:9] 2014-12-04 20:12:28,732 CustomTThreadPoolServer.java (line 
 219) Error occurred during processing of message.
 com.google.common.util.concurrent.UncheckedExecutionException: 
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2199)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3936)
   at 
 com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4806)
   at 
 org.apache.cassandra.service.ClientState.authorize(ClientState.java:352)
   at 
 org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:224)
   at 
 org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:218)
   at 
 org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:202)
   at 
 org.apache.cassandra.thrift.CassandraServer.createMutationList(CassandraServer.java:822)
   at 
 org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:954)
   at com.datastax.bdp.server.DseServer.batch_mutate(DseServer.java:576)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3922)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3906)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.IllegalArgumentException: Comparison method violates its 
 general contract!
   at java.util.TimSort.mergeHi(TimSort.java:868)
   at java.util.TimSort.mergeAt(TimSort.java:485)
   at java.util.TimSort.mergeCollapse(TimSort.java:410)
   at java.util.TimSort.sort(TimSort.java:214)
   at java.util.TimSort.sort(TimSort.java:173)
   at java.util.Arrays.sort(Arrays.java:659)
   at java.util.Collections.sort(Collections.java:217)
   at 
 org.apache.cassandra.locator.AbstractEndpointSnitch.sortByProximity(AbstractEndpointSnitch.java:49)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithScore(DynamicEndpointSnitch.java:157)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithBadness(DynamicEndpointSnitch.java:186)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximity(DynamicEndpointSnitch.java:151)
   at 
 org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1408)
   at 
 org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1402)
   at 
 org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:148)
   at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1223)
   at 
 org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1165)
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:255)

1 2 >

1 - 100 of 159 matches

Mail list logo