[jira] [Created] (CASSANDRA-6452) Cassandra native java driver - can't retrieve list of schema indexes
David Semeria created CASSANDRA-6452: Summary: Cassandra native java driver - can't retrieve list of schema indexes Key: CASSANDRA-6452 URL: https://issues.apache.org/jira/browse/CASSANDRA-6452 Project: Cassandra Issue Type: Bug Reporter: David Semeria Priority: Minor Currently it's possible to query the schema to retrieve the list of tables and the columns within those tables, but it doesn't appear to be possible to do so for indexes. The possibility to create and update the schema from code is very useful - especially when there is a requirement to maintain multiple parallel environments (testing, staging, production, etc). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Issue Comment Deleted] (CASSANDRA-6418) auto_snapshots are not removable via 'nodetool clearsnapshot'
[ https://issues.apache.org/jira/browse/CASSANDRA-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-6418: -- Comment: was deleted (was: Not sure why my ant missed that build problem. Added Keyspace#clearCfSnapshot that takes an extra cfName argument, and also the above nits are tackled in v2. ) auto_snapshots are not removable via 'nodetool clearsnapshot' - Key: CASSANDRA-6418 URL: https://issues.apache.org/jira/browse/CASSANDRA-6418 Project: Cassandra Issue Type: Bug Components: Tools Environment: auto_snapshot: true Reporter: J. Ryan Earl Assignee: Lyuben Todorov Priority: Minor Fix For: 2.0.4 Attachments: 6418_cassandra-2.0.patch Snapshots of deleted CFs created via the auto_snapshot configuration parameter appear to not be tracked. The result is that 'nodetool clearsnapshot keyspace with deleted CFs' does nothing, and short of manually removing the files from the filesystem, deleted CFs remain indefinitely taking up space. I'm not sure if this is intended, but it seems pretty counter-intuitive. I haven't found any documentation that indicates auto_snapshots would be ignored by 'nodetool clearsnapshot'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6418) auto_snapshots are not removable via 'nodetool clearsnapshot'
[ https://issues.apache.org/jira/browse/CASSANDRA-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-6418: -- Attachment: (was: 6418_v2.patch) auto_snapshots are not removable via 'nodetool clearsnapshot' - Key: CASSANDRA-6418 URL: https://issues.apache.org/jira/browse/CASSANDRA-6418 Project: Cassandra Issue Type: Bug Components: Tools Environment: auto_snapshot: true Reporter: J. Ryan Earl Assignee: Lyuben Todorov Priority: Minor Fix For: 2.0.4 Attachments: 6418_cassandra-2.0.patch Snapshots of deleted CFs created via the auto_snapshot configuration parameter appear to not be tracked. The result is that 'nodetool clearsnapshot keyspace with deleted CFs' does nothing, and short of manually removing the files from the filesystem, deleted CFs remain indefinitely taking up space. I'm not sure if this is intended, but it seems pretty counter-intuitive. I haven't found any documentation that indicates auto_snapshots would be ignored by 'nodetool clearsnapshot'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6418) auto_snapshots are not removable via 'nodetool clearsnapshot'
[ https://issues.apache.org/jira/browse/CASSANDRA-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-6418: -- Attachment: 6418_v2.patch Not sure why my ant missed that build problem. Added Keyspace#clearCfSnapshot that takes an extra cfName argument, and also the above nits are tackled in v2. auto_snapshots are not removable via 'nodetool clearsnapshot' - Key: CASSANDRA-6418 URL: https://issues.apache.org/jira/browse/CASSANDRA-6418 Project: Cassandra Issue Type: Bug Components: Tools Environment: auto_snapshot: true Reporter: J. Ryan Earl Assignee: Lyuben Todorov Priority: Minor Fix For: 2.0.4 Attachments: 6418_cassandra-2.0.patch Snapshots of deleted CFs created via the auto_snapshot configuration parameter appear to not be tracked. The result is that 'nodetool clearsnapshot keyspace with deleted CFs' does nothing, and short of manually removing the files from the filesystem, deleted CFs remain indefinitely taking up space. I'm not sure if this is intended, but it seems pretty counter-intuitive. I haven't found any documentation that indicates auto_snapshots would be ignored by 'nodetool clearsnapshot'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6418) auto_snapshots are not removable via 'nodetool clearsnapshot'
[ https://issues.apache.org/jira/browse/CASSANDRA-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-6418: -- Attachment: 6418_v2.patch v2 adds a CFS#clearSnapshot function, and addresses all the nits. auto_snapshots are not removable via 'nodetool clearsnapshot' - Key: CASSANDRA-6418 URL: https://issues.apache.org/jira/browse/CASSANDRA-6418 Project: Cassandra Issue Type: Bug Components: Tools Environment: auto_snapshot: true Reporter: J. Ryan Earl Assignee: Lyuben Todorov Priority: Minor Fix For: 2.0.4 Attachments: 6418_cassandra-2.0.patch, 6418_v2.patch Snapshots of deleted CFs created via the auto_snapshot configuration parameter appear to not be tracked. The result is that 'nodetool clearsnapshot keyspace with deleted CFs' does nothing, and short of manually removing the files from the filesystem, deleted CFs remain indefinitely taking up space. I'm not sure if this is intended, but it seems pretty counter-intuitive. I haven't found any documentation that indicates auto_snapshots would be ignored by 'nodetool clearsnapshot'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6413) Saved KeyCache prints success to log; but no file present
[
https://issues.apache.org/jira/browse/CASSANDRA-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840106#comment-13840106
]
Chris Burroughs commented on CASSANDRA-6413:
Jira's priority guide thingy for minor says easy workaround is present. Is
there a way to get both caches to persist before upgrading with the fix?
Saved KeyCache prints success to log; but no file present
-
Key: CASSANDRA-6413
URL: https://issues.apache.org/jira/browse/CASSANDRA-6413
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 1.2.11
Reporter: Chris Burroughs
Assignee: Mikhail Stepura
Priority: Minor
Fix For: 1.2.13, 2.0.4
Attachments: 6413-v2.txt, CASSANDRA-1.2-6413.patch
Cluster has a single keyspace with 3 CFs. All used to have ROWS_ONLY, two
were switched to KEYS_ONLY about 2 days ago. Row cache continues to save
fine, but there is no saved key cache file present on any node in the cluster.
{noformat}
6925: INFO [CompactionExecutor:12] 2013-11-27 10:12:02,284
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 118 ms
6941:DEBUG [CompactionExecutor:14] 2013-11-27 10:17:02,163
AutoSavingCache.java (line 233) Deleting old RowCache files.
6942: INFO [CompactionExecutor:14] 2013-11-27 10:17:02,310
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 146 ms
8745:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,140
AutoSavingCache.java (line 233) Deleting old RowCache files.
8746: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,283
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 143 ms
8747:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,283
AutoSavingCache.java (line 233) Deleting old KeyCache files.
8748: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,625
AutoSavingCache.java (line 289) Saved KeyCache (21181 items) in 342 ms
8749:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,625
AutoSavingCache.java (line 233) Deleting old RowCache files.
8750: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,759
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 134 ms
8751:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,759
AutoSavingCache.java (line 233) Deleting old RowCache files.
8752: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,893
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 133 ms
8753:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,893
AutoSavingCache.java (line 233) Deleting old RowCache files.
8754: INFO [CompactionExecutor:6] 2013-11-27 10:37:26,026
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 133 ms
9915:DEBUG [CompactionExecutor:18] 2013-11-27 10:42:01,851
AutoSavingCache.java (line 233) Deleting old KeyCache files.
9916: INFO [CompactionExecutor:18] 2013-11-27 10:42:02,185
AutoSavingCache.java (line 289) Saved KeyCache (22067 items) in 334 ms
9917:DEBUG [CompactionExecutor:17] 2013-11-27 10:42:02,279
AutoSavingCache.java (line 233) Deleting old RowCache files.
9918: INFO [CompactionExecutor:17] 2013-11-27 10:42:02,411
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 131 ms
{noformat}
{noformat}
$ ll ~/shared/saved_caches/
total 3472
-rw-rw-r-- 1 cassandra cassandra 3551608 Nov 27 10:42 Foo-Bar-RowCache-b.db
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (CASSANDRA-5832) DELETE CAS on 'primary key only' table
[ https://issues.apache.org/jira/browse/CASSANDRA-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-5832. -- Resolution: Duplicate DELETE CAS on 'primary key only' table -- Key: CASSANDRA-5832 URL: https://issues.apache.org/jira/browse/CASSANDRA-5832 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 2.0 beta 2 Reporter: Blair Zajac Priority: Minor Following up on the CAS on 'primary key only' table issue [1] which added support for atomically creating a primary key only table, this ticket is requesting support for a CAS DELETE of a row from such a table. Currently these two different statements fail: Trying DELETE FROM test1 WHERE k = 456 IF EXISTS using cassandra-dbapi2: cql.apivalues.ProgrammingError: Bad Request: line 0:-1 no viable alternative at input 'EOF' Trying DELETE FROM test1 WHERE k = 456 IF k = 456 cql.apivalues.ProgrammingError: Bad Request: PRIMARY KEY part k found in SET part [1] https://issues.apache.org/jira/browse/CASSANDRA-5715 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6413) Saved KeyCache prints success to log; but no file present
[
https://issues.apache.org/jira/browse/CASSANDRA-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840186#comment-13840186
]
Jonathan Ellis commented on CASSANDRA-6413:
---
No. Minor also means doesn't seriously affect cluster stability or ability
to respond to client requests.
Saved KeyCache prints success to log; but no file present
-
Key: CASSANDRA-6413
URL: https://issues.apache.org/jira/browse/CASSANDRA-6413
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 1.2.11
Reporter: Chris Burroughs
Assignee: Mikhail Stepura
Priority: Minor
Fix For: 1.2.13, 2.0.4
Attachments: 6413-v2.txt, CASSANDRA-1.2-6413.patch
Cluster has a single keyspace with 3 CFs. All used to have ROWS_ONLY, two
were switched to KEYS_ONLY about 2 days ago. Row cache continues to save
fine, but there is no saved key cache file present on any node in the cluster.
{noformat}
6925: INFO [CompactionExecutor:12] 2013-11-27 10:12:02,284
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 118 ms
6941:DEBUG [CompactionExecutor:14] 2013-11-27 10:17:02,163
AutoSavingCache.java (line 233) Deleting old RowCache files.
6942: INFO [CompactionExecutor:14] 2013-11-27 10:17:02,310
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 146 ms
8745:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,140
AutoSavingCache.java (line 233) Deleting old RowCache files.
8746: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,283
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 143 ms
8747:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,283
AutoSavingCache.java (line 233) Deleting old KeyCache files.
8748: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,625
AutoSavingCache.java (line 289) Saved KeyCache (21181 items) in 342 ms
8749:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,625
AutoSavingCache.java (line 233) Deleting old RowCache files.
8750: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,759
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 134 ms
8751:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,759
AutoSavingCache.java (line 233) Deleting old RowCache files.
8752: INFO [CompactionExecutor:6] 2013-11-27 10:37:25,893
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 133 ms
8753:DEBUG [CompactionExecutor:6] 2013-11-27 10:37:25,893
AutoSavingCache.java (line 233) Deleting old RowCache files.
8754: INFO [CompactionExecutor:6] 2013-11-27 10:37:26,026
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 133 ms
9915:DEBUG [CompactionExecutor:18] 2013-11-27 10:42:01,851
AutoSavingCache.java (line 233) Deleting old KeyCache files.
9916: INFO [CompactionExecutor:18] 2013-11-27 10:42:02,185
AutoSavingCache.java (line 289) Saved KeyCache (22067 items) in 334 ms
9917:DEBUG [CompactionExecutor:17] 2013-11-27 10:42:02,279
AutoSavingCache.java (line 233) Deleting old RowCache files.
9918: INFO [CompactionExecutor:17] 2013-11-27 10:42:02,411
AutoSavingCache.java (line 289) Saved RowCache (5 items) in 131 ms
{noformat}
{noformat}
$ ll ~/shared/saved_caches/
total 3472
-rw-rw-r-- 1 cassandra cassandra 3551608 Nov 27 10:42 Foo-Bar-RowCache-b.db
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[1/6] git commit: Fix cache persistence when both row and key cache are enabled patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Updated Branches:
refs/heads/cassandra-1.2 e40bc759c - 57613dc8c
refs/heads/cassandra-2.0 cf83c81d8 - 667e3db91
refs/heads/trunk 6772247f8 - 5ad8a3d3a
Fix cache persistence when both row and key cache are enabled
patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57613dc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57613dc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57613dc8
Branch: refs/heads/cassandra-1.2
Commit: 57613dc8c14d7213ee2d46d41bd642dd7cf697ab
Parents: e40bc75
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Dec 5 09:42:41 2013 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Dec 5 09:42:41 2013 -0600
--
CHANGES.txt | 5 -
.../apache/cassandra/cache/AutoSavingCache.java | 22
2 files changed, 13 insertions(+), 14 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 09a3b07..166c566 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,7 +2,8 @@
* Fix thundering herd on endpoint cache invalidation (CASSANDRA-6345)
* Optimize FD phi calculation (CASSANDRA-6386)
* Improve initial FD phi estimate when starting up (CASSANDRA-6385)
- * Don't list CQL3 table in CLI describe even if named explicitely
(CASSANDRA-5750)
+ * Don't list CQL3 table in CLI describe even if named explicitely
+ (CASSANDRA-5750)
* cqlsh: quote single quotes in strings inside collections (CASSANDRA-6172)
* Improve gossip performance for typical messages (CASSANDRA-6409)
* Throw IRE if a prepared statement has more markers than supported
@@ -11,6 +12,8 @@
* Change snapshot response message verb to INTERNAL to avoid dropping it
(CASSANDRA-6415)
* Warn when collection read has 65K elements (CASSANDRA-5428)
+ * Fix cache persistence when both row and key cache are enabled
+ (CASSANDRA-6413)
1.2.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index d6a37df..a649e88 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -299,22 +299,18 @@ public class AutoSavingCacheK extends CacheKey, V
extends InstrumentingCacheK
private void deleteOldCacheFiles()
{
File savedCachesDir = new
File(DatabaseDescriptor.getSavedCachesLocation());
+assert savedCachesDir.exists() savedCachesDir.isDirectory();
-if (savedCachesDir.exists() savedCachesDir.isDirectory())
+for (File file : savedCachesDir.listFiles())
{
-for (File file : savedCachesDir.listFiles())
-{
-if (file.isFile()
file.getName().endsWith(cacheType.toString()))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (!file.isFile())
+continue; // someone's been messing with our directory.
naughty!
-if (file.isFile()
file.getName().endsWith(CURRENT_VERSION + .db))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (file.getName().endsWith(cacheType.toString())
+|| file.getName().endsWith(String.format(%s-%s.db,
cacheType.toString(), CURRENT_VERSION)))
+{
+if (!file.delete())
+logger.warn(Failed to delete {},
file.getAbsolutePath());
}
}
}
[6/6] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ad8a3d3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ad8a3d3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ad8a3d3 Branch: refs/heads/trunk Commit: 5ad8a3d3aa94b864f72ca587d9117f1458a47f34 Parents: 6772247 667e3db Author: Jonathan Ellis jbel...@apache.org Authored: Thu Dec 5 09:44:18 2013 -0600 Committer: Jonathan Ellis jbel...@apache.org Committed: Thu Dec 5 09:44:18 2013 -0600 -- CHANGES.txt | 5 - .../apache/cassandra/cache/AutoSavingCache.java | 22 2 files changed, 13 insertions(+), 14 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ad8a3d3/CHANGES.txt --
[5/6] git commit: merge from 1.2
merge from 1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/667e3db9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/667e3db9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/667e3db9 Branch: refs/heads/cassandra-2.0 Commit: 667e3db91cbec9928857aa89bfcc63fe85cb558d Parents: cf83c81 57613dc Author: Jonathan Ellis jbel...@apache.org Authored: Thu Dec 5 09:44:10 2013 -0600 Committer: Jonathan Ellis jbel...@apache.org Committed: Thu Dec 5 09:44:10 2013 -0600 -- CHANGES.txt | 5 - .../apache/cassandra/cache/AutoSavingCache.java | 22 2 files changed, 13 insertions(+), 14 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/667e3db9/CHANGES.txt -- diff --cc CHANGES.txt index d485a69,166c566..8996965 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -16,44 -12,11 +16,47 @@@ Merged from 1.2 * Change snapshot response message verb to INTERNAL to avoid dropping it (CASSANDRA-6415) * Warn when collection read has 65K elements (CASSANDRA-5428) + * Fix cache persistence when both row and key cache are enabled +(CASSANDRA-6413) -1.2.12 +2.0.3 + * Fix FD leak on slice read path (CASSANDRA-6275) + * Cancel read meter task when closing SSTR (CASSANDRA-6358) + * free off-heap IndexSummary during bulk (CASSANDRA-6359) + * Recover from IOException in accept() thread (CASSANDRA-6349) + * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338) + * Fix trying to hint timed out counter writes (CASSANDRA-6322) + * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809) + * Avoid flushing compaction_history after each operation (CASSANDRA-6287) + * Fix repair assertion error when tombstones expire (CASSANDRA-6277) + * Skip loading corrupt key cache (CASSANDRA-6260) + * Fixes for compacting larger-than-memory rows (CASSANDRA-6274) + * Compact hottest sstables first and optionally omit coldest from + compaction entirely (CASSANDRA-6109) + * Fix modifying column_metadata from thrift (CASSANDRA-6182) + * cqlsh: fix LIST USERS output (CASSANDRA-6242) + * Add IRequestSink interface (CASSANDRA-6248) + * Update memtable size while flushing (CASSANDRA-6249) + * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252) + * Require Permission.SELECT for CAS updates (CASSANDRA-6247) + * New CQL-aware SSTableWriter (CASSANDRA-5894) + * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270) + * Correctly throw error when frame too large (CASSANDRA-5981) + * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) + * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) + * Fix paging with reversed slices (CASSANDRA-6343) + * Set minTimestamp correctly to be able to drop expired sstables (CASSANDRA-6337) + * Support NaN and Infinity as float literals (CASSANDRA-6003) + * Remove RF from nodetool ring output (CASSANDRA-6289) + * Fix attempting to flush empty rows (CASSANDRA-6374) + * Fix potential out of bounds exception when paging (CASSANDRA-6333) +Merged from 1.2: + * Optimize FD phi calculation (CASSANDRA-6386) + * Improve initial FD phi estimate when starting up (CASSANDRA-6385) - * Don't list CQL3 table in CLI describe even if named explicitely (CASSANDRA-5750) ++ * Don't list CQL3 table in CLI describe even if named explicitely ++ (CASSANDRA-5750) * Invalidate row cache when dropping CF (CASSANDRA-6351) * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/667e3db9/src/java/org/apache/cassandra/cache/AutoSavingCache.java --
[2/6] git commit: Fix cache persistence when both row and key cache are enabled patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Fix cache persistence when both row and key cache are enabled
patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57613dc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57613dc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57613dc8
Branch: refs/heads/cassandra-2.0
Commit: 57613dc8c14d7213ee2d46d41bd642dd7cf697ab
Parents: e40bc75
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Dec 5 09:42:41 2013 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Dec 5 09:42:41 2013 -0600
--
CHANGES.txt | 5 -
.../apache/cassandra/cache/AutoSavingCache.java | 22
2 files changed, 13 insertions(+), 14 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 09a3b07..166c566 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,7 +2,8 @@
* Fix thundering herd on endpoint cache invalidation (CASSANDRA-6345)
* Optimize FD phi calculation (CASSANDRA-6386)
* Improve initial FD phi estimate when starting up (CASSANDRA-6385)
- * Don't list CQL3 table in CLI describe even if named explicitely
(CASSANDRA-5750)
+ * Don't list CQL3 table in CLI describe even if named explicitely
+ (CASSANDRA-5750)
* cqlsh: quote single quotes in strings inside collections (CASSANDRA-6172)
* Improve gossip performance for typical messages (CASSANDRA-6409)
* Throw IRE if a prepared statement has more markers than supported
@@ -11,6 +12,8 @@
* Change snapshot response message verb to INTERNAL to avoid dropping it
(CASSANDRA-6415)
* Warn when collection read has 65K elements (CASSANDRA-5428)
+ * Fix cache persistence when both row and key cache are enabled
+ (CASSANDRA-6413)
1.2.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index d6a37df..a649e88 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -299,22 +299,18 @@ public class AutoSavingCacheK extends CacheKey, V
extends InstrumentingCacheK
private void deleteOldCacheFiles()
{
File savedCachesDir = new
File(DatabaseDescriptor.getSavedCachesLocation());
+assert savedCachesDir.exists() savedCachesDir.isDirectory();
-if (savedCachesDir.exists() savedCachesDir.isDirectory())
+for (File file : savedCachesDir.listFiles())
{
-for (File file : savedCachesDir.listFiles())
-{
-if (file.isFile()
file.getName().endsWith(cacheType.toString()))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (!file.isFile())
+continue; // someone's been messing with our directory.
naughty!
-if (file.isFile()
file.getName().endsWith(CURRENT_VERSION + .db))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (file.getName().endsWith(cacheType.toString())
+|| file.getName().endsWith(String.format(%s-%s.db,
cacheType.toString(), CURRENT_VERSION)))
+{
+if (!file.delete())
+logger.warn(Failed to delete {},
file.getAbsolutePath());
}
}
}
[3/6] git commit: Fix cache persistence when both row and key cache are enabled patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Fix cache persistence when both row and key cache are enabled
patch by Mikhail Stepura and jbellis for CASSANDRA-6413
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57613dc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57613dc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57613dc8
Branch: refs/heads/trunk
Commit: 57613dc8c14d7213ee2d46d41bd642dd7cf697ab
Parents: e40bc75
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Dec 5 09:42:41 2013 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Dec 5 09:42:41 2013 -0600
--
CHANGES.txt | 5 -
.../apache/cassandra/cache/AutoSavingCache.java | 22
2 files changed, 13 insertions(+), 14 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 09a3b07..166c566 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,7 +2,8 @@
* Fix thundering herd on endpoint cache invalidation (CASSANDRA-6345)
* Optimize FD phi calculation (CASSANDRA-6386)
* Improve initial FD phi estimate when starting up (CASSANDRA-6385)
- * Don't list CQL3 table in CLI describe even if named explicitely
(CASSANDRA-5750)
+ * Don't list CQL3 table in CLI describe even if named explicitely
+ (CASSANDRA-5750)
* cqlsh: quote single quotes in strings inside collections (CASSANDRA-6172)
* Improve gossip performance for typical messages (CASSANDRA-6409)
* Throw IRE if a prepared statement has more markers than supported
@@ -11,6 +12,8 @@
* Change snapshot response message verb to INTERNAL to avoid dropping it
(CASSANDRA-6415)
* Warn when collection read has 65K elements (CASSANDRA-5428)
+ * Fix cache persistence when both row and key cache are enabled
+ (CASSANDRA-6413)
1.2.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57613dc8/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index d6a37df..a649e88 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -299,22 +299,18 @@ public class AutoSavingCacheK extends CacheKey, V
extends InstrumentingCacheK
private void deleteOldCacheFiles()
{
File savedCachesDir = new
File(DatabaseDescriptor.getSavedCachesLocation());
+assert savedCachesDir.exists() savedCachesDir.isDirectory();
-if (savedCachesDir.exists() savedCachesDir.isDirectory())
+for (File file : savedCachesDir.listFiles())
{
-for (File file : savedCachesDir.listFiles())
-{
-if (file.isFile()
file.getName().endsWith(cacheType.toString()))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (!file.isFile())
+continue; // someone's been messing with our directory.
naughty!
-if (file.isFile()
file.getName().endsWith(CURRENT_VERSION + .db))
-{
-if (!file.delete())
-logger.warn(Failed to delete {},
file.getAbsolutePath());
-}
+if (file.getName().endsWith(cacheType.toString())
+|| file.getName().endsWith(String.format(%s-%s.db,
cacheType.toString(), CURRENT_VERSION)))
+{
+if (!file.delete())
+logger.warn(Failed to delete {},
file.getAbsolutePath());
}
}
}
[4/6] git commit: merge from 1.2
merge from 1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/667e3db9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/667e3db9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/667e3db9 Branch: refs/heads/trunk Commit: 667e3db91cbec9928857aa89bfcc63fe85cb558d Parents: cf83c81 57613dc Author: Jonathan Ellis jbel...@apache.org Authored: Thu Dec 5 09:44:10 2013 -0600 Committer: Jonathan Ellis jbel...@apache.org Committed: Thu Dec 5 09:44:10 2013 -0600 -- CHANGES.txt | 5 - .../apache/cassandra/cache/AutoSavingCache.java | 22 2 files changed, 13 insertions(+), 14 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/667e3db9/CHANGES.txt -- diff --cc CHANGES.txt index d485a69,166c566..8996965 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -16,44 -12,11 +16,47 @@@ Merged from 1.2 * Change snapshot response message verb to INTERNAL to avoid dropping it (CASSANDRA-6415) * Warn when collection read has 65K elements (CASSANDRA-5428) + * Fix cache persistence when both row and key cache are enabled +(CASSANDRA-6413) -1.2.12 +2.0.3 + * Fix FD leak on slice read path (CASSANDRA-6275) + * Cancel read meter task when closing SSTR (CASSANDRA-6358) + * free off-heap IndexSummary during bulk (CASSANDRA-6359) + * Recover from IOException in accept() thread (CASSANDRA-6349) + * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338) + * Fix trying to hint timed out counter writes (CASSANDRA-6322) + * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809) + * Avoid flushing compaction_history after each operation (CASSANDRA-6287) + * Fix repair assertion error when tombstones expire (CASSANDRA-6277) + * Skip loading corrupt key cache (CASSANDRA-6260) + * Fixes for compacting larger-than-memory rows (CASSANDRA-6274) + * Compact hottest sstables first and optionally omit coldest from + compaction entirely (CASSANDRA-6109) + * Fix modifying column_metadata from thrift (CASSANDRA-6182) + * cqlsh: fix LIST USERS output (CASSANDRA-6242) + * Add IRequestSink interface (CASSANDRA-6248) + * Update memtable size while flushing (CASSANDRA-6249) + * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252) + * Require Permission.SELECT for CAS updates (CASSANDRA-6247) + * New CQL-aware SSTableWriter (CASSANDRA-5894) + * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270) + * Correctly throw error when frame too large (CASSANDRA-5981) + * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) + * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) + * Fix paging with reversed slices (CASSANDRA-6343) + * Set minTimestamp correctly to be able to drop expired sstables (CASSANDRA-6337) + * Support NaN and Infinity as float literals (CASSANDRA-6003) + * Remove RF from nodetool ring output (CASSANDRA-6289) + * Fix attempting to flush empty rows (CASSANDRA-6374) + * Fix potential out of bounds exception when paging (CASSANDRA-6333) +Merged from 1.2: + * Optimize FD phi calculation (CASSANDRA-6386) + * Improve initial FD phi estimate when starting up (CASSANDRA-6385) - * Don't list CQL3 table in CLI describe even if named explicitely (CASSANDRA-5750) ++ * Don't list CQL3 table in CLI describe even if named explicitely ++ (CASSANDRA-5750) * Invalidate row cache when dropping CF (CASSANDRA-6351) * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/667e3db9/src/java/org/apache/cassandra/cache/AutoSavingCache.java --
[jira] [Updated] (CASSANDRA-6453) Improve error message for invalid property values during parsing.
[ https://issues.apache.org/jira/browse/CASSANDRA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian ONeill updated CASSANDRA-6453: Attachment: CASSANDRA-6354-patch.txt Improve error message for invalid property values during parsing. - Key: CASSANDRA-6453 URL: https://issues.apache.org/jira/browse/CASSANDRA-6453 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brian ONeill Priority: Trivial Attachments: CASSANDRA-6354-patch.txt Trivial change to the error message returned for invalid property values. Previously, it would just say Invalid property value : ?. If you were constructing a large prepared statement, with multiple question marks, it was difficult to track down which one the server was complaining about. This enhancement tells you which one. =) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (CASSANDRA-6416) MoveTest fails in 1.2+
[ https://issues.apache.org/jira/browse/CASSANDRA-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler resolved CASSANDRA-6416. --- Resolution: Fixed MoveTest fails in 1.2+ -- Key: CASSANDRA-6416 URL: https://issues.apache.org/jira/browse/CASSANDRA-6416 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 1.2.13, 2.0.4 One test fails in 1.2, two in 2.0/trunk. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840285#comment-13840285 ] Radovan Zvoncek commented on CASSANDRA-5742: Hello. I would like to look into this one. Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-2238) Allow nodetool to print out hostnames given an option
[ https://issues.apache.org/jira/browse/CASSANDRA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840288#comment-13840288 ] Daneel Yaitskov commented on CASSANDRA-2238: I want to have an option in nodetool to enable reverse IP resolving. It's very inconvenient. My cluster is maintained by chef. So I recall about IPs at VM creation process only. Look at that. I see tens of IPs and one node failed. I had to remember bunch of numbers each time to enter them in neighbor console. IPv6 is just around the corner. What could you do with that?! Networking toolkit route makes reverse resolving by default. Allow nodetool to print out hostnames given an option - Key: CASSANDRA-2238 URL: https://issues.apache.org/jira/browse/CASSANDRA-2238 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Joaquin Casares Priority: Trivial Give nodetool the option of either displaying IPs or hostnames for the nodes in a ring. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6416) MoveTest fails in 1.2+
[ https://issues.apache.org/jira/browse/CASSANDRA-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840279#comment-13840279 ] Michael Shuler commented on CASSANDRA-6416: --- trunk looks OK: [junit] Testsuite: org.apache.cassandra.service.MoveTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 11.137 sec 2.0 looks OK: [junit] Testsuite: org.apache.cassandra.service.MoveTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 7.376 sec and 1.2: [junit] Testsuite: org.apache.cassandra.service.MoveTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 6.52 sec Fix looks like a winner: commit d8c4e89b3e85e8cb41a438963845cb10a923a3d6 Author: Marcus Eriksson marc...@spotify.com Date: Wed Dec 4 20:17:30 2013 +0100 fix MoveTest MoveTest fails in 1.2+ -- Key: CASSANDRA-6416 URL: https://issues.apache.org/jira/browse/CASSANDRA-6416 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 1.2.13, 2.0.4 One test fails in 1.2, two in 2.0/trunk. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6416) MoveTest fails in 1.2+
[ https://issues.apache.org/jira/browse/CASSANDRA-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840304#comment-13840304 ] Marcus Eriksson commented on CASSANDRA-6416: oops, didnt see this MoveTest fails in 1.2+ -- Key: CASSANDRA-6416 URL: https://issues.apache.org/jira/browse/CASSANDRA-6416 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 1.2.13, 2.0.4 One test fails in 1.2, two in 2.0/trunk. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-6414: -- Attachment: 6414_out.txt BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-6414: -- Attachment: (was: 6414_out) BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-6414: -- Attachment: 6414_out The difference between the output without the above conf edit (first in 6414_out attachment) and with the edit (ran it a couple times at the end of the file) is interesting and hopefully helpful with regards to the java.io.EOFException errors being the cause, I think. Setting the batch_window to 1s results in the test simply timing out, which isn't very helpful. Bisecting next (I'm gonna guess it is the multithread commit Dec 2). BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840310#comment-13840310 ] Michael Shuler edited comment on CASSANDRA-6414 at 12/5/13 5:42 PM: The difference between the output without the above conf edit (first in 6414_out attachment) and with the edit (ran it a couple times at the end of the file) is interesting and hopefully helpful with regards to the java.io.EOFException errors being the cause, I think. Setting the batch_window to 1s results in the test simply timing out, which isn't very helpful. Bisecting next was (Author: mshuler): The difference between the output without the above conf edit (first in 6414_out attachment) and with the edit (ran it a couple times at the end of the file) is interesting and hopefully helpful with regards to the java.io.EOFException errors being the cause, I think. Setting the batch_window to 1s results in the test simply timing out, which isn't very helpful. Bisecting next (I'm gonna guess it is the multithread commit Dec 2). BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat
Alex Liu created CASSANDRA-6454: --- Summary: Pig support for hadoop CqlInputFormat Key: CASSANDRA-6454 URL: https://issues.apache.org/jira/browse/CASSANDRA-6454 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Alex Liu CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6453) Improve error message for invalid property values during parsing.
[ https://issues.apache.org/jira/browse/CASSANDRA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6453: -- Reviewer: Aleksey Yeschenko Assignee: Brian ONeill Improve error message for invalid property values during parsing. - Key: CASSANDRA-6453 URL: https://issues.apache.org/jira/browse/CASSANDRA-6453 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brian ONeill Assignee: Brian ONeill Priority: Trivial Attachments: CASSANDRA-6354-patch.txt Trivial change to the error message returned for invalid property values. Previously, it would just say Invalid property value : ?. If you were constructing a large prepared statement, with multiple question marks, it was difficult to track down which one the server was complaining about. This enhancement tells you which one. =) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840358#comment-13840358 ] Jonathan Ellis commented on CASSANDRA-5742: --- Go for it Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6455) Improve concurrency of repair process
Yuki Morishita created CASSANDRA-6455: - Summary: Improve concurrency of repair process Key: CASSANDRA-6455 URL: https://issues.apache.org/jira/browse/CASSANDRA-6455 Project: Cassandra Issue Type: Improvement Reporter: Yuki Morishita Assignee: Yuki Morishita Priority: Minor Currently, most of the repair tasks (taking snapshots, send/receiving merkle tree, compute MT difference, etc) are done on single threaded AntiEntropyStage. This causes a problem like CASSANDRA-6415 and likely to cause unnecessary wait. Also, repair is done one CF at the time. I think we can parallelize this(concurrency is configurable by a user based on # of CF and load of the nodes) for faster processing. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli reassigned CASSANDRA-5742: Assignee: sankalp kohli Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Assignee: sankalp kohli Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840440#comment-13840440 ] sankalp kohli commented on CASSANDRA-5742: -- I am working on it. Let me assign it to myself. I will have a patch ready by this weekend. Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840456#comment-13840456 ] sankalp kohli commented on CASSANDRA-5742: -- ./nodetool -h localhost listsnapshots The output will look something like Snapshot Details: Snapshot Name:1386271615474 keyspace_namecolumnfamily_name disk_used Keyspace2Standard1 402.51 MB Keyspace1Standard1 146.23 MB Snapshot Name:1386271380933 keyspace_namecolumnfamily_name disk_used Keyspace2Standard1 286.53 MB Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Assignee: sankalp kohli Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3578: -- Attachment: 3578-logging-v2.txt I'm not really sure what users should take away from the % numbers. v2 simplifies to just a count and average lag duration. Multithreaded commitlog --- Key: CASSANDRA-3578 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1 Attachments: 0001-CASSANDRA-3578.patch, 3578-logging-v2.txt, ComitlogStress.java, Current-CL.png, Multi-Threded-CL.png, TestEA.java, latency.svg, oprate.svg, parallel_commit_log_2.patch Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate) can improve performance, since you're not bottlenecking on a single thread to do all the copying and CRC computation. Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable. (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840466#comment-13840466 ] Michael Shuler commented on CASSANDRA-6414: --- a few different good/bad strategies - this one seems to be closest: ((fa0b7cf...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad There are only 'skip'ped commits left to test. The first bad commit could be any of: 29605aedd9e19f2f07042cd0aa6b31b6c94a4aea 873ce0cb3f05d55753f205092e681c963cd20fc4 fa0b7cf8b279c568aa7b2fcbaad91510797d838f We cannot bisect more! BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840472#comment-13840472 ] Benedict commented on CASSANDRA-3578: - The idea of the % was to give some sense of scale of the problem. The problem with just offering a count is that it's only averaged over the number of lagged commits, which doesn't tell the whole story (if most of the commits didn't lag, it might not be worth worrying about), and averaging over all the commits might be overly dismissive because a lot of commits may be NO-OPs, so giving a ratio of time spent committing to time spent lagged seemed a useful compromise. Though perhaps simply total time spent lagged and total time spent committing might be more useful. Multithreaded commitlog --- Key: CASSANDRA-3578 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1 Attachments: 0001-CASSANDRA-3578.patch, 3578-logging-v2.txt, ComitlogStress.java, Current-CL.png, Multi-Threded-CL.png, TestEA.java, latency.svg, oprate.svg, parallel_commit_log_2.patch Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate) can improve performance, since you're not bottlenecking on a single thread to do all the copying and CRC computation. Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable. (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-5742) Add command list snapshots to nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840508#comment-13840508 ] sankalp kohli commented on CASSANDRA-5742: -- Let me add that too. Add command list snapshots to nodetool Key: CASSANDRA-5742 URL: https://issues.apache.org/jira/browse/CASSANDRA-5742 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.2.1 Reporter: Geert Schuring Assignee: sankalp kohli Priority: Minor Labels: lhf It would be nice if the nodetool could tell me which snapshots are present on the system instead of me having to browse the filesystem to fetch the names of the snapshots. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6356) Proposal: Statistics.db (SSTableMetadata) format change
[ https://issues.apache.org/jira/browse/CASSANDRA-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-6356: -- Attachment: 6356-v2.txt also: https://github.com/yukim/cassandra/commits/6356-v2 v2 modified new metadata format to have TOC section at the beginning of the file to easily seek to the position of desired metadata. Proposal: Statistics.db (SSTableMetadata) format change --- Key: CASSANDRA-6356 URL: https://issues.apache.org/jira/browse/CASSANDRA-6356 Project: Cassandra Issue Type: Improvement Reporter: Yuki Morishita Assignee: Yuki Morishita Priority: Minor Fix For: 2.1 Attachments: 6356-v2.txt We started to distinguish what's loaded to heap, and what's not from Statistics.db. For now, ancestors are loaded as they needed. Current serialization format is so adhoc that adding new metadata that are not permanently hold onto memory is somewhat difficult and messy. I propose to change serialization format so that a group of stats can be loaded as needed. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3578: -- Attachment: 3578-logging-v3.txt v3 lags average sync duration (for all syncs) and average overage (for just the laggy ones). think that's as good as we'll get without logging histograms :) Multithreaded commitlog --- Key: CASSANDRA-3578 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1 Attachments: 0001-CASSANDRA-3578.patch, 3578-logging-v2.txt, 3578-logging-v3.txt, ComitlogStress.java, Current-CL.png, Multi-Threded-CL.png, TestEA.java, latency.svg, oprate.svg, parallel_commit_log_2.patch Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate) can improve performance, since you're not bottlenecking on a single thread to do all the copying and CRC computation. Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable. (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Liu updated CASSANDRA-6309:
Attachment: 6309-v3.txt
V3 patch is based on the latest 2.0 branch
Pig CqlStorage generates ERROR 1108: Duplicate schema alias
Key: CASSANDRA-6309
URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Thunder Stumpges
Assignee: Alex Liu
Attachments: 6309-2.0.txt, 6309-v2-2.0-branch.txt, 6309-v3.txt,
LOCAL_ONE-write-for-all-strategies-v2.txt,
LOCAL_ONE-write-for-all-strategies.txt
In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
contents, I receive:
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
dump cm
ERROR org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias cm
...
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
at
org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
running 'describe cm' gives:
cm: {message_id: chararray,author: chararray,author: chararray,body:
chararray,message_id: chararray}
The original table schema in Cassandra is:
CREATE TABLE cassandra_messages (
message_id text,
author text,
body text,
PRIMARY KEY (message_id, author)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='null' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes
the keys columns (in my case, message_id and author) and appends the
columns from getColumnMeta (which has all three columns). Thus the keys
columns are duplicated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840621#comment-13840621
]
Alex Liu commented on CASSANDRA-6309:
-
6309-fix-pig-test-compiling.txt fixes the build issue for pig test
Pig CqlStorage generates ERROR 1108: Duplicate schema alias
Key: CASSANDRA-6309
URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Thunder Stumpges
Assignee: Alex Liu
Attachments: 6309-2.0.txt, 6309-fix-pig-test-compiling.txt,
6309-v2-2.0-branch.txt, 6309-v3.txt,
LOCAL_ONE-write-for-all-strategies-v2.txt,
LOCAL_ONE-write-for-all-strategies.txt
In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
contents, I receive:
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
dump cm
ERROR org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias cm
...
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
at
org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
running 'describe cm' gives:
cm: {message_id: chararray,author: chararray,author: chararray,body:
chararray,message_id: chararray}
The original table schema in Cassandra is:
CREATE TABLE cassandra_messages (
message_id text,
author text,
body text,
PRIMARY KEY (message_id, author)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='null' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes
the keys columns (in my case, message_id and author) and appends the
columns from getColumnMeta (which has all three columns). Thus the keys
columns are duplicated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[3/3] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e1f2a08f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e1f2a08f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e1f2a08f Branch: refs/heads/trunk Commit: e1f2a08fc6c795b64b28c0672d735b008916acb9 Parents: 5ad8a3d f7efaff Author: Brandon Williams brandonwilli...@apache.org Authored: Thu Dec 5 16:18:14 2013 -0600 Committer: Brandon Williams brandonwilli...@apache.org Committed: Thu Dec 5 16:18:14 2013 -0600 -- --
[2/3] git commit: Pig: fix duplicate schema alias Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-6309
Pig: fix duplicate schema alias
Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-6309
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7efaffa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7efaffa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7efaffa
Branch: refs/heads/trunk
Commit: f7efaffadace3e344eeb4a1384fa72c73d8422b0
Parents: 667e3db
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Dec 5 16:14:43 2013 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Dec 5 16:14:43 2013 -0600
--
build.xml | 1 +
.../apache/cassandra/db/ConsistencyLevel.java | 2 -
.../hadoop/pig/AbstractCassandraStorage.java| 21 +---
.../cassandra/pig/CqlTableDataTypeTest.java | 35 ++
.../org/apache/cassandra/pig/CqlTableTest.java | 9 +-
.../pig/ThriftColumnFamilyDataTypeTest.java | 21
.../cassandra/pig/ThriftColumnFamilyTest.java | 121 +--
7 files changed, 48 insertions(+), 162 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/build.xml
--
diff --git a/build.xml b/build.xml
index 41a7fb0..e21091a 100644
--- a/build.xml
+++ b/build.xml
@@ -990,6 +990,7 @@
/classpath
src path=${test.unit.src}/
src path=${test.long.src}/
+ src path=${test.pig.src}/
/javac
!-- Non-java resources needed by the test suite --
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index cbb4bb1..0f6aba7 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -285,9 +285,7 @@ public enum ConsistencyLevel
{
switch (this)
{
-case LOCAL_QUORUM:
case EACH_QUORUM:
-case LOCAL_ONE:
requireNetworkTopologyStrategy(keyspaceName);
break;
case SERIAL:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
--
diff --git
a/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
index 9e2e301..3fb1c5a 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
@@ -614,20 +614,7 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
cfDef.default_validation_class =
ByteBufferUtil.string(cqlRow.columns.get(3).value);
cfDef.key_validation_class =
ByteBufferUtil.string(cqlRow.columns.get(4).value);
String keyAliases =
ByteBufferUtil.string(cqlRow.columns.get(5).value);
-ListString keys = FBUtilities.fromJsonList(keyAliases);
-// classis thrift tables
-if (keys.size() == 0)
-{
-CFDefinition cfDefinition = getCfDefinition(keyspace,
column_family, client);
-for (ColumnIdentifier column : cfDefinition.keys.keySet())
-{
-String key = column.toString();
-String type =
cfDefinition.keys.get(column).type.toString();
-logger.debug(name: {}, type: {} , key, type);
-keys.add(key);
-}
-}
-else
+if (FBUtilities.fromJsonList(keyAliases).size() 0)
cql3Table = true;
}
cfDef.column_metadata = getColumnMetadata(client);
@@ -666,7 +653,8 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
{
String query = SELECT column_name, +
validator, +
- index_type +
+ index_type, +
+ type +
FROM system.schema_columns +
WHERE keyspace_name = '%s' +
AND columnfamily_name = '%s';
@@ -717,6 +705,9 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
{
CqlRow row = iterator.next();
ColumnDef cDef = new ColumnDef();
+String type =
[1/3] git commit: Pig: fix duplicate schema alias Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-6309
Updated Branches:
refs/heads/cassandra-2.0 667e3db91 - f7efaffad
refs/heads/trunk 5ad8a3d3a - e1f2a08fc
Pig: fix duplicate schema alias
Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-6309
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7efaffa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7efaffa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7efaffa
Branch: refs/heads/cassandra-2.0
Commit: f7efaffadace3e344eeb4a1384fa72c73d8422b0
Parents: 667e3db
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Dec 5 16:14:43 2013 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Dec 5 16:14:43 2013 -0600
--
build.xml | 1 +
.../apache/cassandra/db/ConsistencyLevel.java | 2 -
.../hadoop/pig/AbstractCassandraStorage.java| 21 +---
.../cassandra/pig/CqlTableDataTypeTest.java | 35 ++
.../org/apache/cassandra/pig/CqlTableTest.java | 9 +-
.../pig/ThriftColumnFamilyDataTypeTest.java | 21
.../cassandra/pig/ThriftColumnFamilyTest.java | 121 +--
7 files changed, 48 insertions(+), 162 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/build.xml
--
diff --git a/build.xml b/build.xml
index 41a7fb0..e21091a 100644
--- a/build.xml
+++ b/build.xml
@@ -990,6 +990,7 @@
/classpath
src path=${test.unit.src}/
src path=${test.long.src}/
+ src path=${test.pig.src}/
/javac
!-- Non-java resources needed by the test suite --
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index cbb4bb1..0f6aba7 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -285,9 +285,7 @@ public enum ConsistencyLevel
{
switch (this)
{
-case LOCAL_QUORUM:
case EACH_QUORUM:
-case LOCAL_ONE:
requireNetworkTopologyStrategy(keyspaceName);
break;
case SERIAL:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7efaffa/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
--
diff --git
a/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
index 9e2e301..3fb1c5a 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
@@ -614,20 +614,7 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
cfDef.default_validation_class =
ByteBufferUtil.string(cqlRow.columns.get(3).value);
cfDef.key_validation_class =
ByteBufferUtil.string(cqlRow.columns.get(4).value);
String keyAliases =
ByteBufferUtil.string(cqlRow.columns.get(5).value);
-ListString keys = FBUtilities.fromJsonList(keyAliases);
-// classis thrift tables
-if (keys.size() == 0)
-{
-CFDefinition cfDefinition = getCfDefinition(keyspace,
column_family, client);
-for (ColumnIdentifier column : cfDefinition.keys.keySet())
-{
-String key = column.toString();
-String type =
cfDefinition.keys.get(column).type.toString();
-logger.debug(name: {}, type: {} , key, type);
-keys.add(key);
-}
-}
-else
+if (FBUtilities.fromJsonList(keyAliases).size() 0)
cql3Table = true;
}
cfDef.column_metadata = getColumnMetadata(client);
@@ -666,7 +653,8 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
{
String query = SELECT column_name, +
validator, +
- index_type +
+ index_type, +
+ type +
FROM system.schema_columns +
WHERE keyspace_name = '%s' +
AND columnfamily_name = '%s';
@@ -717,6 +705,9 @@ public abstract class AbstractCassandraStorage extends
LoadFunc implements Store
{
CqlRow row =
[jira] [Updated] (CASSANDRA-6158) Nodetool command to purge hints
[ https://issues.apache.org/jira/browse/CASSANDRA-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-6158: - Attachment: trunk-6158.txt Nodetool command to purge hints --- Key: CASSANDRA-6158 URL: https://issues.apache.org/jira/browse/CASSANDRA-6158 Project: Cassandra Issue Type: Improvement Components: Core Reporter: sankalp kohli Assignee: sankalp kohli Priority: Minor Attachments: trunk-6158.txt The only way to truncate all hints in Cassandra is to truncate the hints CF in system table. It would be cleaner to have a nodetool command for it. Also ability to selectively remove hints by host or DC would also be nice rather than removing all the hints. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840698#comment-13840698 ] Benedict commented on CASSANDRA-3578: - +1 Multithreaded commitlog --- Key: CASSANDRA-3578 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1 Attachments: 0001-CASSANDRA-3578.patch, 3578-logging-v2.txt, 3578-logging-v3.txt, ComitlogStress.java, Current-CL.png, Multi-Threded-CL.png, TestEA.java, latency.svg, oprate.svg, parallel_commit_log_2.patch Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate) can improve performance, since you're not bottlenecking on a single thread to do all the copying and CRC computation. Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable. (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6415) Snapshot repair blocks for ever if something happens to the I made my snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRA-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840696#comment-13840696
]
Jeremy Hanna commented on CASSANDRA-6415:
-
See the longer term solution here: CASSANDRA-6455
Snapshot repair blocks for ever if something happens to the I made my
snapshot response
-
Key: CASSANDRA-6415
URL: https://issues.apache.org/jira/browse/CASSANDRA-6415
Project: Cassandra
Issue Type: Bug
Reporter: Jeremiah Jordan
Assignee: Yuki Morishita
Labels: repair
Fix For: 1.2.13
Attachments: 6415-1.2.txt
The snapshotLatch.await(); can be waiting for ever and block all repair
operations indefinitely if something happens that another node doesn't
respond.
{noformat}
public void makeSnapshots(CollectionInetAddress endpoints)
{
try
{
snapshotLatch = new CountDownLatch(endpoints.size());
IAsyncCallback callback = new IAsyncCallback()
{
public boolean isLatencyForSnitch()
{
return false;
}
public void response(MessageIn msg)
{
RepairJob.this.snapshotLatch.countDown();
}
};
for (InetAddress endpoint : endpoints)
MessagingService.instance().sendRR(new
SnapshotCommand(tablename, cfname, sessionName, false).createMessage(),
endpoint, callback);
snapshotLatch.await();
snapshotLatch = null;
}
catch (InterruptedException e)
{
throw new RuntimeException(e);
}
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840702#comment-13840702 ] Michael Shuler commented on CASSANDRA-6414: --- I misunderstood the logback spam as the WARN entries from CASSANDRA-3578. So ignoring all the ERROR traces in the output (which I was marking bad above), just straight SUCESSFUL/FAILED test gets me: ((f3dc188...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad f3dc188e203b3db980ee81df05390968043cb601 is the first bad commit commit f3dc188e203b3db980ee81df05390968043cb601 Author: Jonathan Ellis jbel...@apache.org Date: Fri Nov 22 17:33:07 2013 -0600 move setting lastCompactedKey to before the return-if-nothing-added :04 04 0c616036c3aac24861a0f43b21f5812faecf2c92 33fb0a0761168d605a224dc87522785afe8cb5a1 M src BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840702#comment-13840702 ] Michael Shuler edited comment on CASSANDRA-6414 at 12/5/13 11:51 PM: - (edit: added commit, since it's a one-liner) I misunderstood the logback spam as the WARN entries from CASSANDRA-3578. So ignoring all the ERROR traces in the output (which I was marking bad above), just straight SUCESSFUL/FAILED test gets me: ((f3dc188...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad f3dc188e203b3db980ee81df05390968043cb601 is the first bad commit commit f3dc188e203b3db980ee81df05390968043cb601 Author: Jonathan Ellis jbel...@apache.org Date: Fri Nov 22 17:33:07 2013 -0600 move setting lastCompactedKey to before the return-if-nothing-added :04 04 0c616036c3aac24861a0f43b21f5812faecf2c92 33fb0a0761168d605a224dc87522785afe8cb5a1 M src $ git show f3dc188e203b3db980ee81df05390968043cb601 commit f3dc188e203b3db980ee81df05390968043cb601 (HEAD, refs/bisect/bad) Author: Jonathan Ellis jbel...@apache.org Date: Fri Nov 22 17:33:07 2013 -0600 move setting lastCompactedKey to before the return-if-nothing-added diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java b/src/java/org/apache/cassandra/ index 76f51d1..232d1f7 100644 --- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java +++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java @@ -142,6 +142,7 @@ public class LeveledManifest int thisLevel = remove(sstable); minLevel = Math.min(minLevel, thisLevel); } +lastCompactedKeys[minLevel] = SSTableReader.sstableOrdering.max(added).last; // it's valid to do a remove w/o an add (e.g. on truncate) if (added.isEmpty()) @@ -152,7 +153,6 @@ public class LeveledManifest for (SSTableReader ssTableReader : added) add(ssTableReader); -lastCompactedKeys[minLevel] = SSTableReader.sstableOrdering.max(added).last; } public synchronized void repairOverlappingSSTables(int level) was (Author: mshuler): I misunderstood the logback spam as the WARN entries from CASSANDRA-3578. So ignoring all the ERROR traces in the output (which I was marking bad above), just straight SUCESSFUL/FAILED test gets me: ((f3dc188...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad f3dc188e203b3db980ee81df05390968043cb601 is the first bad commit commit f3dc188e203b3db980ee81df05390968043cb601 Author: Jonathan Ellis jbel...@apache.org Date: Fri Nov 22 17:33:07 2013 -0600 move setting lastCompactedKey to before the return-if-nothing-added :04 04 0c616036c3aac24861a0f43b21f5812faecf2c92 33fb0a0761168d605a224dc87522785afe8cb5a1 M src BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840702#comment-13840702
]
Michael Shuler edited comment on CASSANDRA-6414 at 12/5/13 11:52 PM:
-
(edit: added commit, since it's a one-liner)
I misunderstood the logback spam as the WARN entries from CASSANDRA-3578. So
ignoring all the ERROR traces in the output (which I was marking bad above),
just straight SUCESSFUL/FAILED test gets me:
((f3dc188...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad
f3dc188e203b3db980ee81df05390968043cb601 is the first bad commit
commit f3dc188e203b3db980ee81df05390968043cb601
Author: Jonathan Ellis jbel...@apache.org
Date: Fri Nov 22 17:33:07 2013 -0600
move setting lastCompactedKey to before the return-if-nothing-added
:04 04 0c616036c3aac24861a0f43b21f5812faecf2c92
33fb0a0761168d605a224dc87522785afe8cb5a1 M src
$ git show f3dc188e203b3db980ee81df05390968043cb601
commit f3dc188e203b3db980ee81df05390968043cb601 (HEAD, refs/bisect/bad)
Author: Jonathan Ellis jbel...@apache.org
Date: Fri Nov 22 17:33:07 2013 -0600
move setting lastCompactedKey to before the return-if-nothing-added
{code}
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
b/src/java/org/apache/cassandra/
index 76f51d1..232d1f7 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -142,6 +142,7 @@ public class LeveledManifest
int thisLevel = remove(sstable);
minLevel = Math.min(minLevel, thisLevel);
}
+lastCompactedKeys[minLevel] =
SSTableReader.sstableOrdering.max(added).last;
// it's valid to do a remove w/o an add (e.g. on truncate)
if (added.isEmpty())
@@ -152,7 +153,6 @@ public class LeveledManifest
for (SSTableReader ssTableReader : added)
add(ssTableReader);
-lastCompactedKeys[minLevel] =
SSTableReader.sstableOrdering.max(added).last;
}
public synchronized void repairOverlappingSSTables(int level)
{code}
was (Author: mshuler):
(edit: added commit, since it's a one-liner)
I misunderstood the logback spam as the WARN entries from CASSANDRA-3578. So
ignoring all the ERROR traces in the output (which I was marking bad above),
just straight SUCESSFUL/FAILED test gets me:
((f3dc188...)|BISECTING)mshuler@hana:~/git/cassandra$ git bisect bad
f3dc188e203b3db980ee81df05390968043cb601 is the first bad commit
commit f3dc188e203b3db980ee81df05390968043cb601
Author: Jonathan Ellis jbel...@apache.org
Date: Fri Nov 22 17:33:07 2013 -0600
move setting lastCompactedKey to before the return-if-nothing-added
:04 04 0c616036c3aac24861a0f43b21f5812faecf2c92
33fb0a0761168d605a224dc87522785afe8cb5a1 M src
$ git show f3dc188e203b3db980ee81df05390968043cb601
commit f3dc188e203b3db980ee81df05390968043cb601 (HEAD, refs/bisect/bad)
Author: Jonathan Ellis jbel...@apache.org
Date: Fri Nov 22 17:33:07 2013 -0600
move setting lastCompactedKey to before the return-if-nothing-added
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
b/src/java/org/apache/cassandra/
index 76f51d1..232d1f7 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -142,6 +142,7 @@ public class LeveledManifest
int thisLevel = remove(sstable);
minLevel = Math.min(minLevel, thisLevel);
}
+lastCompactedKeys[minLevel] =
SSTableReader.sstableOrdering.max(added).last;
// it's valid to do a remove w/o an add (e.g. on truncate)
if (added.isEmpty())
@@ -152,7 +153,6 @@ public class LeveledManifest
for (SSTableReader ssTableReader : added)
add(ssTableReader);
-lastCompactedKeys[minLevel] =
SSTableReader.sstableOrdering.max(added).last;
}
public synchronized void repairOverlappingSSTables(int level)
BlacklistingCompactionsTest fails in trunk
--
Key: CASSANDRA-6414
URL: https://issues.apache.org/jira/browse/CASSANDRA-6414
Project: Cassandra
Issue Type: Bug
Components: Tests
Reporter: Jonathan Ellis
Assignee: Michael Shuler
Fix For: 2.1
Attachments: 6414_out.txt
Passes in 2.0 HEAD. Bisect should be relatively easy.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (CASSANDRA-6456) log listen address at startup
Jeremy Hanna created CASSANDRA-6456: --- Summary: log listen address at startup Key: CASSANDRA-6456 URL: https://issues.apache.org/jira/browse/CASSANDRA-6456 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna Priority: Trivial When looking through logs from a cluster, sometimes it's handy to know the address a node is from the logs. It would be convenient if on startup, we indicated the listen address for that node. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-6456) log listen address at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna reassigned CASSANDRA-6456: --- Assignee: Jeremy Hanna log listen address at startup - Key: CASSANDRA-6456 URL: https://issues.apache.org/jira/browse/CASSANDRA-6456 Project: Cassandra Issue Type: Wish Components: Core Reporter: Jeremy Hanna Assignee: Jeremy Hanna Priority: Trivial When looking through logs from a cluster, sometimes it's handy to know the address a node is from the logs. It would be convenient if on startup, we indicated the listen address for that node. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6365) Bisect unit test failures on 2.0 branch
[ https://issues.apache.org/jira/browse/CASSANDRA-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840722#comment-13840722 ] Michael Shuler commented on CASSANDRA-6365: --- Latest several 2.0 branch unit test builds have passed successfully - good work! http://cassci.datastax.com/job/cassandra-2.0_test/ Bisect unit test failures on 2.0 branch --- Key: CASSANDRA-6365 URL: https://issues.apache.org/jira/browse/CASSANDRA-6365 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Attachments: 2.0.1-utest.txt, C-2.0.1_tag_utests.txt Unit tests pass in 2.0.1. They do not in 2.0.2. Let's find where the failures were introduced. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (CASSANDRA-6365) Bisect unit test failures on 2.0 branch
[ https://issues.apache.org/jira/browse/CASSANDRA-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler resolved CASSANDRA-6365. --- Resolution: Fixed Bisect unit test failures on 2.0 branch --- Key: CASSANDRA-6365 URL: https://issues.apache.org/jira/browse/CASSANDRA-6365 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Attachments: 2.0.1-utest.txt, C-2.0.1_tag_utests.txt Unit tests pass in 2.0.1. They do not in 2.0.2. Let's find where the failures were introduced. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840739#comment-13840739
]
Alex Liu commented on CASSANDRA-6309:
-
Trunk bumped Pig version to 0.11.0, the pig unit tests fail
{code}
[junit] - ---
[junit] Testcase:
testCqlStorageListType(org.apache.cassandra.pig.CqlTableDataTypeTest):
Caused an ERROR
[junit] Unable to open iterator for alias list_rows
[junit] org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
Unable to open iterator for alias list_rows
[junit] at org.apache.pig.PigServer.openIterator(PigServer.java:836)
[junit] at
org.apache.cassandra.pig.CqlTableDataTypeTest.testCqlStorageListType(CqlTableDataTypeTest.java:332)
[junit] Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store
alias list_rows
[junit] at org.apache.pig.PigServer.storeEx(PigServer.java:935)
[junit] at org.apache.pig.PigServer.store(PigServer.java:898)
[junit] at org.apache.pig.PigServer.openIterator(PigServer.java:811)
[junit] Caused by:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
ERROR 2017: Internal error creating job configuration.
[junit] at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:848)
[junit] at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:294)
[junit] at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
[junit] at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
[junit] at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
[junit] at org.apache.pig.PigServer.storeEx(PigServer.java:931)
[junit] Caused by: java.io.IOException: Serialization error:
org.apache.log4j.Level
[junit] at
org.apache.pig.impl.util.ObjectSerializer.serialize(ObjectSerializer.java:47)
[junit] at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:526)
[junit] Caused by: java.io.NotSerializableException: org.apache.log4j.Level
[junit] at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
[junit] at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528)
[junit] at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
[junit] at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416)
[junit] at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
[junit] at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346)
[junit] at
org.apache.pig.impl.util.ObjectSerializer.serialize(ObjectSerializer.java:43)
{code}
Pig CqlStorage generates ERROR 1108: Duplicate schema alias
Key: CASSANDRA-6309
URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Thunder Stumpges
Assignee: Alex Liu
Attachments: 6309-2.0.txt, 6309-fix-pig-test-compiling.txt,
6309-v2-2.0-branch.txt, 6309-v3.txt,
LOCAL_ONE-write-for-all-strategies-v2.txt,
LOCAL_ONE-write-for-all-strategies.txt
In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
contents, I receive:
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
dump cm
ERROR org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias cm
...
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in cm
at
org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
running 'describe cm' gives:
cm: {message_id: chararray,author: chararray,author: chararray,body:
chararray,message_id: chararray}
The original table schema in Cassandra is:
CREATE TABLE cassandra_messages (
message_id text,
author text,
body text,
PRIMARY KEY (message_id, author)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='null' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
[jira] [Created] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
Michael Shuler created CASSANDRA-6457: - Summary: Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-6457: -- Description: Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 was:Identify and bisect tests failing in trunk (2.1). Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840766#comment-13840766 ] Michael Shuler commented on CASSANDRA-6457: --- $ ant -q test [echo] apache-cassandra: /home/mshuler/DataStax/repos/cassandra/build.xml [echo] running unit tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/home/mshuler/DataStax/repos/cassandra/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Test org.apache.cassandra.cli.CliTest FAILED (timeout) [junit] Test org.apache.cassandra.config.DefsTest FAILED (timeout) [junit] Test org.apache.cassandra.db.CleanupTest FAILED [junit] Test org.apache.cassandra.db.ColumnFamilyStoreTest FAILED (timeout) [junit] Test org.apache.cassandra.db.compaction.BlacklistingCompactionsTest FAILED [junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest FAILED (timeout) [junit] Test org.apache.cassandra.db.compaction.CompactionsTest FAILED (timeout) [junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED (timeout) [junit] Test org.apache.cassandra.dht.OrderPreservingPartitionerTest FAILED [junit] Exception in thread StorageServiceShutdownHook java.lang.IllegalStateException: No configured daemon [junit] at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:305) [junit] at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:357) [junit] at org.apache.cassandra.service.StorageService.access$000(StorageService.java:93) [junit] at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:538) [junit] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) [junit] at java.lang.Thread.run(Thread.java:744) [junit] Test org.apache.cassandra.io.sstable.IndexSummaryManagerTest FAILED (timeout) [junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED (timeout) [junit] Test org.apache.cassandra.service.LeaveAndBootstrapTest FAILED [junit] Test org.apache.cassandra.streaming.StreamingTransferTest FAILED (timeout) BUILD FAILED Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[
https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840766#comment-13840766
]
Michael Shuler edited comment on CASSANDRA-6457 at 12/6/13 1:05 AM:
{code}
$ ant -q test
[echo] apache-cassandra: /home/mshuler/DataStax/repos/cassandra/build.xml
[echo] running unit tests
[junit] WARNING: multiple versions of ant detected in path for junit
[junit]
jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] and
jar:file:/home/mshuler/DataStax/repos/cassandra/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
[junit] Test org.apache.cassandra.cli.CliTest FAILED (timeout)
[junit] Test org.apache.cassandra.config.DefsTest FAILED (timeout)
[junit] Test org.apache.cassandra.db.CleanupTest FAILED
[junit] Test org.apache.cassandra.db.ColumnFamilyStoreTest FAILED (timeout)
[junit] Test org.apache.cassandra.db.compaction.BlacklistingCompactionsTest
FAILED
[junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest FAILED
(timeout)
[junit] Test org.apache.cassandra.db.compaction.CompactionsTest FAILED
(timeout)
[junit] Test
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
(timeout)
[junit] Test org.apache.cassandra.dht.OrderPreservingPartitionerTest FAILED
[junit] Exception in thread StorageServiceShutdownHook
java.lang.IllegalStateException: No configured daemon
[junit] at
org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:305)
[junit] at
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:357)
[junit] at
org.apache.cassandra.service.StorageService.access$000(StorageService.java:93)
[junit] at
org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:538)
[junit] at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[junit] at java.lang.Thread.run(Thread.java:744)
[junit] Test org.apache.cassandra.io.sstable.IndexSummaryManagerTest FAILED
(timeout)
[junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
(timeout)
[junit] Test org.apache.cassandra.service.LeaveAndBootstrapTest FAILED
[junit] Test org.apache.cassandra.streaming.StreamingTransferTest FAILED
(timeout)
BUILD FAILED
{code}
was (Author: mshuler):
$ ant -q test
[echo] apache-cassandra: /home/mshuler/DataStax/repos/cassandra/build.xml
[echo] running unit tests
[junit] WARNING: multiple versions of ant detected in path for junit
[junit]
jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] and
jar:file:/home/mshuler/DataStax/repos/cassandra/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
[junit] Test org.apache.cassandra.cli.CliTest FAILED (timeout)
[junit] Test org.apache.cassandra.config.DefsTest FAILED (timeout)
[junit] Test org.apache.cassandra.db.CleanupTest FAILED
[junit] Test org.apache.cassandra.db.ColumnFamilyStoreTest FAILED (timeout)
[junit] Test org.apache.cassandra.db.compaction.BlacklistingCompactionsTest
FAILED
[junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest FAILED
(timeout)
[junit] Test org.apache.cassandra.db.compaction.CompactionsTest FAILED
(timeout)
[junit] Test
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
(timeout)
[junit] Test org.apache.cassandra.dht.OrderPreservingPartitionerTest FAILED
[junit] Exception in thread StorageServiceShutdownHook
java.lang.IllegalStateException: No configured daemon
[junit] at
org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:305)
[junit] at
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:357)
[junit] at
org.apache.cassandra.service.StorageService.access$000(StorageService.java:93)
[junit] at
org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:538)
[junit] at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[junit] at java.lang.Thread.run(Thread.java:744)
[junit] Test org.apache.cassandra.io.sstable.IndexSummaryManagerTest FAILED
(timeout)
[junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
(timeout)
[junit] Test org.apache.cassandra.service.LeaveAndBootstrapTest FAILED
[junit] Test org.apache.cassandra.streaming.StreamingTransferTest FAILED
(timeout)
BUILD FAILED
Bisect unit test failures on trunk (2.1)
Key: CASSANDRA-6457
URL: https://issues.apache.org/jira/browse/CASSANDRA-6457
Project: Cassandra
Issue Type: Bug
Components:
[jira] [Comment Edited] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840774#comment-13840774 ] Michael Shuler edited comment on CASSANDRA-6457 at 12/6/13 1:11 AM: Ignoring the timeout ones, atm - CleanupTest passes on another run.. was (Author: mshuler): Ignoring the timeout one, atm - CleanupTest passes on another run.. Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840774#comment-13840774 ] Michael Shuler commented on CASSANDRA-6457: --- Ignoring the timeout one, atm - CleanupTest passes on another run.. Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6457) Bisect unit test failures on trunk (2.1)
[ https://issues.apache.org/jira/browse/CASSANDRA-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840778#comment-13840778 ] Michael Shuler commented on CASSANDRA-6457: --- So does OrderPreservingPartitionerTest Bisect unit test failures on trunk (2.1) Key: CASSANDRA-6457 URL: https://issues.apache.org/jira/browse/CASSANDRA-6457 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Michael Shuler Assignee: Michael Shuler Identify and bisect tests failing in trunk (2.1). BlacklistingCompactionsTest - CASSANDRA-6414 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840857#comment-13840857 ] Jeremiah Jordan commented on CASSANDRA-6268: [~jbellis] we good to commit this now? Poor performance of Hadoop if any DC is using VNodes Key: CASSANDRA-6268 URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Piotr Kołaczkowski Assignee: Piotr Kołaczkowski Attachments: 6268-src-1.2.txt, 6268-src-2.0.txt, 6268-thrift-1.2.txt, 6268-thrift-2.0.txt Some customers are complaining about huge number of splits in Hadoop caused by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are generated from the results of describe_ring, which returns a huge number of ranges anyways, and doesn't take into account that there will be huge number of consecutive ranges residing on the nodes we'd like the M/R job to be run. The proposed fix: 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - defaults to all Hadoop DCs) 2. merges consecutive ranges before generating Hadoop splits, so we don't have artificial range splitting caused by vnodes in the other DCs For non-DSE users this feature is turned off by default and doesn't change the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
git commit: warn about sync delays every 5s instead of as-it-happens patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-3578
Updated Branches:
refs/heads/trunk e1f2a08fc - e97803425
warn about sync delays every 5s instead of as-it-happens
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-3578
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9780342
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9780342
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9780342
Branch: refs/heads/trunk
Commit: e978034259de12619f2e4c32e29b034e9d6a2149
Parents: e1f2a08
Author: belliottsmith git...@sub.laerad.com
Authored: Tue Dec 3 19:40:51 2013 +
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Dec 5 20:37:19 2013 -0600
--
.../db/commitlog/AbstractCommitLogService.java | 38 +---
.../db/commitlog/CommitLogReplayer.java | 2 +-
2 files changed, 35 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9780342/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
--
diff --git
a/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
b/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
index 2f9b236..caa2c65 100644
--- a/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
+++ b/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogService.java
@@ -28,6 +28,9 @@ import static
org.apache.cassandra.db.commitlog.CommitLogSegment.Allocation;
public abstract class AbstractCommitLogService
{
+// how often should we log syngs that lag behind our desired period
+private static final long LAG_REPORT_INTERVAL =
TimeUnit.MINUTES.toMillis(5);
+
private final Thread thread;
private volatile boolean shutdown = false;
@@ -59,6 +62,12 @@ public abstract class AbstractCommitLogService
{
public void run()
{
+long firstLagAt = 0;
+long totalSyncDuration = 0; // total time spent syncing since
firstLagAt
+long syncExceededIntervalBy = 0; // time that syncs exceeded
pollInterval since firstLagAt
+int lagCount = 0;
+int syncCount = 0;
+
boolean run = true;
while (run)
{
@@ -73,14 +82,35 @@ public abstract class AbstractCommitLogService
lastSyncedAt = syncStarted;
syncComplete.signalAll();
+
// sleep any time we have left before the next one is
due
-long sleep = syncStarted + pollIntervalMillis -
System.currentTimeMillis();
+long now = System.currentTimeMillis();
+long sleep = syncStarted + pollIntervalMillis - now;
if (sleep 0)
{
-logger.warn(String.format(Commit log sync took
longer than sync interval (by %.2fs), indicating it is a bottleneck, sleep /
-1000d));
-// don't sleep, as we probably have work to do
-continue;
+// if we have lagged noticeably, update our lag
counter
+if (firstLagAt == 0)
+{
+firstLagAt = now;
+totalSyncDuration = syncExceededIntervalBy =
syncCount = lagCount = 0;
+}
+syncExceededIntervalBy -= sleep;
+lagCount++;
}
+syncCount++;
+totalSyncDuration += now - syncStarted;
+
+if (firstLagAt 0 now - firstLagAt =
LAG_REPORT_INTERVAL)
+{
+logger.warn(String.format(Out of %d commit log
syncs over the past %ds with average duration of %.2fms, %d have exceeded the
configured commit interval by an average of %.2fms,
+ syncCount, (now -
firstLagAt) / 1000, (double) totalSyncDuration / syncCount, lagCount, (double)
syncExceededIntervalBy / lagCount));
+firstLagAt = 0;
+}
+
+// if we have lagged this round, we probably have work
to do already so we don't sleep
+if (sleep 0)
+continue;
+
try
{
haveWork.tryAcquire(sleep, TimeUnit.MILLISECONDS);
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840868#comment-13840868 ] Jonathan Ellis commented on CASSANDRA-3578: --- committed Multithreaded commitlog --- Key: CASSANDRA-3578 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1 Attachments: 0001-CASSANDRA-3578.patch, 3578-logging-v2.txt, 3578-logging-v3.txt, ComitlogStress.java, Current-CL.png, Multi-Threded-CL.png, TestEA.java, latency.svg, oprate.svg, parallel_commit_log_2.patch Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate) can improve performance, since you're not bottlenecking on a single thread to do all the copying and CRC computation. Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable. (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6414) BlacklistingCompactionsTest fails in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840877#comment-13840877 ] Jonathan Ellis commented on CASSANDRA-6414: --- That should be purely cosmetic, and indeed reverting f3dc188e203b3db980ee81df05390968043cb601 still leaves BCT erroring out. BlacklistingCompactionsTest fails in trunk -- Key: CASSANDRA-6414 URL: https://issues.apache.org/jira/browse/CASSANDRA-6414 Project: Cassandra Issue Type: Bug Components: Tests Reporter: Jonathan Ellis Assignee: Michael Shuler Fix For: 2.1 Attachments: 6414_out.txt Passes in 2.0 HEAD. Bisect should be relatively easy. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-5872) Bundle JNA
[ https://issues.apache.org/jira/browse/CASSANDRA-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-5872: - Assignee: Lyuben Todorov (was: Brandon Williams) Bundle JNA -- Key: CASSANDRA-5872 URL: https://issues.apache.org/jira/browse/CASSANDRA-5872 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Lyuben Todorov Priority: Minor Fix For: 2.1 JNA 4.0 is reported to be dual-licensed LGPL/APL. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840883#comment-13840883 ] Jonathan Ellis commented on CASSANDRA-6268: --- Unless I'm missing something the 2.0 patch does not bump the Thrift version to 19.39.0. Poor performance of Hadoop if any DC is using VNodes Key: CASSANDRA-6268 URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Piotr Kołaczkowski Assignee: Piotr Kołaczkowski Attachments: 6268-src-1.2.txt, 6268-src-2.0.txt, 6268-thrift-1.2.txt, 6268-thrift-2.0.txt Some customers are complaining about huge number of splits in Hadoop caused by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are generated from the results of describe_ring, which returns a huge number of ranges anyways, and doesn't take into account that there will be huge number of consecutive ranges residing on the nodes we'd like the M/R job to be run. The proposed fix: 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - defaults to all Hadoop DCs) 2. merges consecutive ranges before generating Hadoop splits, so we don't have artificial range splitting caused by vnodes in the other DCs For non-DSE users this feature is turned off by default and doesn't change the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840899#comment-13840899 ] Jeremiah Jordan commented on CASSANDRA-6268: [~pkolaczk] Looks like interface/thrift/gen-java/org/apache/cassandra/thrift/Constants.java is missing from the thrift-2.0 patch Poor performance of Hadoop if any DC is using VNodes Key: CASSANDRA-6268 URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Piotr Kołaczkowski Assignee: Piotr Kołaczkowski Attachments: 6268-src-1.2.txt, 6268-src-2.0.txt, 6268-thrift-1.2.txt, 6268-thrift-2.0.txt Some customers are complaining about huge number of splits in Hadoop caused by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are generated from the results of describe_ring, which returns a huge number of ranges anyways, and doesn't take into account that there will be huge number of consecutive ranges residing on the nodes we'd like the M/R job to be run. The proposed fix: 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - defaults to all Hadoop DCs) 2. merges consecutive ranges before generating Hadoop splits, so we don't have artificial range splitting caused by vnodes in the other DCs For non-DSE users this feature is turned off by default and doesn't change the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6427) Counter writes shouldn't be resubmitted after timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840916#comment-13840916 ] Jonathan Ellis commented on CASSANDRA-6427: --- +1 Counter writes shouldn't be resubmitted after timeouts -- Key: CASSANDRA-6427 URL: https://issues.apache.org/jira/browse/CASSANDRA-6427 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Minor Fix For: 1.2.13, 2.0.4 Attachments: 6427.txt CASSANDRA-4753 made SP.counterWriteTask() return a LocalMutationRunnable instead of the usual DroppableRunnalbe, and LMR resubmits the original runnable in case of timing out instead of simply dropping it. For counters this is not the right option since it would lead to overcounting if the mutation got dropped-then-resubmitted and then retried by the user. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-4448) CQL3: allow to define a per-cf default consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840985#comment-13840985
]
Jonathan Ellis commented on CASSANDRA-4448:
---
This was removed in CASSANDRA-4734, between 1.2.0b1 and 1.2.0b2. So not only
was this not removed in 1.2.10 but it was never included in 1.2.0-final.
CQL3: allow to define a per-cf default consistency level
Key: CASSANDRA-4448
URL: https://issues.apache.org/jira/browse/CASSANDRA-4448
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 1.2.0 beta 1
Attachments: 4448.txt
One of the goal of CQL3 is that client library should not have to parse
queries to provide a good experience. In particular, that means such client
(that don't want to parse queries) won't be able to allow the user to define
a specific default read/write consistency level per-CF, forcing user to
specific the consistency level with every query, which is not very user
friendly.
This ticket suggests the addition of per-cf default read/write consitency
level. Typically the syntax would be:
{noformat}
CREATE TABLE foo (...)
WITH DEFAULT_READ_CONSISTENCY = QUORUM
AND DEFAULT_WRITE_CONSISTENCY = QUORUM
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-4448) CQL3: allow to define a per-cf default consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13840997#comment-13840997
]
David Webb commented on CASSANDRA-4448:
---
Here is a schema from DSE 3.1.3 / C* 1.2.6.6 that includes them. This is just
to let you know that I think they did sneak in there for a while. Some tools
like the Pentaho Cassandra connector were depending on their existence, and it
appears the were removed somewhere between 1.2.8 and 1.2.10. I cannot tell for
sure. Thanks for shedding light on the situation and we will proceed as if
they will not exists ever again. :) The connector will be updated soon. Thanks
Jonathan.
{code}
[cqlsh 3.1.2 | Cassandra 1.2.6.6 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
Use HELP for help.
cqlsh use system
... ;
cqlsh:system describe table schema_columnfamilies;
CREATE TABLE schema_columnfamilies (
keyspace_name text,
columnfamily_name text,
bloom_filter_fp_chance double,
caching text,
column_aliases text,
comment text,
compaction_strategy_class text,
compaction_strategy_options text,
comparator text,
compression_parameters text,
default_read_consistency text,
default_validator text,
default_write_consistency text,
gc_grace_seconds int,
id int,
key_alias text,
key_aliases text,
key_validator text,
local_read_repair_chance double,
max_compaction_threshold int,
min_compaction_threshold int,
populate_io_cache_on_flush boolean,
read_repair_chance double,
replicate_on_write boolean,
subcomparator text,
type text,
value_alias text,
PRIMARY KEY (keyspace_name, columnfamily_name)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='ColumnFamily definitions' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=8640 AND
read_repair_chance=0.00 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
{code}
CQL3: allow to define a per-cf default consistency level
Key: CASSANDRA-4448
URL: https://issues.apache.org/jira/browse/CASSANDRA-4448
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 1.2.0 beta 1
Attachments: 4448.txt
One of the goal of CQL3 is that client library should not have to parse
queries to provide a good experience. In particular, that means such client
(that don't want to parse queries) won't be able to allow the user to define
a specific default read/write consistency level per-CF, forcing user to
specific the consistency level with every query, which is not very user
friendly.
This ticket suggests the addition of per-cf default read/write consitency
level. Typically the syntax would be:
{noformat}
CREATE TABLE foo (...)
WITH DEFAULT_READ_CONSISTENCY = QUORUM
AND DEFAULT_WRITE_CONSISTENCY = QUORUM
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-4448) CQL3: allow to define a per-cf default consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841007#comment-13841007
]
Jonathan Ellis commented on CASSANDRA-4448:
---
If you upgraded from 1.2.0b1, we didn't bother ripping them out.
CQL3: allow to define a per-cf default consistency level
Key: CASSANDRA-4448
URL: https://issues.apache.org/jira/browse/CASSANDRA-4448
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 1.2.0 beta 1
Attachments: 4448.txt
One of the goal of CQL3 is that client library should not have to parse
queries to provide a good experience. In particular, that means such client
(that don't want to parse queries) won't be able to allow the user to define
a specific default read/write consistency level per-CF, forcing user to
specific the consistency level with every query, which is not very user
friendly.
This ticket suggests the addition of per-cf default read/write consitency
level. Typically the syntax would be:
{noformat}
CREATE TABLE foo (...)
WITH DEFAULT_READ_CONSISTENCY = QUORUM
AND DEFAULT_WRITE_CONSISTENCY = QUORUM
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-4448) CQL3: allow to define a per-cf default consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841010#comment-13841010
]
Jonathan Ellis commented on CASSANDRA-4448:
---
Ah, I see -- the actual columns weren't removed until cb0a0cd this August. But
they've been doing absolutely nothing in the meantime.
CQL3: allow to define a per-cf default consistency level
Key: CASSANDRA-4448
URL: https://issues.apache.org/jira/browse/CASSANDRA-4448
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 1.2.0 beta 1
Attachments: 4448.txt
One of the goal of CQL3 is that client library should not have to parse
queries to provide a good experience. In particular, that means such client
(that don't want to parse queries) won't be able to allow the user to define
a specific default read/write consistency level per-CF, forcing user to
specific the consistency level with every query, which is not very user
friendly.
This ticket suggests the addition of per-cf default read/write consitency
level. Typically the syntax would be:
{noformat}
CREATE TABLE foo (...)
WITH DEFAULT_READ_CONSISTENCY = QUORUM
AND DEFAULT_WRITE_CONSISTENCY = QUORUM
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6418) auto_snapshots are not removable via 'nodetool clearsnapshot'
[
https://issues.apache.org/jira/browse/CASSANDRA-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841036#comment-13841036
]
Mikhail Stepura commented on CASSANDRA-6418:
* {{getCFDirectory}} assumes that CF resides in a single location. If I'm
correct, a CF can be in multiple dirs from
{{DatabaseDescriptor.getAllDataFileLocations()}}
* I suspect {{File.listFiles()}} can return NULL if a directory doesn't exist
Minor remarks
* {{cfDir.exists() cfDir.isDirectory()}} can be reduced to just
{{cfDir.isDirectory()}}
* You can use {{dataFileLocations}} instead of
{{DatabaseDescriptor.getAllDataFileLocations()}}
auto_snapshots are not removable via 'nodetool clearsnapshot'
-
Key: CASSANDRA-6418
URL: https://issues.apache.org/jira/browse/CASSANDRA-6418
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: auto_snapshot: true
Reporter: J. Ryan Earl
Assignee: Lyuben Todorov
Priority: Minor
Fix For: 2.0.4
Attachments: 6418_cassandra-2.0.patch, 6418_v2.patch
Snapshots of deleted CFs created via the auto_snapshot configuration
parameter appear to not be tracked. The result is that 'nodetool
clearsnapshot keyspace with deleted CFs' does nothing, and short of
manually removing the files from the filesystem, deleted CFs remain
indefinitely taking up space.
I'm not sure if this is intended, but it seems pretty counter-intuitive. I
haven't found any documentation that indicates auto_snapshots would be
ignored by 'nodetool clearsnapshot'.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
