date:20130409

git commit: Decide on a gcBefore before sending out TreeRequests

2013-04-09 Thread marcuse

Updated Branches:
  refs/heads/trunk 3e0b21c45 - efbdee237


Decide on a gcBefore before sending out TreeRequests

patch by marcuse, reviewed by yukim, for CASSANDRA-4932


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/efbdee23
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/efbdee23
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/efbdee23

Branch: refs/heads/trunk
Commit: efbdee23705fed6be49a2fb20a79f18c98ff
Parents: 3e0b21c
Author: Marcus Eriksson marc...@spotify.com
Authored: Tue Apr 9 08:08:32 2013 +0200
Committer: Marcus Eriksson marc...@spotify.com
Committed: Tue Apr 9 08:08:32 2013 +0200

--
 CHANGES.txt|3 +-
 .../cassandra/db/compaction/CompactionManager.java |5 ++-
 .../cassandra/service/ActiveRepairService.java |   27 ++
 .../data/serialization/2.0/service.TreeRequest.bin |  Bin 121 - 129 bytes
 .../serialization/2.0/service.TreeResponse.bin |  Bin 930 - 946 bytes
 .../compaction/LeveledCompactionStrategyTest.java  |3 +-
 .../service/AntiEntropyServiceTestAbstract.java|3 +-
 .../cassandra/service/SerializationsTest.java  |2 +-
 8 files changed, 30 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/efbdee23/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bb00642..b322425 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -28,7 +28,8 @@
  * remove row-level bloom filters (CASSANDRA-4885)
  * Change Kernel Page Cache skipping into row preheating (disabled by default)
(CASSANDRA-4937)
-
+ * Improve repair by deciding on a gcBefore before sending
+   out TreeRequests (CASSANDRA-4932)
 
 1.2.4
  * Ensure that PerRowSecondaryIndex updates see the most recent values

http://git-wip-us.apache.org/repos/asf/cassandra/blob/efbdee23/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 64cf253..b5aefee 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -618,7 +618,10 @@ public class CompactionManager implements 
CompactionManagerMBean
 // we don't mark validating sstables as compacting in DataTracker, 
so we have to mark them referenced
 // instead so they won't be cleaned up if they do get compacted 
during the validation
 sstables = cfs.markCurrentSSTablesReferenced();
-gcBefore = getDefaultGcBefore(cfs);
+if (validator.request.gcBefore  0)
+gcBefore = validator.request.gcBefore;
+else
+gcBefore = getDefaultGcBefore(cfs);
 }
 
 CompactionIterable ci = new ValidationCompactionIterable(cfs, 
sstables, validator.request.range, gcBefore);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/efbdee23/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java 
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index f103771..f7a5fa4 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -442,7 +442,7 @@ public class ActiveRepairService
 public void doVerb(MessageInTreeRequest message, int id)
 {
 TreeRequest remotereq = message.payload;
-TreeRequest request = new TreeRequest(remotereq.sessionid, 
message.from, remotereq.range, remotereq.cf);
+TreeRequest request = new TreeRequest(remotereq.sessionid, 
message.from, remotereq.range, remotereq.gcBefore, remotereq.cf);
 
 // trigger read-only compaction
 ColumnFamilyStore store = 
Table.open(request.cf.left).getColumnFamilyStore(request.cf.right);
@@ -462,7 +462,7 @@ public class ActiveRepairService
 {
 // deserialize the remote tree, and register it
 Validator response = message.payload;
-TreeRequest request = new TreeRequest(response.request.sessionid, 
message.from, response.request.range, response.request.cf);
+TreeRequest request = new TreeRequest(response.request.sessionid, 
message.from, response.request.range, response.request.gcBefore, 
response.request.cf);
 ActiveRepairService.instance.rendezvous(request, response.tree);
 }
 }
@@ -489,20 +489,22

[Cassandra Wiki] Trivial Update of SoilaGray75 by SoilaGray75

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The SoilaGray75 page has been changed by SoilaGray75:
http://wiki.apache.org/cassandra/SoilaGray75

New page:
32 years old Armed forces band member Johanna is virtually hooked on Used 
automobile sales, rock, jet engines.BR
Last of all he gets motivated by exploring and heading for different cities and 
foreign territories specifically to Moguer.BR
BR
my web page ... 
[[http://videos.Sweethazzard.com/read_blog/14462/euro-weakness-boost-for-tax-havens|http://videos.Sweethazzard.com/read_blog/14462/euro-weakness-boost-for-tax-havens]]

[jira] [Commented] (CASSANDRA-5444) TTL does not compact rows

2013-04-09 Thread Jeremy Hanna (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626381#comment-13626381
]

Jeremy Hanna commented on CASSANDRA-5444:
-

Brandon: to be clear, they saw that when they did the alter table command, it
would continue to have the range ghost after the ttl + gc_grace + flush +
compact. However if they create a new column family with that gc_grace and do
the same process, the range ghost is no longer there.

TTL does not compact rows
--

Key: CASSANDRA-5444
URL: https://issues.apache.org/jira/browse/CASSANDRA-5444
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.9
Environment: Linux
Reporter: Sathish
Fix For: 1.1.9

I find that the rows are not compacted when gc_grace_seconds is set through
alter table command. Following were the steps performed:
1. Created the following table:
CREATE COLUMNFAMILY MirrorSmokeTest (
id uuid PRIMARY KEY,
test_message varchar
);
2. Ran an alter command:
alter table MirrorSmokeTest WITH gc_grace_seconds = 60;
3. Insert a row into the above table:
insert into MirrorSmokeTest (id, test_message) Values
('4a7fad82-8298-4d91-85de-8255b7c7e4f5', '2') USING ttl 60;
4. Waited for 60 seconds and ran a query that results in the following output:
select * from MirrorSmokeTest;
id
--
4a7fad82-8298-4d91-85de-8255b7c7e4f5
5. Ran a flush and compact and reran the query that results in the following:
select * from MirrorSmokeTest;
id
--
4a7fad82-8298-4d91-85de-8255b7c7e4f5

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Trivial Update of YvonneCul by YvonneCul

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The YvonneCul page has been changed by YvonneCul:
http://wiki.apache.org/cassandra/YvonneCul

New page:
Not much to say about myself I think.BR
Lovely to be a part of apache.org.BR
I just hope Im useful in some way here.BR
BR
Check out my homepage ... 
[[http://www.rentacarlasrosas.com/tenerife/cheap-car-rental-for-south-tenerife-and-the-airport/1282/|car
 rental tenerife airport]]

[jira] [Commented] (CASSANDRA-5444) TTL does not compact rows

2013-04-09 Thread Sylvain Lebresne (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626397#comment-13626397
]

Sylvain Lebresne commented on CASSANDRA-5444:
-

To start with, in the TTL example you need to wait of at least 120 seconds (TTL
+ gc_grace), not just 60 seconds as in the description above. The 2nd thing to
check is to make sure a compaction did occur. For instance, if you use LCS and
there is only 1 sstable, it won't compact it if memory serves.

TTL does not compact rows
--

[Cassandra Wiki] Trivial Update of ClintonFo by ClintonFo

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The ClintonFo page has been changed by ClintonFo:
http://wiki.apache.org/cassandra/ClintonFo

New page:
Lorene Steadham is what's written on my birth certificate although it is not 
the name on my birth certificate. My job is a meter reader. I've always loved 
living in Alaska and will never move.BR
To canoe is something I really enjoy doing. I've been working on my website for 
some time now. Check it out here: 
http://michiganjunkyard.com/used-auto-parts-in-ann-arbor-michigan/

[jira] [Updated] (CASSANDRA-5074) Add an official way to disable compaction

2013-04-09 Thread Marcus Eriksson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-5074:
---

Attachment: 0001-CASSANDRA-5074-v2.patch

+1 on all the comments

now it is a compaction_strategy_option in cli, and a compaction = {..} option 
in CQL.

i also removed a call to cfs.setCompactionThresholds in the STCS constructor, 
couldn't find a reason it was there.

and, if there is a better way of disallowing min/max thresholds = 0 than 
checking in cql, cql3 and cli that does not break existing schemas, let me know

 Add an official way to disable compaction
 -

 Key: CASSANDRA-5074
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5074
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.0

 Attachments: 
 0001-CASSANDRA-5074-make-it-possible-to-disable-autocompa.patch, 
 0001-CASSANDRA-5074-v2.patch


 We've traditionally used min or max compaction threshold = 0 to disable 
 compaction, but this isn't exactly intuitive and it's inconsistently 
 implemented -- allowed from jmx, not allowed from cli.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-5422) Binary protocol sanity check

2013-04-09 Thread Marcus Eriksson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-5422:
--

Assignee: Daniel Norberg  (was: Marcus Eriksson)

 Binary protocol sanity check
 

 Key: CASSANDRA-5422
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5422
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Daniel Norberg
 Attachments: 5422-test.txt


 With MutationStatement.execute turned into a no-op, I only get about 33k 
 insert_prepared ops/s on my laptop.  That is: this is an upper bound for our 
 performance if Cassandra were infinitely fast, limited by netty handling the 
 protocol + connections.
 This is up from about 13k/s with MS.execute running normally.
 ~40% overhead from netty seems awfully high to me, especially for 
 insert_prepared where the return value is tiny.  (I also used 4-byte column 
 values to minimize that part as well.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Trivial Update of JoleneMjf by JoleneMjf

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The JoleneMjf page has been changed by JoleneMjf:
http://wiki.apache.org/cassandra/JoleneMjf

New page:
Roger is what you can call him up but it's not the quite a number of masucline 
name out there. As a man what he really loves is climbing but he will not make 
it his profession. Some time in he chose to live over Alaska but his wife would 
love them to move. Hiring precisely what he does in his employment. If you want 
to find uot more the look at his website: 
http://www.brbdumpsterrental.com/florida/dumpster-rental-in-bradenton-fl/

[Cassandra Wiki] Trivial Update of MerissaCP by MerissaCP

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The MerissaCP page has been changed by MerissaCP:
http://wiki.apache.org/cassandra/MerissaCP

New page:
Not much to say about myself at all.BR
Hurrey Im here and a part of apache.org.BR
I really hope I'm useful at allBR
BR
Feel free to visit my page: [[http://www.lagbook.com/blogs/4219573|Vehicle 
insurance Ratings India]]

[jira] [Commented] (CASSANDRA-2698) Instrument repair to be able to assess it's efficiency (precision)

2013-04-09 Thread Benedict (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626520#comment-13626520
]

Benedict commented on CASSANDRA-2698:
-

Hi Yuki,

Without in some way collecting (or at least sampling) the size of the
differences, I don't know what bucket sizes to use. Since I need to reinsert
all the records once I've decided this anyway, I need to retain them all, which
I chose to do in EstimatedHistogram as they do, in effect, constitute a
histogram. I also sample the largest records which I figure could be useful for
debugging purposes (though that was just a guess). I don't see why 1000s of
items is a major issue?

I agree that logging is suboptimal for this data. Presumably similar data for
other tasks may be optionally logged in future, and so I would guess this
should form part of a discussion about metric logging?

{quote}
fix coding style (especially whitespace) to match other code.
{quote}
Do you have an Eclipse formatter profile I could use for your coding
convention? I did my best to keep it correct manually, but it is difficult to
spot differences in an unfamiliar convention. Whitespace should be
comparatively easy though.

{quote}
EstimatedHistogram#testGroupBy is failing.
{quote}
Noted - will fix and resubmit

{quote}
comparator in Arrays#sort in EstimatedHistogram#logSummary has the same
conditions in both if and else if.
{quote}
Thanks, good spot. I'm surprised Eclipse didn't warn me.

Instrument repair to be able to assess it's efficiency (precision)
--

Key: CASSANDRA-2698
URL: https://issues.apache.org/jira/browse/CASSANDRA-2698
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benedict
Priority: Minor
Labels: lhf
Attachments: nodetool_repair_and_cfhistogram.tar.gz,
patch_2698_v1.txt, patch.diff, patch-rebased.diff

Some reports indicate that repair sometime transfer huge amounts of data. One
hypothesis is that the merkle tree precision may deteriorate too much at some
data size. To check this hypothesis, it would be reasonably to gather
statistic during the merkle tree building of how many rows each merkle tree
range account for (and the size that this represent). It is probably an
interesting statistic to have anyway.

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

2013-04-09 Thread T Jake Luciani (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626570#comment-13626570
]

T Jake Luciani commented on CASSANDRA-5417:
---

I started looking at this and so far my comments/questions are:

-Why do we need AbstractCompositeType now that we use AbstractCType?
-Why are you using the word 'Dense' internally when CQL3 uses 'Compact'? I'd
prefer to see s/Dense/Compact/

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

CompositeType happens to be very useful and is now widely used: CQL3 heavily
rely on it, and super columns are now using it too internally. Besides,
CompositeType has been advised as a replacement of super columns on the
thrift side for a while, so it's safe to assume that it's generally used
there too.
CompositeType has initially been introduced as just another AbstractType.
Meaning that the storage engine has no nothing whatsoever of composites
being, well, composite. This has the following drawbacks:
* Because internally a composite value is handled as just a ByteBuffer, we
end up doing a lot of extra work. Typically, each time we compare 2 composite
value, we end up deserializing the components (which, while it doesn't copy
data per-se because we just slice the global ByteBuffer, still waste some cpu
cycles and allocate a bunch of ByteBuffer objects). And since compare can be
called *a lot*, this is likely not negligible.
* This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive
use of composites, and since it gets backs ByteBuffer from the internal
columns, it always have to check if it's actually a compositeType or not, and
then split it and pick the different parts it needs. It's only an API
problem, but having things exposed as composites directly would definitively
make thinks cleaner. In particular, in most cases, CQL3 don't care whether it
has a composite with only one component or a non-really-composite value, but
we still always distinguishes both cases. Lastly, if we do expose composites
more directly internally, it's not a lot more work to internalize better
the different parts of the cell name that CQL3 uses (what's the clustering
key, what's the actuall CQL3 column name, what's the collection element),
making things cleaner. Last but not least, there is currently a bunch of
places where methods take a ByteBuffer as argument and it's hard to know
whether it expects a cell name or a CQL3 column name. This is pretty error
prone.
* It makes it hard (or impossible) to do a number of performance
improvements. Consider CASSANDRA-4175, I'm not really sure how you can do it
properly (in memory) if cell names are just ByteBuffer (since CQL3 column
names are just one of the component in general). But we also miss
oportunities of sharing prefixes. If we were able to share prefixes of
composite names in memory we would 1) lower the memory footprint and 2)
potentially speed-up comparison (of the prefixes) by checking reference
equality first (also, doing prefix sharing on-disk, which is a separate
concern btw, might be easier to do if we do prefix sharing in memory).
So I suggest pushing CompositeType support inside the storage engine. What I
mean by that concretely would be change the internal {{Column.name}} from
ByteBuffer to some CellName type. A CellName would API-wise just be a list of
ByteBuffer. But in practice, we'd have a specific CellName implementation for
not-really-composite names, and the truly composite implementation will allow
some prefix sharing. From an external API however, nothing would change, we
would pack the composite as usual before sending it back to the client, but
at least internally, comparison won't have to deserialize the components
every time, and CQL3 code will be cleaner.

[jira] [Commented] (CASSANDRA-5062) Support CAS

2013-04-09 Thread Sylvain Lebresne (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626574#comment-13626574
]

Sylvain Lebresne commented on CASSANDRA-5062:
-

bq. I don't think PaxosState introduces artificial contention in the sense that
we do need to serialize paxos operations, so this is definitely better than One
Big Lock

Sorry that wasn't clear, I was think of contention between proposers, not
contention local to a replica. Because we bucket the paxos states, we will
reuse the same state for different keys. So in other words, the current
algorithm does not do Paxos at the row level, but rather paxos at the level of
all rows whose key's hashcode modulo 1024 is equal, and 2 proposers on 2
different row keys may compete with each other.

And since proposer contention is so damn expensive, I wonder if that artificial
contention won't trump any gain we have from keeping the paxos state in memory
(in cases where people have a non-trivial load of CAS operation, even though it
is on different row keys). Sure we could play with the STATE_BUCKETS value,
but I'm still of the opinion that it's premature optimization at this point.

On the part about serializing Paxos operations on a given replica, I agree with
you that it's necessary, and for that we could absolutely use lock buckets
(like we use to do for 2ndary indexes) to optimize. But we can do that without
bucketing the whole paxos states.

As a side note, I think that we may want to TTL the Paxos state (we do store
the MRC update after all, if someone has done an CAS on a row and don't do any
CAS anymore, it would be sad to keep that MRC forever). And we can use a
setting like gc_grace, since provided you do guarantee that everything older
than gc_grace is consistent, then you can forget about older than gc_grace
states. Of course, in practice a setting separated from gc_grace would make
sense.

bq. Would rather make them follow-on tickets

I'm fine with that. Just wanted to mention it cause I don't think we should
release CAS without those tickets done. Maybe we can create a top-level ticket
to group all those tickets, making sure we don't forget something important?

bq. We wait for as many nodes as were alive at the start of the paxos request.

Fair enough, I misread that part. *But*, this in turn means that we'll timeout
as soon as any node that we though were alive happens to be dead, even if we
have a QUORUM of responses. And since the failure detector takes in general at
least a handful of seconds to detect a node, it means that each time a node
dies, *all* CAS operations will timeout until at least one other node detects
it dead (and while it's true that currently some reads may timeout when a node
dies, this is at least not all reads and besides, it's known to be bad for
latency and one of the reason why CASSANDRA-4705 is useful). Overall I think we
should make progress as soon as we have QUORUM answers.

Furthermore, another problem is that we throw an exception if we haven't a
quorum on the last MRC. But that means in turns that if amongst all live nodes
we don't have a QUORUM of nodes that have learn that last MRC, we don't do
anything to fix the issue (we throw an exception in fact). So if we have really
lost messages, until something external magically repair the nodes that are
missing the last MRC, we'll be in a state when no CAS operation on the row can
proceed at all. Even believing that hinted handoff will be that external
repair, which would assume that hinted handoffs are bullet-proof (and they are
not), there could still be a reasonably long window of unavailability just
because HH delivery is not prioritized/timely.

Lastly, I note that it's not unlikely not to get a QUORUM of node on the last
MRC due to timing, because the last MRC commit may have reach some replica but
not all of them (it will even likely be common when proposers contend, because
the one that gets to be 2nd will do it's prepare phase more or less at the same
time the previous one does his commit). And in that case, note that nothing is
wrong with the cluster per se, all replica may be alive. And thus throwing an
UnavailableException in that case would imo be a very unfriendly user
experience.

So to recall, what I suggest instead would be to 1) only wait for QUORUM
responses on prepare then 2) if not everyone is on the same last MRC, then
repair those are aren't by committing said last MRC (with the MRC ballot in
particular, not our ballot. This is not part of the Paxos algorithm per-se,
this is just us re-sending the learn message of the previous round to make sure
a quorum has it) and then 3) restart from the beginning (i.e. re-prepare).

I believe this does not suffer from the problems above: we prepare, and if the
first quorum we get hasn't got the last MRC yet (which can be due to timing or

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

2013-04-09 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13626622#comment-13626622
 ] 

Sylvain Lebresne commented on CASSANDRA-5417:
-

bq. Why do we need AbstractCompositeType now that we use AbstractCType?

One reason is that people may use a CompositeType as column value validator for 
instance. Removing it would break those. And as of this patch, CompositeType is 
still used for the row key even if it's composite. On that last part, I do 
think we should replace it by CType internally at some point because that would 
make sense, but I felt the current patch was big enough currently that this 
could wait a followup ticket.

Another reason is that it's simpler for compatibility. The fact is, even if we 
don't use CompositeType internally, for compatibility sake we need to style 
parse and return stuffs like CompositeType(UTF8Type, IntegerType) in thrift 
CfDef and such. And more generally, there is a few of places in the code where 
it's just simpler to just convert the CType to an equivalent old style 
CompositeType (the cql2 has some of those for instance). Another more minor 
reason is that if someone is crazy enough to use a super column with a 
compositeType comparator or subcomparator today, we'll still use the old 
CompositeType. As for AbstractCompositeType specifically, it's also reused for 
DynamicCompositeType.

That being said, I think that if we do the follow up ticket I mention above of 
using CType for composite row keys to, then I think we can remove some bits 
like the CompositeType.Builder.

bq. Why are you using the word 'Dense' internally when CQL3 uses 'Compact'?

Because they are not equivalent :). I'll admit, this could just be a bad choice 
of terms, but I haven't found much better. But anyway, internally there is 2 
dimensions to the layouts used distinguished by CQL3 which are:
# whether the comparator is either composite or not. In the code I use 
isPacked() to mean not composite at times, though that's also a I haven't 
found better term.
# whether one of the component (the last one or 2nd to last if there is a 
collection) of the comparator stores the CQL3 column name. If that's the case, 
then the CF is not 'dense' (I might use sparse at times in that case), 
otherwise it is. Another more pragramatic to see that is to say that dense CFs 
are those where 1 CQL3 row == 1 internal cell.

Those are the distinction that are useful internally.

But while the 4 combinations of those 2 dimensions make sense, only one of them 
(non packed, non dense) is the default CQL3 layout. All the others are 
created in CQL3 using WITH COMPACT STORAGE. And so in particular, a COMPACT 
static CF, i.e. something like
{noformat}
CREATE TABLE foo {
  k int PRIMARY KEY,
  c1 int,
  c2 text,
  c3 double
} WITH COMPACT STORAGE
{noformat}
is *not* dense (but it is packed). One CQL3 row correspond to a group of 
internal cells.

I'm happy to change the term 'dense' internally if someone has a better 
suggestion, but using 'compact' would confuse things more than anything imo.

 Push composites support in the storage engine
 -

 Key: CASSANDRA-5417
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.0


 CompositeType happens to be very useful and is now widely used: CQL3 heavily 
 rely on it, and super columns are now using it too internally. Besides, 
 CompositeType has been advised as a replacement of super columns on the 
 thrift side for a while, so it's safe to assume that it's generally used 
 there too.
 CompositeType has initially been introduced as just another AbstractType.  
 Meaning that the storage engine has no nothing whatsoever of composites 
 being, well, composite. This has the following drawbacks:
 * Because internally a composite value is handled as just a ByteBuffer, we 
 end up doing a lot of extra work. Typically, each time we compare 2 composite 
 value, we end up deserializing the components (which, while it doesn't copy 
 data per-se because we just slice the global ByteBuffer, still waste some cpu 
 cycles and allocate a bunch of ByteBuffer objects). And since compare can be 
 called *a lot*, this is likely not negligible.
 * This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive 
 use of composites, and since it gets backs ByteBuffer from the internal 
 columns, it always have to check if it's actually a compositeType or not, and 
 then split it and pick the different parts it needs. It's only an API 
 problem, but having things exposed as composites directly would definitively 
 make thinks cleaner. In particular, in most cases, CQL3 don't care whether it

[Cassandra Wiki] Trivial Update of MinnieWZD by MinnieWZD

2013-04-09 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The MinnieWZD page has been changed by MinnieWZD:
http://wiki.apache.org/cassandra/MinnieWZD

New page:
小売店の地主は積極的に見つけるしようとしています。ポップアップ格納に変更失敗ベンダー。期間、このメトロポリスできます感じるのような海苦痛の。BR
BR
Feel free to visit my homepage: 
[[http://www.lsucb.ro/forum/membru/2457-noahphill/|Website personal 
http://www.ruivitonja.com/]]

98 matches

Mail list logo