[jira] [Issue Comment Edited] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[
https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057419#comment-13057419
]
Jonathan Ellis edited comment on CASSANDRA-2045 at 6/29/11 7:36 PM:
bq. losing hints during upgrade isn't the end of the world
Right. I'm saying we should do this:
{noformat}
Hints: {// cf
dest ip: { // key
key: {// super-column
table-cf: id// column
mutation: mutation // column
}
}
}
{noformat}
So we denormalize but we gain not having to do secondary-lookup-per-mutation,
which is our main motivation for the change. (And single-destination-per-hint
is by far the common case.)
bq. Can we open this as a dependent ticket?
WFM.
was (Author: jbellis):
bq. losing hints during upgrade isn't the end of the world
Right. I'm saying we should do this:
Hints: {// cf
dest ip: { // key
key: {// super-column
table-cf: id// column
mutation: mutation // column
}
}
}
So we denormalize but we gain not having to do secondary-lookup-per-mutation,
which is our main motivation for the change. (And single-destination-per-hint
is by far the common case.)
bq. Can we open this as a dependent ticket?
WFM.
Simplify HH to decrease read load when nodes come back
--
Key: CASSANDRA-2045
URL: https://issues.apache.org/jira/browse/CASSANDRA-2045
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Goffinet
Assignee: Nicholas Telford
Fix For: 1.0
Attachments:
0001-Changed-storage-of-Hints-to-store-a-serialized-RowMu.patch,
0002-Refactored-HintedHandoffManager.sendRow-to-reduce-co.patch,
0003-Fixed-some-coding-style-issues.patch,
0004-Fixed-direct-usage-of-Gossiper.getEndpointStateForEn.patch,
0005-Removed-duplicate-failure-detection-conditionals.-It.patch,
0006-Removed-handling-of-old-style-hints.patch, 2045-v3.txt,
CASSANDRA-2045-simplify-hinted-handoff-001.diff,
CASSANDRA-2045-simplify-hinted-handoff-002.diff
Currently when HH is enabled, hints are stored, and when a node comes back,
we begin sending that node data. We do a lookup on the local node for the row
to send. To help reduce read load (if a node is offline for long period of
time) we should store the data we want forward the node locally instead. We
wouldn't have to do any lookups, just take byte[] and send to the destination.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[
https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057466#comment-13057466
]
Jonathan Ellis edited comment on CASSANDRA-2045 at 6/29/11 8:58 PM:
oops, didn't look too closely to what I was pasting.
{noformat}
Hints: {// cf
dest ip: { // key
uuid: { // super-column
table: table// columns
key: key
mutation: mutation
}
}
}
{noformat}
(Mutations can contain multiple CFs so storing a single CF value wouldn't make
sense.)
was (Author: jbellis):
oops, didn't look too closely to what I was pasting.
{noformat}
Hints: {// cf
dest ip: { // key
uuid: { // super-column
table: table// columns
key: key
mutation: mutation
}
}
}
(Mutations can contain multiple CFs so storing a single CF value wouldn't make
sense.)
Simplify HH to decrease read load when nodes come back
--
Key: CASSANDRA-2045
URL: https://issues.apache.org/jira/browse/CASSANDRA-2045
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Goffinet
Assignee: Nicholas Telford
Fix For: 1.0
Attachments:
0001-Changed-storage-of-Hints-to-store-a-serialized-RowMu.patch,
0002-Refactored-HintedHandoffManager.sendRow-to-reduce-co.patch,
0003-Fixed-some-coding-style-issues.patch,
0004-Fixed-direct-usage-of-Gossiper.getEndpointStateForEn.patch,
0005-Removed-duplicate-failure-detection-conditionals.-It.patch,
0006-Removed-handling-of-old-style-hints.patch, 2045-v3.txt,
CASSANDRA-2045-simplify-hinted-handoff-001.diff,
CASSANDRA-2045-simplify-hinted-handoff-002.diff
Currently when HH is enabled, hints are stored, and when a node comes back,
we begin sending that node data. We do a lookup on the local node for the row
to send. To help reduce read load (if a node is offline for long period of
time) we should store the data we want forward the node locally instead. We
wouldn't have to do any lookups, just take byte[] and send to the destination.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[ https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986185#comment-12986185 ] Ryan King edited comment on CASSANDRA-2045 at 6/23/11 5:18 PM: --- I think the two approaches are suitable for different kinds of data models. The pointer approach is almost certainly better for narrow rows, while worse for large, dynamic rows. was (Author: kingryan): I think the two approaches are suitable for different kinds of data models. The point approach is almost certainly better for narrow rows, while worse for large, dynamic rows. Simplify HH to decrease read load when nodes come back -- Key: CASSANDRA-2045 URL: https://issues.apache.org/jira/browse/CASSANDRA-2045 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Nicholas Telford Fix For: 1.0 Attachments: 0001-Changed-storage-of-Hints-to-store-a-serialized-RowMu.patch, 0002-Refactored-HintedHandoffManager.sendRow-to-reduce-co.patch, 0003-Fixed-some-coding-style-issues.patch, 0004-Fixed-direct-usage-of-Gossiper.getEndpointStateForEn.patch, 0005-Removed-duplicate-failure-detection-conditionals.-It.patch, 0006-Removed-handling-of-old-style-hints.patch, CASSANDRA-2045-simplify-hinted-handoff-001.diff, CASSANDRA-2045-simplify-hinted-handoff-002.diff Currently when HH is enabled, hints are stored, and when a node comes back, we begin sending that node data. We do a lookup on the local node for the row to send. To help reduce read load (if a node is offline for long period of time) we should store the data we want forward the node locally instead. We wouldn't have to do any lookups, just take byte[] and send to the destination. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[
https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13048625#comment-13048625
]
Nicholas Telford edited comment on CASSANDRA-2045 at 6/13/11 4:18 PM:
--
* Rebased patch to latest trunk.
* Changed serialization to use cached RowMutation.getSerializedBuffer
I agree with your thoughts on removing the old-style hinting, although I
haven't given much thought to the impact on compatibility. As far as I can
tell, using my patch removing the old style hinting would be a matter of
removing {{hintedMutation.apply();}} from {{RowMutationVerbHandler@63}}. For
now, I've left it as-is.
I tried to keep to the CodingStyle as much as possible; is there anything
specific you noticed that was wrong?
Btw, the tests I promised have been delayed by nightmarish work-schedule. I'll
try to get them sorted ASAP though.
was (Author: nicktelford):
* Rebased patch to latest trunk.
* Changed serialization to use cached RowMutation.getSerializedBuffer
I agree with your thoughts on removing the old-style hinting, although I
haven't given the impact on compatibility. As far as I can tell, using my patch
removing the old style hinting would be a matter of removing
{{hintedMutation.apply();}} from {{RowMutationVerbHandler@63}}. For now, I've
left it as-is.
I tried to keep to the CodingStyle as much as possible; is there anything
specific you noticed that was wrong?
Btw, the tests I promised have been delayed by nightmarish work-schedule. I'll
try to get them sorted ASAP though.
Simplify HH to decrease read load when nodes come back
--
Key: CASSANDRA-2045
URL: https://issues.apache.org/jira/browse/CASSANDRA-2045
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Goffinet
Assignee: Nicholas Telford
Fix For: 1.0
Attachments: CASSANDRA-2045-simplify-hinted-handoff-001.diff,
CASSANDRA-2045-simplify-hinted-handoff-002.diff
Currently when HH is enabled, hints are stored, and when a node comes back,
we begin sending that node data. We do a lookup on the local node for the row
to send. To help reduce read load (if a node is offline for long period of
time) we should store the data we want forward the node locally instead. We
wouldn't have to do any lookups, just take byte[] and send to the destination.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[
https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13048625#comment-13048625
]
Nicholas Telford edited comment on CASSANDRA-2045 at 6/13/11 4:19 PM:
--
* Rebased patch to latest trunk.
* Changed serialization to use cached RowMutation.getSerializedBuffer
I agree with your thoughts on removing the old-style hinting, although I
haven't given much thought to the impact on compatibility. As far as I can
tell, using my patch and removing the old style hinting would be a matter of
removing {{hintedMutation.apply();}} from {{RowMutationVerbHandler@63}}. For
now, I've left it as-is.
I tried to keep to the CodingStyle as much as possible; is there anything
specific you noticed that was wrong?
Btw, the tests I promised have been delayed by nightmarish work-schedule. I'll
try to get them sorted ASAP though.
was (Author: nicktelford):
* Rebased patch to latest trunk.
* Changed serialization to use cached RowMutation.getSerializedBuffer
I agree with your thoughts on removing the old-style hinting, although I
haven't given much thought to the impact on compatibility. As far as I can
tell, using my patch removing the old style hinting would be a matter of
removing {{hintedMutation.apply();}} from {{RowMutationVerbHandler@63}}. For
now, I've left it as-is.
I tried to keep to the CodingStyle as much as possible; is there anything
specific you noticed that was wrong?
Btw, the tests I promised have been delayed by nightmarish work-schedule. I'll
try to get them sorted ASAP though.
Simplify HH to decrease read load when nodes come back
--
Key: CASSANDRA-2045
URL: https://issues.apache.org/jira/browse/CASSANDRA-2045
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Goffinet
Assignee: Nicholas Telford
Fix For: 1.0
Attachments: CASSANDRA-2045-simplify-hinted-handoff-001.diff,
CASSANDRA-2045-simplify-hinted-handoff-002.diff
Currently when HH is enabled, hints are stored, and when a node comes back,
we begin sending that node data. We do a lookup on the local node for the row
to send. To help reduce read load (if a node is offline for long period of
time) we should store the data we want forward the node locally instead. We
wouldn't have to do any lookups, just take byte[] and send to the destination.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[ https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986087#action_12986087 ] Chris Goffinet edited comment on CASSANDRA-2045 at 1/24/11 6:20 PM: I disagree, storing a pointer is not almost free. The trade off you make is read performance when nodes come back up. At the moment read performance could very well cause cascading failures too. You trade storage vs I/O. Nodes down for long periods of time, would have a bigger impact on the nodes trying to send HH data. Can you clarify, aren't we sending the entire row as well? We might be doing paging but if I modify 1 column, and it has 1M columns, 1M + 1 columns still get sent? I would agree to having it be tunable, but I prefer consistency of performance over storage when it comes to I/O. We brought this up because we actually saw this in our live clusters. was (Author: lenn0x): No, storing a pointer is not almost free. The trade off you make is read performance when nodes come back up. At the moment read performance could very well cause cascading failures too. You trade storage vs I/O. Nodes down for long periods of time, would have a bigger impact on the nodes trying to send HH data. Aren't we sending the entire row as well? We might be doing paging but if I modify 1 column, and it has 1M columns, 1M + 1 columns still get sent? I would agree to having it be tunable, but I prefer performance consistency over storage when it comes to I/O. Simplify HH to decrease read load when nodes come back -- Key: CASSANDRA-2045 URL: https://issues.apache.org/jira/browse/CASSANDRA-2045 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Fix For: 0.7.2 Currently when HH is enabled, hints are stored, and when a node comes back, we begin sending that node data. We do a lookup on the local node for the row to send. To help reduce read load (if a node is offline for long period of time) we should store the data we want forward the node locally instead. We wouldn't have to do any lookups, just take byte[] and send to the destination. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
