[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301179#comment-17301179
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 3/14/21, 3:46 PM:
--
bq. The patch breaks the CommitLogSegment's closing process (whether capacity
becomes full, or endOfBuffer gets re-assigned before buffer becomes null).
Investigating this further…
CommitLogSegment depended on the {{compareAndSet}}, to keep {{allocatePosition
< endOfBuffer}}. Given that the atomic add approach isn't applicable to this
code, I have applied the ConstantBackoffCAS algorithm (see CASSANDRA-15922)
here.
\\
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
–
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/482/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/482/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
–
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/483/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/483/pipeline]
was (Author: michaelsembwever):
bq. The patch breaks the CommitLogSegment's closing process (whether capacity
becomes full, or endOfBuffer gets re-assigned before buffer becomes null).
Investigating this further…
CommitLogSegment depended on the {{compareAndSet}}, to keep {{allocatePosition
< endOfBuffer}}. Given that the atomic add approach isn't applicable to this
code, I have applied the ConstantBackoffCAS algorithm (see CASSANDRA-15922)
here.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
–
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/482/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/482/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
–
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/483/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/483/pipeline]
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188367#comment-17188367
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 9/3/20, 1:18 PM:
-
Thanks for the feedback [~blerer].
bq. If the slab size is around 1GB, the maximum hint size will be around 500MB.
I see now that the slab size can in fact be 2GB, and max mutation size 1GB.
This makes the problem worse (even if very edge-case). {{HintsBuffer.position}}
has been changed to {{AtomicLong}}
bq. Regarding CommitLogSegment, it will be good to have a comment explaining
the negative value logic.
Done. Two comments added, explaining when the overflow is harmless, and when it
isn't and hence the cast to long.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/2/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/3/pipeline]
was (Author: michaelsembwever):
Thanks for the feedback [~blerer].
bq. If the slab size is around 1GB, the maximum hint size will be around 500MB.
I see now that the slab size can in fact be 2GB, and max mutation size 1GB.
This makes the problem worse (even if very edge-case). {{HintsBuffer.position}}
has been changed to {{AtomicLong}}
bq. Regarding CommitLogSegment, it will be good to have a comment explaining
the negative value logic.
Done. Two comments added, explaining when the overflow is harmless, and when it
isn't and hence the cast to long.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/301/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/303/pipeline]
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186739#comment-17186739
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/29/20, 7:18 PM:
--
[~blerer], {{HintsBuffer}} look should better now.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/296/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/297/pipeline]
was (Author: michaelsembwever):
[~blerer], {{HintsBuffer}} look should better now.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/294/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/295/pipeline]
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186739#comment-17186739
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/28/20, 6:08 PM:
--
[~blerer], {{HintsBuffer}} look should better now.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/294/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/295/pipeline]
was (Author: michaelsembwever):
[~blerer], {{HintsBuffer}} look should better now.
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/292/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/293/pipeline]
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186401#comment-17186401
]
Benjamin Lerer edited comment on CASSANDRA-16072 at 8/28/20, 9:14 AM:
--
I might be missing something but {{HintsBuffer.allocateBytes}} logic seems
wrong.
{code}
private int allocateBytes(int totalSize)
{
int prev = position.getAndAdd(totalSize);
if (prev == CLOSED) // the slab has been 'closed'
return CLOSED;
if ((prev + totalSize) > slab.capacity())
{
position.set(CLOSED); // mark the slab as no longer allocating if
we've exceeded its capacity
return CLOSED;
}
return prev;
}
{code}
As the following scenario will not produce the expected out come:
# Thread A calls {{allocateBytes}} and exit with the return value {{CLOSED}}
and {{position == CLOSED}}
# Thead B calls {{allocateBytes}} with {{totalSize == size_B}} and exit with
the return value {{CLOSED}} and {{position == size_B -1}} as {{CLOSED == -1}}
in practice.
# Thread C calls {{allocateBytes}} with {{totalSize == size_C}} and exit with
the return value {{size_B - 1}} and and {{position == size_B + size_C -1}}
was (Author: blerer):
I might be missing something but {{HintsBuffer.allocateBytes}} logic seems
wrong.
{code}
private int allocateBytes(int totalSize)
{
int prev = position.getAndAdd(totalSize);
if (prev == CLOSED) // the slab has been 'closed'
return CLOSED;
if ((prev + totalSize) > slab.capacity())
{
position.set(CLOSED); // mark the slab as no longer allocating if
we've exceeded its capacity
return CLOSED;
}
return prev;
}
{code}
As the following scenario will not produce the expected out come:
# Thread A calls {{allocateBytes}} and exit with the return value {{CLOSED}}
and {{position == CLOSED}}
# Thead B calls {{allocateBytes}} with {{totalSize == size_B}} and exit with
the return value {{CLOSED}} and and {{position == size_B -1}} as {{CLOSED ==
-1}} in practice.
# Thread C calls {{allocateBytes}} with {{totalSize == size_C}} and exit with
the return value {{size_B - 1}} and and {{position == size_B + size_C -1}}
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185047#comment-17185047
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/26/20, 9:08 AM:
--
[~Bereng], ways {{(prev + size)}} could wrap around…
- prev itself has wrapped around, the method still then returns negative,
- {{`prev > endOfBuffer`}} and {{`prev + size > Integer.MAX_VALUE`}}.
In the later, {{`endOfBuffer + size> Integer.MAX_VALUE`}} must also be true, so
it would be when {{commitlog_segment_size_in_mb}} is set to some value close
(relative to the size of mutations) to 2GB.
To prevent this would it be enough to
{code}if (((long)prev + size) >= endOfBuffer){code}
was (Author: michaelsembwever):
[~Bereng], ways {{(prev + size)}} could wrap around…
- prev itself has wrapped around, the method still then returns negative,
- {{prev > endOfBuffer}} and {{prev + size > Integer.MAX_VALUE}}.
In the later, {{endOfBuffer + size> Integer.MAX_VALUE}} must also be true, so
it would be when commitlog_segment_size_in_mb is set to some value close
(relative to the size of mutations) to 2GB.
To prevent this would it be enough to
{code}if (((long)prev + size) >= endOfBuffer){code}
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183127#comment-17183127
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/24/20, 12:38 PM:
---
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/259/pipeline]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
with CI
[run|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/260/pipeline]
was (Author: michaelsembwever):
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
(will update with CI later today)
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183127#comment-17183127
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/24/20, 10:02 AM:
---
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements]
(will update with CI later today)
was (Author: michaelsembwever):
Patches
-
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_cas_improvements]
-
[trunk|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements
(will update with CI later today)
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
