[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14740: - Status: Open (was: Patch Available) > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Assignee: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15223) OutboundTcpConnection leaks direct memory
[
https://issues.apache.org/jira/browse/CASSANDRA-15223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-15223:
-
Reviewers: Benedict
> OutboundTcpConnection leaks direct memory
> -
>
> Key: CASSANDRA-15223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15223
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Benedict
>Assignee: Venkata Harikrishna Nukala
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> On disconnect we set {{out}} to null without first closing it
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15225) FileUtils.close() does not handle non-IOException
[
https://issues.apache.org/jira/browse/CASSANDRA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-15225:
-
Reviewers: Benedict
> FileUtils.close() does not handle non-IOException
> -
>
> Key: CASSANDRA-15225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15225
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Benedict
>Assignee: Liudmila Kornilova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This can lead to {{close}} not being invoked on remaining items
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15232: - Reviewers: Benedict > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15227) Remove StageManager
[ https://issues.apache.org/jira/browse/CASSANDRA-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15227: - Reviewers: Benedict > Remove StageManager > --- > > Key: CASSANDRA-15227 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15227 > Project: Cassandra > Issue Type: Task > Components: Local/Other >Reporter: Benedict >Assignee: Venkata Harikrishna Nukala >Priority: Normal > Fix For: 4.0 > > > his is a minor cleanup; this class should not exist, but should be embedded > in the Stage enum. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891475#comment-16891475 ] Benedict commented on CASSANDRA-15232: -- Thanks [~Override] for the patch. I'll take a closer look as soon as I have the chance, hopefully within the next week or so. > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891474#comment-16891474 ] Benedict edited comment on CASSANDRA-14740 at 7/23/19 11:44 PM: Patch available [here|https://github.com/belliottsmith/cassandra/tree/14740]; [tests|https://circleci.com/workflow-run/9b4bc0a4-ba24-4a29-a6fd-4bf78f125432] was (Author: benedict): Patch available [here|https://github.com/belliottsmith/cassandra/tree/14740], [tests|https://circleci.com/workflow-run/9b4bc0a4-ba24-4a29-a6fd-4bf78f125432] > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Assignee: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14740: - Test and Documentation Plan: dtest included Status: Patch Available (was: In Progress) Patch available [here|https://github.com/belliottsmith/cassandra/tree/14740], [tests|https://circleci.com/workflow-run/9b4bc0a4-ba24-4a29-a6fd-4bf78f125432] > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Assignee: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15099) Debian 10 buster: init file require PID file to be owned by root
[
https://issues.apache.org/jira/browse/CASSANDRA-15099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891248#comment-16891248
]
Holger Schramm commented on CASSANDRA-15099:
This won't work because the error message after the given change would be
{quote}start-stop-daemon: matching on world-writable pidfile
/var/run/cassandra/cassandra.pid is insecure{quote}
So here is a working and tested fix:
{code}
@@ -210,10 +210,12 @@
if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT"
$cassandra_parms -cp "$CLASSPATH" $props "$class" <&- &
[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+chmod 664 "$pidpath"
true
else
exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH"
$props "$class" <&- &
[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+chmod 664 "$pidpath"
true
fi
fi
{code}
> Debian 10 buster: init file require PID file to be owned by root
>
>
> Key: CASSANDRA-15099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15099
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yury Vidineev
>Priority: Normal
>
> Restarting Cassandra: cassandrastart-stop-daemon: matching only on non-root
> pidfile /var/run/cassandra/cassandra.pid is insecure
> failed!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890996#comment-16890996 ] Jon Meredith commented on CASSANDRA-15170: -- Here's the latest - had all the updated branches passing on macOS and a container based on the CircleCI image. I need to make one more pass over it and would like to move the gossip and streaming shutdown from trunk to 2.2 and merge forward. 2.2 | [Branch|https://github.com/jonmeredith/cassandra/commits/in-jvm-dtest-fixes-v3-2.2] | [CircleCI|https://circleci.com/gh/jonmeredith/cassandra/tree/in-jvm-dtest-fixes-v3-2%2E2] 3.0 | [Branch|https://github.com/jonmeredith/cassandra/commits/in-jvm-dtest-fixes-v3-3.0] | [CircleCI|https://circleci.com/gh/jonmeredith/cassandra/tree/in-jvm-dtest-fixes-v3-3%2E0] 3.11 | [Branch|https://github.com/jonmeredith/cassandra/commits/in-jvm-dtest-fixes-v3-3.11] | [CircleCI|https://circleci.com/gh/jonmeredith/cassandra/tree/in-jvm-dtest-fixes-v3-3%2E11] trunk | [Branch|https://github.com/jonmeredith/cassandra/commits/in-jvm-dtest-fixes-v3-trunk] | [CircleCI|https://circleci.com/gh/jonmeredith/cassandra/tree/in-jvm-dtest-fixes-v3-trunk] > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890992#comment-16890992
]
Paulo Motta commented on CASSANDRA-14227:
-
{quote}This results 584043668 and not 63072 ! and then updating the row
with fetched ttl loses the original ttl value (20 years) so resuming
localDeletionTime based on ttl the value will not be feasible if someone
chooses CAP or CAP_NOWARN option as in sstable, TTL value is no more 20 years
(63072).
{quote}
The fact that {{SELECT TTL(*)}} does not return the original TTL does not mean
we cannot resume the original TTL once this is fixed. Data will need to be
rewritten via "upgradesstables" to restore the original TTL once this
limitation is fixed. If you are updating the row with fetched TTL you are
overwriting the original value, so it's expected behavior that you lose the
original TTL.
If you are willing to take a stab at fixing this, I think the simplest approach
is to change the {{localDeletionTime}} field from integer to long.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13990) Remove OldNetworkTopologyStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-13990: Test and Documentation Plan: Ran `ant test`. 3 test suites failed (IndexSummaryTest, ConnectionTest, and RepairJobTest), but they were failing on trunk as well. Status: Patch Available (was: In Progress) Patch: https://github.com/sedulam/CASSANDRA-13990/commit/bb043e3987812fb0ea5a471120c52e425e637b1f > Remove OldNetworkTopologyStrategy > - > > Key: CASSANDRA-13990 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13990 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Jeremy Hanna >Assignee: Pedro Gordo >Priority: Low > Labels: lhf > Attachments: 13990-trunk.txt > > > RackAwareStrategy was renamed OldNetworkTopologyStrategy back in 0.7 > (CASSANDRA-1392) and it's still around. Is there any reason to keep this > relatively dead code in the codebase at this point? I'm not aware of its use > and it sometimes confuses users. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13990) Remove OldNetworkTopologyStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-13990: Attachment: 13990-trunk.txt > Remove OldNetworkTopologyStrategy > - > > Key: CASSANDRA-13990 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13990 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Jeremy Hanna >Assignee: Pedro Gordo >Priority: Low > Labels: lhf > Attachments: 13990-trunk.txt > > > RackAwareStrategy was renamed OldNetworkTopologyStrategy back in 0.7 > (CASSANDRA-1392) and it's still around. Is there any reason to keep this > relatively dead code in the codebase at this point? I'm not aware of its use > and it sometimes confuses users. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890801#comment-16890801 ] Liudmila Kornilova edited comment on CASSANDRA-15232 at 7/23/19 8:58 AM: - Hello [~benedict], This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 I created this work-in-process pull request https://github.com/apache/cassandra/pull/334 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented was (Author: override): Hello [~benedict], This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 I created this WIP commit https://github.com/apache/cassandra/pull/334 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890801#comment-16890801 ] Liudmila Kornilova commented on CASSANDRA-15232: This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890801#comment-16890801 ] Liudmila Kornilova edited comment on CASSANDRA-15232 at 7/23/19 8:57 AM: - Hello [~benedict], This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 I created this WIP commit https://github.com/apache/cassandra/pull/334 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented was (Author: override): Hello [~benedict], This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890801#comment-16890801 ] Liudmila Kornilova edited comment on CASSANDRA-15232 at 7/23/19 8:57 AM: - Hello [~benedict], This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented was (Author: override): This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 1 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333. (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.1 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15232: --- Labels: pull-request-available (was: ) > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Assignee: Liudmila Kornilova >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15242) Race condition between flushing and compaction stalls compaction indefinitely
[
https://issues.apache.org/jira/browse/CASSANDRA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson reassigned CASSANDRA-15242:
---
Assignee: Marcus Eriksson
> Race condition between flushing and compaction stalls compaction indefinitely
> -
>
> Key: CASSANDRA-15242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15242
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction/LCS
>Reporter: Tom van der Woerdt
>Assignee: Marcus Eriksson
>Priority: Normal
> Attachments: snippet.txt
>
>
> Seen on Cassandra 3.11.4 with OpenJDK 8u212, although I've seen this a few
> times before, also on 3.11.3. It's a rare issue so I've not bothered with
> trying to trace it until now.
> {noformat}
> DEBUG [NativePoolCleaner] 2019-07-18 01:12:41,799 ColumnFamilyStore.java:1325
> - Flushing largest CFS(Keyspace='keyspacename', ColumnFamily='tablename') to
> free up room. Used total: 0.10/0.33, live: 0.10/0.33, flushing: 0.00/0.00,
> this: 0.09/0.19
> DEBUG [NativePoolCleaner] 2019-07-18 01:12:41,800 ColumnFamilyStore.java:935
> - Enqueuing flush of tablename: 267.930MiB (9%) on-heap, 575.580MiB (19%)
> off-heap
> DEBUG [PerDiskMemtableFlushWriter_0:204] 2019-07-18 01:12:42,480
> Memtable.java:456 - Writing Memtable-tablename@498336646(520.721MiB
> serialized bytes, 870200 ops, 9%/19% of on/off-heap limit), flushed range =
> (min(-9223372036854775808), max(9223372036854775807)]
> INFO [Service Thread] 2019-07-18 01:12:43,616 GCInspector.java:284 - G1
> Young Generation GC in 227ms. G1 Eden Space: 14713618432 -> 0; G1 Old Gen:
> 13240876928 -> 13259198848; G1 Survivor Space: 276824064 -> 268435456;
> INFO [Service Thread] 2019-07-18 01:12:56,251 GCInspector.java:284 - G1
> Young Generation GC in 206ms. G1 Eden Space: 14713618432 -> 0; G1 Old Gen:
> 13259198848 -> 13285123456; G1 Survivor Space: 268435456 -> 285212672;
> DEBUG [PerDiskMemtableFlushWriter_0:204] 2019-07-18 01:12:56,693
> Memtable.java:485 - Completed flushing
> /cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db
> (524.023MiB) for commitlog position
> CommitLogPosition(segmentId=1563386911266, position=32127822)
> DEBUG [MemtableFlushWriter:204] 2019-07-18 01:12:57,620
> ColumnFamilyStore.java:1233 - Flushed to
> [BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db')]
> (1 sstables, 518.714MiB), biggest 518.714MiB, smallest 518.714MiB
> WARN [CompactionExecutor:1617] 2019-07-18 01:12:57,628
> LeveledCompactionStrategy.java:144 - Could not acquire references for
> compacting SSTables
> [BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292348-big-Data.db'),
>
> BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db')]
> which is not a problem per se,unless it happens frequently, in which case it
> must be reported. Will retry later.
> {noformat}
> This final line then starts repeating about once per minute:
> {noformat}
> WARN [CompactionExecutor:1610] 2019-07-18 01:13:18,898
> LeveledCompactionStrategy.java:144 - Could not acquire references for
> compacting SSTables
> [BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292348-big-Data.db'),
>
> BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db')]
> which is not a problem per se,unless it happens frequently, in which case it
> must be reported. Will retry later.
> WARN [CompactionExecutor:1611] 2019-07-18 01:14:18,899
> LeveledCompactionStrategy.java:144 - Could not acquire references for
> compacting SSTables
> [BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292348-big-Data.db'),
>
> BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db')]
> which is not a problem per se,unless it happens frequently, in which case it
> must be reported. Will retry later.
> WARN [CompactionExecutor:1622] 2019-07-18 01:15:18,899
> LeveledCompactionStrategy.java:144 - Could not acquire references for
> compacting SSTables
> [BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292348-big-Data.db'),
>
> BigTableReader(path='/cassandra-data/data/keyspacename/tablename-2c5325e042e911e8a07e9db72d27cf67/md-292358-big-Data.db')]
> which is not a problem per se,unless it happens frequently, in which case it
> must be reported. Will retry later.
> WARN [CompactionExecutor:1436]
