frankgh commented on code in PR #1754:
URL: https://github.com/apache/cassandra/pull/1754#discussion_r931664189
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -252,6 +253,21 @@ public LongBytesPerSecondBound(long bytesPerSecond)
{
this(bytesPerSecond, BYTES_PER_SECOND);
}
+
+ // this one should be used only for backward compatibility for
stream_throughput_outbound and inter_dc_stream_throughput_outbound
+ // which were in megabits per second in 4.0. Do not start using it for
any new properties
+ public static LongBytesPerSecondBound
megabitsPerSecondInBytesPerSecond(long megabitsPerSecond)
+ {
+ final double BYTES_PER_MEGABIT = 125_000;
+ double bytesPerSecond = (double) megabitsPerSecond *
BYTES_PER_MEGABIT;
Review Comment:
Why use double here? long should be fine
```suggestion
final long BYTES_PER_MEGABIT = 125_000L;
long bytesPerSecond = megabitsPerSecond * BYTES_PER_MEGABIT;
```
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -252,6 +253,21 @@ public LongBytesPerSecondBound(long bytesPerSecond)
{
this(bytesPerSecond, BYTES_PER_SECOND);
}
+
+ // this one should be used only for backward compatibility for
stream_throughput_outbound and inter_dc_stream_throughput_outbound
+ // which were in megabits per second in 4.0. Do not start using it for
any new properties
+ public static LongBytesPerSecondBound
megabitsPerSecondInBytesPerSecond(long megabitsPerSecond)
Review Comment:
should we annotate with `@Deprecated` if this is not intended to be used
##########
src/java/org/apache/cassandra/service/StorageServiceMBean.java:
##########
@@ -625,26 +625,71 @@ default int upgradeSSTables(String keyspaceName, boolean
excludeCurrentVersion,
public long getTruncateRpcTimeout();
public void setStreamThroughputMbitPerSec(int value);
+ /**
+ * Please use the new getStreamThroughputMbitPerSecAsDouble method as this
one will provide a rounded value
+ * @return stream_throughput_outbound in megabits
Review Comment:
maybe used the deprecated annotation for javadocs here?
```suggestion
* @return stream_throughput_outbound in megabits
* @deprecated Use getStreamThroughputMbitPerSecAsDouble instead
```
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -212,7 +213,7 @@ public boolean equals(Object obj)
@Override
public String toString()
{
- return Math.round(quantity) + unit.symbol;
+ return DoubleMath.isMathematicalInteger(quantity) ? ((long)quantity +
unit.symbol) : (quantity + unit.symbol);
Review Comment:
this changes behavior. `Math.round` of `999.999` will return -> `1000`.
Whereas, `(long)999.999` returns `999`. Is this intended?
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -252,6 +253,21 @@ public LongBytesPerSecondBound(long bytesPerSecond)
{
this(bytesPerSecond, BYTES_PER_SECOND);
}
+
+ // this one should be used only for backward compatibility for
stream_throughput_outbound and inter_dc_stream_throughput_outbound
+ // which were in megabits per second in 4.0. Do not start using it for
any new properties
+ public static LongBytesPerSecondBound
megabitsPerSecondInBytesPerSecond(long megabitsPerSecond)
+ {
+ final double BYTES_PER_MEGABIT = 125_000;
+ double bytesPerSecond = (double) megabitsPerSecond *
BYTES_PER_MEGABIT;
+
+ if (megabitsPerSecond >= Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid data rate: " +
megabitsPerSecond + " megabits per second; " +
+ "stream_throughput_outbound
and inter_dc_stream_throughput_outbound" +
+ " should be between 0 and "
+ Integer.MAX_VALUE + " in megabits per second");
+
+ return new LongBytesPerSecondBound(bytesPerSecond,
BYTES_PER_SECOND);
Review Comment:
`LongBytesPerSecondBound`'s constructor takes a double, but it should really
take a long. We cast from double to long internally. I tried making
`DataRateSpec`'s `quantity` a `long`, and tests are passing.
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -252,6 +253,21 @@ public LongBytesPerSecondBound(long bytesPerSecond)
{
this(bytesPerSecond, BYTES_PER_SECOND);
}
+
+ // this one should be used only for backward compatibility for
stream_throughput_outbound and inter_dc_stream_throughput_outbound
+ // which were in megabits per second in 4.0. Do not start using it for
any new properties
+ public static LongBytesPerSecondBound
megabitsPerSecondInBytesPerSecond(long megabitsPerSecond)
+ {
+ final double BYTES_PER_MEGABIT = 125_000;
Review Comment:
should we declare it as a field constant instead?
##########
src/java/org/apache/cassandra/config/Config.java:
##########
@@ -315,7 +315,7 @@ public MemtableOptions()
public Integer unlogged_batch_across_partitions_warn_threshold = 10;
public volatile Integer concurrent_compactors;
@Replaces(oldName = "compaction_throughput_mb_per_sec", converter =
Converters.MEBIBYTES_PER_SECOND_DATA_RATE, deprecated = true)
- public volatile DataRateSpec.IntMebibytesPerSecondBound
compaction_throughput = new DataRateSpec.IntMebibytesPerSecondBound("16MiB/s");
Review Comment:
do we want to keep the `DataRateSpec.IntMebibytesPerSecondBound` class? I
only see it being used in tests. Maybe remove early before someone starts
relying on it.
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -252,6 +253,21 @@ public LongBytesPerSecondBound(long bytesPerSecond)
{
this(bytesPerSecond, BYTES_PER_SECOND);
}
+
+ // this one should be used only for backward compatibility for
stream_throughput_outbound and inter_dc_stream_throughput_outbound
+ // which were in megabits per second in 4.0. Do not start using it for
any new properties
+ public static LongBytesPerSecondBound
megabitsPerSecondInBytesPerSecond(long megabitsPerSecond)
+ {
+ final double BYTES_PER_MEGABIT = 125_000;
+ double bytesPerSecond = (double) megabitsPerSecond *
BYTES_PER_MEGABIT;
+
+ if (megabitsPerSecond >= Integer.MAX_VALUE)
Review Comment:
Did you mean `>` here?
```suggestion
if (megabitsPerSecond > Integer.MAX_VALUE)
```
##########
src/java/org/apache/cassandra/config/DataRateSpec.java:
##########
@@ -385,7 +386,7 @@ public double toMegabitsPerSecond(double d)
{
if (d > MAX / (MEGABITS_PER_MEBIBYTE))
return MAX;
- return Math.round(d * MEGABITS_PER_MEBIBYTE);
+ return d * MEGABITS_PER_MEBIBYTE;
Review Comment:
this also changes behavior but it's probably desired
##########
src/java/org/apache/cassandra/config/DatabaseDescriptor.java:
##########
@@ -2016,59 +2055,115 @@ public static int
getStreamThroughputOutboundMegabitsPerSec()
return conf.stream_throughput_outbound.toMegabitsPerSecondAsInt();
}
+ public static double getStreamThroughputOutboundMegabitsPerSecAsDouble()
+ {
+ return conf.stream_throughput_outbound.toMegabitsPerSecond();
+ }
+
public static double getStreamThroughputOutboundMebibytesPerSec()
{
return conf.stream_throughput_outbound.toMebibytesPerSecond();
}
- public static void setStreamThroughputOutboundMegabitsPerSec(int value)
+ public static double getStreamThroughputOutboundBytesPerSec()
{
- conf.stream_throughput_outbound =
DataRateSpec.IntMebibytesPerSecondBound.megabitsPerSecondInMebibytesPerSecond(value);
+ return conf.stream_throughput_outbound.toBytesPerSecond();
}
- public static int
getEntireSSTableStreamThroughputOutboundMebibytesPerSecAsInt()
+ public static int getStreamThroughputOutboundMebibytesPerSecAsInt()
{
- return
conf.entire_sstable_stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ return conf.stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ }
+
+ public static void setStreamThroughputOutboundMebibytesPerSecAsInt(int
value)
+ {
+ if (MEBIBYTES_PER_SECOND.toMegabitsPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
+ conf.stream_throughput_outbound = new
DataRateSpec.LongBytesPerSecondBound(valueInBytes);
+ }
+
+ public static void setStreamThroughputOutboundMegabitsPerSec(int value)
+ {
+ conf.stream_throughput_outbound =
DataRateSpec.LongBytesPerSecondBound.megabitsPerSecondInBytesPerSecond(value);
}
public static double
getEntireSSTableStreamThroughputOutboundMebibytesPerSec()
{
return
conf.entire_sstable_stream_throughput_outbound.toMebibytesPerSecond();
}
+ public static double getEntireSSTableStreamThroughputOutboundBytesPerSec()
+ {
+ return
conf.entire_sstable_stream_throughput_outbound.toBytesPerSecond();
+ }
+
public static void
setEntireSSTableStreamThroughputOutboundMebibytesPerSec(int value)
{
- conf.entire_sstable_stream_throughput_outbound = new
DataRateSpec.IntMebibytesPerSecondBound(value);
+ if (MEBIBYTES_PER_SECOND.toMebibytesPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
entire_sstable_stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
+ conf.entire_sstable_stream_throughput_outbound = new
DataRateSpec.LongBytesPerSecondBound(valueInBytes);
}
public static int getInterDCStreamThroughputOutboundMegabitsPerSec()
{
return
conf.inter_dc_stream_throughput_outbound.toMegabitsPerSecondAsInt();
}
+ public static double
getInterDCStreamThroughputOutboundMegabitsPerSecAsDouble()
+ {
+ return conf.inter_dc_stream_throughput_outbound.toMegabitsPerSecond();
+ }
+
public static double getInterDCStreamThroughputOutboundMebibytesPerSec()
{
return conf.inter_dc_stream_throughput_outbound.toMebibytesPerSecond();
}
+ public static double getInterDCStreamThroughputOutboundBytesPerSec()
+ {
+ return conf.inter_dc_stream_throughput_outbound.toBytesPerSecond();
+ }
+
+ public static int getInterDCStreamThroughputOutboundMebibytesPerSecAsInt()
+ {
+ return
conf.inter_dc_stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ }
+
+ public static void
setInterDCStreamThroughputOutboundMebibytesPerSecAsInt(int value)
+ {
+ if (MEBIBYTES_PER_SECOND.toMegabitsPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
inter_dc_stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
+ conf.inter_dc_stream_throughput_outbound = new
DataRateSpec.LongBytesPerSecondBound(valueInBytes);
+ }
+
public static void setInterDCStreamThroughputOutboundMegabitsPerSec(int
value)
{
- conf.inter_dc_stream_throughput_outbound =
DataRateSpec.IntMebibytesPerSecondBound.megabitsPerSecondInMebibytesPerSecond(value);
+ conf.inter_dc_stream_throughput_outbound =
DataRateSpec.LongBytesPerSecondBound.megabitsPerSecondInBytesPerSecond(value);
}
- public static double
getEntireSSTableInterDCStreamThroughputOutboundMebibytesPerSec()
+ public static double
getEntireSSTableInterDCStreamThroughputOutboundBytesPerSec()
{
- return
conf.entire_sstable_inter_dc_stream_throughput_outbound.toMebibytesPerSecond();
+ return
conf.entire_sstable_inter_dc_stream_throughput_outbound.toBytesPerSecond();
}
- public static int
getEntireSSTableInterDCStreamThroughputOutboundMebibytesPerSecAsInt()
+ public static double
getEntireSSTableInterDCStreamThroughputOutboundMebibytesPerSec()
{
- return
conf.entire_sstable_inter_dc_stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ return
conf.entire_sstable_inter_dc_stream_throughput_outbound.toMebibytesPerSecond();
}
public static void
setEntireSSTableInterDCStreamThroughputOutboundMebibytesPerSec(int value)
{
- conf.entire_sstable_inter_dc_stream_throughput_outbound = new
DataRateSpec.IntMebibytesPerSecondBound(value);
+ if (MEBIBYTES_PER_SECOND.toMebibytesPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
entire_sstable_inter_dc_stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
Review Comment:
It feels like we do this a sufficient number of times that it might be worth
defining a constant
` static final int ONE_MEBIBTE_IN_BYTES = 1024L * 1024L;`
##########
src/java/org/apache/cassandra/config/DatabaseDescriptor.java:
##########
@@ -906,6 +898,36 @@ else if (conf.max_value_size.toMebibytes() >= 2048)
logInitializationOutcome(logger);
}
+ @VisibleForTesting
+ static void validateUpperBoundStreamingConfig() throws
ConfigurationException
+ {
+ // below 2 checks are needed in order to match the pre-CASSANDRA-15234
upper bound for those parameters which were still in megabits per second
+ if (conf.stream_throughput_outbound.toMegabitsPerSecond() >=
Integer.MAX_VALUE)
Review Comment:
I'm confused why we exclude `Integer.MAX_VALUE`. Should this be
`(conf.stream_throughput_outbound.toMegabitsPerSecond() > Integer.MAX_VALUE)`
instead?
##########
src/java/org/apache/cassandra/config/DatabaseDescriptor.java:
##########
@@ -3186,7 +3281,10 @@ public static int getCacheLoadTimeout()
@VisibleForTesting
public static void setCacheLoadTimeout(int seconds)
{
- conf.cache_load_timeout = new DurationSpec.IntSecondsBound(seconds);
+ if (seconds < 0)
Review Comment:
minor nit:
```suggestion
if (seconds <= 0)
```
##########
src/java/org/apache/cassandra/config/DatabaseDescriptor.java:
##########
@@ -2016,59 +2055,115 @@ public static int
getStreamThroughputOutboundMegabitsPerSec()
return conf.stream_throughput_outbound.toMegabitsPerSecondAsInt();
}
+ public static double getStreamThroughputOutboundMegabitsPerSecAsDouble()
+ {
+ return conf.stream_throughput_outbound.toMegabitsPerSecond();
+ }
+
public static double getStreamThroughputOutboundMebibytesPerSec()
{
return conf.stream_throughput_outbound.toMebibytesPerSecond();
}
- public static void setStreamThroughputOutboundMegabitsPerSec(int value)
+ public static double getStreamThroughputOutboundBytesPerSec()
{
- conf.stream_throughput_outbound =
DataRateSpec.IntMebibytesPerSecondBound.megabitsPerSecondInMebibytesPerSecond(value);
+ return conf.stream_throughput_outbound.toBytesPerSecond();
}
- public static int
getEntireSSTableStreamThroughputOutboundMebibytesPerSecAsInt()
+ public static int getStreamThroughputOutboundMebibytesPerSecAsInt()
{
- return
conf.entire_sstable_stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ return conf.stream_throughput_outbound.toMebibytesPerSecondAsInt();
+ }
+
+ public static void setStreamThroughputOutboundMebibytesPerSecAsInt(int
value)
+ {
+ if (MEBIBYTES_PER_SECOND.toMegabitsPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
+ conf.stream_throughput_outbound = new
DataRateSpec.LongBytesPerSecondBound(valueInBytes);
+ }
+
+ public static void setStreamThroughputOutboundMegabitsPerSec(int value)
+ {
+ conf.stream_throughput_outbound =
DataRateSpec.LongBytesPerSecondBound.megabitsPerSecondInBytesPerSecond(value);
}
public static double
getEntireSSTableStreamThroughputOutboundMebibytesPerSec()
{
return
conf.entire_sstable_stream_throughput_outbound.toMebibytesPerSecond();
}
+ public static double getEntireSSTableStreamThroughputOutboundBytesPerSec()
+ {
+ return
conf.entire_sstable_stream_throughput_outbound.toBytesPerSecond();
+ }
+
public static void
setEntireSSTableStreamThroughputOutboundMebibytesPerSec(int value)
{
- conf.entire_sstable_stream_throughput_outbound = new
DataRateSpec.IntMebibytesPerSecondBound(value);
+ if (MEBIBYTES_PER_SECOND.toMebibytesPerSecond(value) >=
Integer.MAX_VALUE)
+ throw new IllegalArgumentException("Invalid value of
entire_sstable_stream_throughput_outbound: " + value);
+
+ long valueInBytes = value * 1024L * 1024L;
+ conf.entire_sstable_stream_throughput_outbound = new
DataRateSpec.LongBytesPerSecondBound(valueInBytes);
}
public static int getInterDCStreamThroughputOutboundMegabitsPerSec()
{
return
conf.inter_dc_stream_throughput_outbound.toMegabitsPerSecondAsInt();
}
+ public static double
getInterDCStreamThroughputOutboundMegabitsPerSecAsDouble()
+ {
+ return conf.inter_dc_stream_throughput_outbound.toMegabitsPerSecond();
+ }
+
public static double getInterDCStreamThroughputOutboundMebibytesPerSec()
{
return conf.inter_dc_stream_throughput_outbound.toMebibytesPerSecond();
}
+ public static double getInterDCStreamThroughputOutboundBytesPerSec()
+ {
+ return conf.inter_dc_stream_throughput_outbound.toBytesPerSecond();
+ }
+
+ public static int getInterDCStreamThroughputOutboundMebibytesPerSecAsInt()
Review Comment:
is this only used for JMX calls?
##########
src/java/org/apache/cassandra/tools/nodetool/GetCompactionThroughput.java:
##########
@@ -17,17 +17,34 @@
*/
package org.apache.cassandra.tools.nodetool;
+import com.google.common.math.DoubleMath;
+
import io.airlift.airline.Command;
+import io.airlift.airline.Option;
import org.apache.cassandra.tools.NodeProbe;
import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
-@Command(name = "getcompactionthroughput", description = "Print the MiB/s
throughput cap for compaction in the system")
+@Command(name = "getcompactionthroughput", description = "Print the MiB/s
throughput cap for compaction in the system as a rounded number")
public class GetCompactionThroughput extends NodeToolCmd
{
+ @SuppressWarnings("UnusedDeclaration")
+ @Option(name = { "-d", "--precise-mib" }, description = "Print the MiB/s
throughput cap for compaction in the system as a precise number (double)")
+ private boolean compactionThroughputAsDouble;
+
@Override
public void execute(NodeProbe probe)
{
- probe.output().out.println("Current compaction throughput: " +
probe.getCompactionThroughput() + " MiB/s");
+ double throughput = probe.getCompactionThroughputMebibytesAsDouble();
+
+ if (compactionThroughputAsDouble)
+ probe.output().out.println("Current compaction throughput: " +
throughput + " MiB/s");
+ else
+ {
+ if (!DoubleMath.isMathematicalInteger(throughput))
+ throw new RuntimeException("You should use -d to get exact
throughput in MiB/s");
Review Comment:
```suggestion
throw new RuntimeException("Use the -d flag to quiet this
error and get the exact throughput in MiB/s");
```
##########
test/unit/org/apache/cassandra/tools/nodetool/SetGetEntireSSTableStreamThroughputTest.java:
##########
@@ -34,6 +34,8 @@
*/
public class SetGetEntireSSTableStreamThroughputTest extends CQLTester
{
+ private static final int MAX_INT_CONFIG_VALUE_MIB = Integer.MAX_VALUE - 1;
Review Comment:
ah ok, max value is `Integer.MAX_VALUE - 1`
##########
src/java/org/apache/cassandra/tools/nodetool/GetCompactionThroughput.java:
##########
@@ -17,17 +17,34 @@
*/
package org.apache.cassandra.tools.nodetool;
+import com.google.common.math.DoubleMath;
+
import io.airlift.airline.Command;
+import io.airlift.airline.Option;
import org.apache.cassandra.tools.NodeProbe;
import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
-@Command(name = "getcompactionthroughput", description = "Print the MiB/s
throughput cap for compaction in the system")
+@Command(name = "getcompactionthroughput", description = "Print the MiB/s
throughput cap for compaction in the system as a rounded number")
public class GetCompactionThroughput extends NodeToolCmd
{
+ @SuppressWarnings("UnusedDeclaration")
+ @Option(name = { "-d", "--precise-mib" }, description = "Print the MiB/s
throughput cap for compaction in the system as a precise number (double)")
+ private boolean compactionThroughputAsDouble;
+
@Override
public void execute(NodeProbe probe)
{
- probe.output().out.println("Current compaction throughput: " +
probe.getCompactionThroughput() + " MiB/s");
+ double throughput = probe.getCompactionThroughputMebibytesAsDouble();
+
+ if (compactionThroughputAsDouble)
+ probe.output().out.println("Current compaction throughput: " +
throughput + " MiB/s");
+ else
+ {
+ if (!DoubleMath.isMathematicalInteger(throughput))
+ throw new RuntimeException("You should use -d to get exact
throughput in MiB/s");
+
+ probe.output().out.println("Current compaction throughput: " +
probe.getCompactionThroughput() + " MB/s");
Review Comment:
good point, when will we switch to MiB/s?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
