[
https://issues.apache.org/jira/browse/CASSANDRA-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558427#comment-16558427
]
ASF GitHub Bot commented on CASSANDRA-14556:
--------------------------------------------
Github user iamaleksey commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/239#discussion_r205416649
--- Diff:
src/java/org/apache/cassandra/io/sstable/format/big/BigTableBlockWriter.java ---
@@ -48,51 +47,61 @@
import org.apache.cassandra.schema.TableId;
import org.apache.cassandra.schema.TableMetadataRef;
+import static java.lang.String.format;
+import static org.apache.cassandra.utils.FBUtilities.prettyPrintMemory;
+
public class BigTableBlockWriter extends SSTable implements
SSTableMultiWriter
{
+ private static final Logger logger =
LoggerFactory.getLogger(BigTableBlockWriter.class);
+
private final TableMetadataRef metadata;
- private final LifecycleTransaction txn;
private volatile SSTableReader finalReader;
private final Map<Component.Type, SequentialWriter> componentWriters;
- private final Logger logger =
LoggerFactory.getLogger(BigTableBlockWriter.class);
-
- private final SequentialWriterOption writerOption =
SequentialWriterOption.newBuilder()
-
.trickleFsync(false)
-
.bufferSize(2 * 1024 * 1024)
-
.bufferType(BufferType.OFF_HEAP)
-
.build();
- public static final ImmutableSet<Component> supportedComponents =
ImmutableSet.of(Component.DATA, Component.PRIMARY_INDEX, Component.STATS,
-
Component.COMPRESSION_INFO, Component.FILTER, Component.SUMMARY,
-
Component.DIGEST, Component.CRC);
+ private static final SequentialWriterOption WRITER_OPTION =
+ SequentialWriterOption.newBuilder()
+ .trickleFsync(false)
+ .bufferSize(2 << 20)
+ .bufferType(BufferType.OFF_HEAP)
+ .build();
+
+ private static final ImmutableSet<Component> SUPPORTED_COMPONENTS =
+ ImmutableSet.of(Component.DATA,
+ Component.PRIMARY_INDEX,
+ Component.SUMMARY,
+ Component.STATS,
+ Component.COMPRESSION_INFO,
+ Component.FILTER,
+ Component.DIGEST,
+ Component.CRC);
public BigTableBlockWriter(Descriptor descriptor,
TableMetadataRef metadata,
LifecycleTransaction txn,
final Set<Component> components)
{
- super(descriptor, ImmutableSet.copyOf(components), metadata,
- DatabaseDescriptor.getDiskOptimizationStrategy());
+ super(descriptor, ImmutableSet.copyOf(components), metadata,
DatabaseDescriptor.getDiskOptimizationStrategy());
+
txn.trackNew(this);
this.metadata = metadata;
- this.txn = txn;
- this.componentWriters = new HashMap<>(components.size());
+ this.componentWriters = new EnumMap<>(Component.Type.class);
- assert supportedComponents.containsAll(components) :
String.format("Unsupported streaming component detected %s",
-
new HashSet(components).removeAll(supportedComponents));
+ if (!SUPPORTED_COMPONENTS.containsAll(components))
+ throw new AssertionError(format("Unsupported streaming
component detected %s",
+ Sets.difference(components,
SUPPORTED_COMPONENTS)));
--- End diff --
Neat. I either forgot, or didn't know that `Sets.difference()` was a thing.
This is nicer than the way I proposed (:
> Optimize streaming path in Cassandra
> ------------------------------------
>
> Key: CASSANDRA-14556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14556
> Project: Cassandra
> Issue Type: Improvement
> Components: Streaming and Messaging
> Reporter: Dinesh Joshi
> Assignee: Dinesh Joshi
> Priority: Major
> Labels: Performance
> Fix For: 4.x
>
>
> During streaming, Cassandra reifies the sstables into objects. This creates
> unnecessary garbage and slows down the whole streaming process as some
> sstables can be transferred as a whole file rather than individual
> partitions. The objective of the ticket is to detect when a whole sstable can
> be transferred and skip the object reification. We can also use a zero-copy
> path to avoid bringing data into user-space on both sending and receiving
> side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
