[jira] [Commented] (CASSANDRA-13760) presize collections
[ https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191546#comment-16191546 ] Jeff Jirsa commented on CASSANDRA-13760: Kicked off another run because I hate seeing all the failures, but none of the dtest failures in 329 or 357 ( https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/357/testReport/ ) seem to be unique to this patch, so I'm asserting they're all existing or environmental problems. Looks good to me to commit. +1 for either your version or my version > presize collections > --- > > Key: CASSANDRA-13760 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13760 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.x > > Attachments: 0001-Presize-collections.patch, 13760.txt, Screen Shot > 2017-09-18 at 9.50.17 PM.png, test-output.png > > > presize collections where sane, to avoid reallocs, or excess garbage -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13760) presize collections
[ https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171126#comment-16171126 ] Jeff Jirsa commented on CASSANDRA-13760: Ran this again and still getting a lot of failures. https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/303/testReport/ Attaching screenshot. I havent had a chance to check them all yet, but I had to rebase and manually merge in some repair patches anyway, so I'm starting a clean run again: https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/329/ The fact that these have been pretty messy so far suggests to me there may be a real problem, though I don't have time to identify it. We'll see if 329 comes out clean, I guess. > presize collections > --- > > Key: CASSANDRA-13760 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13760 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.x > > Attachments: 0001-Presize-collections.patch, 13760.txt, > test-output.png > > > presize collections where sane, to avoid reallocs, or excess garbage -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13760) presize collections
[
https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154822#comment-16154822
]
Jeff Jirsa commented on CASSANDRA-13760:
Just realized that those are both fixed by
{{e1aa7d32c95ff3f06de97803a186ff432237ecab}}, which isn't in your branch, so
I've rebased and force-pushed. The new dtest will be @
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/283/
> presize collections
> ---
>
> Key: CASSANDRA-13760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13760
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Dave Brosius
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 4.x
>
> Attachments: 0001-Presize-collections.patch, 13760.txt,
> test-output.png
>
>
> presize collections where sane, to avoid reallocs, or excess garbage
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13760) presize collections
[
https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154818#comment-16154818
]
Jeff Jirsa commented on CASSANDRA-13760:
Attached screenshot of dtest results - 2 failures aren't typical failures:
!test-output.png|thumbnail!
Checked console output,
{{repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test}}
looks like a netty error:
{code}
Unexpected error in node2 log, error:
WARN [epollEventLoopGroup-2-8] 2017-08-31 09:12:42,153 Slf4JLogger.java:151 -
An exceptionCaught() event was fired, and it reached at the tail of the
pipeline. It usually means the last handler in the pipeline did not handle the
exception.
io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
Connection reset by peer
at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
{code}
Meanwhile {{native_transport_ssl_test.NativeTransportSSL.connect_to_ssl_test}}
throws an actual ssl error:
{code}
WARN [epollEventLoopGroup-2-7] 2017-08-31 07:46:04,104 Slf4JLogger.java:151 -
An exceptionCaught() event was fired, and it reached at the tail of the
pipeline. It usually means the last handler in the pipeline did not handle the
exception.
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
04100100160001000b43514c5f56455253494f4e0005332e302e30
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1057)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:934)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302)
~[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
[netty-all-4.0.44.Final.jar:4.0.44.Final]
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
[netty-all-4.0.44.Final.jar:4.0.44.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
{code}
> presize collections
> ---
>
> Key: CASSANDRA-13760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13760
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Dave Brosius
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 4.x
>
> Attachments: 0001-Presize-collections.patch, 13760.txt,
> test-output.png
>
>
> presize collections where sane, to avoid reallocs, or excess garbage
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13760) presize collections
[
https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148400#comment-16148400
]
Jeff Jirsa commented on CASSANDRA-13760:
The unit test run was clean. The dtest run timed out before I saw the result,
so I've restarted it
[here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/254/]
I'm +1 on the patch as long as the dtest run comes back no worse than the other
trunk dtests.
I've made two small changes to the version I've uploaded as
{{0001-Presize-collections.patch}} :
On
{{src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java}}
, I added the expected size to the currGroup list made within the loop as well:
{code}
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -539,7 +539,7 @@ public abstract class AbstractCompactionStrategy
Collections.sort(sortedSSTablesToGroup,
SSTableReader.sstableComparator);
Collection groupedSSTables = new
ArrayList<>();
-Collection currGroup = new ArrayList<>();
+Collection currGroup = new ArrayList<>(groupSize);
for (SSTableReader sstable : sortedSSTablesToGroup)
{
@@ -547,7 +547,7 @@ public abstract class AbstractCompactionStrategy
if (currGroup.size() == groupSize)
{
groupedSSTables.add(currGroup);
-currGroup = new ArrayList<>();
+currGroup = new ArrayList<>(groupSize);
}
}
{code}
Similarly in
{{src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java}}
, added the initial size to the nested group as well:
{code}
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -223,14 +223,14 @@ public class LeveledCompactionStrategy extends
AbstractCompactionStrategy
for (Collection levelOfSSTables :
sstablesByLevel.values())
{
-Collection currGroup = new ArrayList<>();
+Collection currGroup = new ArrayList<>(groupSize);
for (SSTableReader sstable : levelOfSSTables)
{
currGroup.add(sstable);
if (currGroup.size() == groupSize)
{
groupedSSTables.add(currGroup);
-currGroup = new ArrayList<>();
+currGroup = new ArrayList<>(groupSize);
}
}
{code}
Other than that, +1, looks good.
> presize collections
> ---
>
> Key: CASSANDRA-13760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13760
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Dave Brosius
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 4.x
>
> Attachments: 0001-Presize-collections.patch, 13760.txt
>
>
> presize collections where sane, to avoid reallocs, or excess garbage
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13760) presize collections
[ https://issues.apache.org/jira/browse/CASSANDRA-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124793#comment-16124793 ] Jeff Jirsa commented on CASSANDRA-13760: Pushing to CI while I review: [utest|https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-13760] [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/179/] > presize collections > --- > > Key: CASSANDRA-13760 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13760 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.x > > Attachments: 13760.txt > > > presize collections where sane, to avoid reallocs, or excess garbage -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
