[jira] [Commented] (CASSANDRA-7045) Investigate changes to threading model for network consumers to improve performance for small messages
[ https://issues.apache.org/jira/browse/CASSANDRA-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976379#comment-13976379 ] Jonathan Ellis commented on CASSANDRA-7045: --- I exchanged a few emails with the author of Kilim about 18 months ago. Kind of scary to get that deep in the bytecode with a project that doesn't look to be used seriously in production. Investigate changes to threading model for network consumers to improve performance for small messages -- Key: CASSANDRA-7045 URL: https://issues.apache.org/jira/browse/CASSANDRA-7045 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance See CASSANDRA-7029 for original discussion. We can (maybe) optimise the threading model for network consumers: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7045) Investigate changes to threading model for network consumers to improve performance for small messages
[ https://issues.apache.org/jira/browse/CASSANDRA-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971423#comment-13971423 ] Jason Brown commented on CASSANDRA-7045: I started an investigation into something like the quasar project (http://docs.paralleluniverse.co/quasar/), where you are doing something like like golang's goroutines / erlang's process. Basically, lightweight event processing not occupying a full thread. While I thought bringing in the whole quasar project was more than what we would want, I thought the basic ideas (scheduler + lightweight routines) would be a great thing to look at. Investigate changes to threading model for network consumers to improve performance for small messages -- Key: CASSANDRA-7045 URL: https://issues.apache.org/jira/browse/CASSANDRA-7045 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance See CASSANDRA-7029 for original discussion. We can (maybe) optimise the threading model for network consumers: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7045) Investigate changes to threading model for network consumers to improve performance for small messages
[ https://issues.apache.org/jira/browse/CASSANDRA-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971426#comment-13971426 ] Benedict commented on CASSANDRA-7045: - Yeah, [Kilim|http://www.malhar.net/sriram/kilim/] took my eye recently as well. It's quite a bit more involved as an undertaking, though, as we'd probably need to transform the whole pipeline. Investigate changes to threading model for network consumers to improve performance for small messages -- Key: CASSANDRA-7045 URL: https://issues.apache.org/jira/browse/CASSANDRA-7045 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance See CASSANDRA-7029 for original discussion. We can (maybe) optimise the threading model for network consumers: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7045) Investigate changes to threading model for network consumers to improve performance for small messages
[ https://issues.apache.org/jira/browse/CASSANDRA-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971440#comment-13971440 ] Jason Brown commented on CASSANDRA-7045: Interesting comments about kilim in this quasar blog: http://blog.paralleluniverse.co/2013/05/02/quasar-pulsar/ Investigate changes to threading model for network consumers to improve performance for small messages -- Key: CASSANDRA-7045 URL: https://issues.apache.org/jira/browse/CASSANDRA-7045 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance See CASSANDRA-7029 for original discussion. We can (maybe) optimise the threading model for network consumers: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. -- This message was sent by Atlassian JIRA (v6.2#6252)
