[jira] [Updated] (CASSANDRA-12379) CQLSH completion test broken by #12236
[ https://issues.apache.org/jira/browse/CASSANDRA-12379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-12379: - Issue Type: Test (was: Bug) > CQLSH completion test broken by #12236 > -- > > Key: CASSANDRA-12379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12379 > Project: Cassandra > Issue Type: Test > Components: Tools >Reporter: Sylvain Lebresne >Assignee: Stefania > Fix For: 3.8 > > > The commit of CASSANDRA-12236 appears to have broken [cqlsh completion > tests|http://cassci.datastax.com/job/cassandra-3.8_cqlsh_tests/6/cython=yes,label=ctool-lab/testReport/junit/cqlshlib.test.test_cqlsh_completion/TestCqlshCompletion/test_complete_in_create_columnfamily/]. > For the error message I suspect this may have to do with something like the > test comparing the completion output to what DESCRIBE shows and the later now > doesn't include the {{cdc}} option by default. > Anyway, I'm not really familiar with cqlsh completion nor it's test so I'm > not sure what's the best option. I don't think we want to remove {{cdc}} from > completion so I suspect we want to either special case the test somehow (no > clue how to do that), or make the test run with cdc enabled so it doesn't > complain (which I think mostly apply a change to the CI environment since it > seems the tests themselves don't spin up the cluster). > Anyway, pushing that fix to someone else as I'm not competent here and I > have't even be able to run those cqlsh test so far (getting stuck at the test > telling me that "No appropriate python interpreter found", even though I > totally have an appropriate interpreter and cqlsh works perfectly if I > execute it directly). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12379) CQLSH completion test broken by #12236
[ https://issues.apache.org/jira/browse/CASSANDRA-12379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-12379: - Resolution: Fixed Fix Version/s: 3.8 Status: Resolved (was: Patch Available) > CQLSH completion test broken by #12236 > -- > > Key: CASSANDRA-12379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12379 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Sylvain Lebresne >Assignee: Stefania > Fix For: 3.8 > > > The commit of CASSANDRA-12236 appears to have broken [cqlsh completion > tests|http://cassci.datastax.com/job/cassandra-3.8_cqlsh_tests/6/cython=yes,label=ctool-lab/testReport/junit/cqlshlib.test.test_cqlsh_completion/TestCqlshCompletion/test_complete_in_create_columnfamily/]. > For the error message I suspect this may have to do with something like the > test comparing the completion output to what DESCRIBE shows and the later now > doesn't include the {{cdc}} option by default. > Anyway, I'm not really familiar with cqlsh completion nor it's test so I'm > not sure what's the best option. I don't think we want to remove {{cdc}} from > completion so I suspect we want to either special case the test somehow (no > clue how to do that), or make the test run with cdc enabled so it doesn't > complain (which I think mostly apply a change to the CI environment since it > seems the tests themselves don't spin up the cluster). > Anyway, pushing that fix to someone else as I'm not competent here and I > have't even be able to run those cqlsh test so far (getting stuck at the test > telling me that "No appropriate python interpreter found", even though I > totally have an appropriate interpreter and cqlsh works perfectly if I > execute it directly). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12379) CQLSH completion test broken by #12236
[ https://issues.apache.org/jira/browse/CASSANDRA-12379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-12379: - Component/s: Tools > CQLSH completion test broken by #12236 > -- > > Key: CASSANDRA-12379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12379 > Project: Cassandra > Issue Type: Test > Components: Tools >Reporter: Sylvain Lebresne >Assignee: Stefania > Fix For: 3.8 > > > The commit of CASSANDRA-12236 appears to have broken [cqlsh completion > tests|http://cassci.datastax.com/job/cassandra-3.8_cqlsh_tests/6/cython=yes,label=ctool-lab/testReport/junit/cqlshlib.test.test_cqlsh_completion/TestCqlshCompletion/test_complete_in_create_columnfamily/]. > For the error message I suspect this may have to do with something like the > test comparing the completion output to what DESCRIBE shows and the later now > doesn't include the {{cdc}} option by default. > Anyway, I'm not really familiar with cqlsh completion nor it's test so I'm > not sure what's the best option. I don't think we want to remove {{cdc}} from > completion so I suspect we want to either special case the test somehow (no > clue how to do that), or make the test run with cdc enabled so it doesn't > complain (which I think mostly apply a change to the CI environment since it > seems the tests themselves don't spin up the cluster). > Anyway, pushing that fix to someone else as I'm not competent here and I > have't even be able to run those cqlsh test so far (getting stuck at the test > telling me that "No appropriate python interpreter found", even though I > totally have an appropriate interpreter and cqlsh works perfectly if I > execute it directly). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12379) CQLSH completion test broken by #12236
[ https://issues.apache.org/jira/browse/CASSANDRA-12379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411230#comment-15411230 ] Stefania commented on CASSANDRA-12379: -- I've edited the cqlsh CI jobs for [3.9|http://cassci.datastax.com/view/cassandra-3.9/job/cassandra-3.9_cqlsh_tests/8//] and [trunk|http://cassci.datastax.com/view/trunk/job/trunk_cqlsh_tests/34/] and relaunched them. They both completed without errors, closing. > CQLSH completion test broken by #12236 > -- > > Key: CASSANDRA-12379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12379 > Project: Cassandra > Issue Type: Bug >Reporter: Sylvain Lebresne >Assignee: Stefania > > The commit of CASSANDRA-12236 appears to have broken [cqlsh completion > tests|http://cassci.datastax.com/job/cassandra-3.8_cqlsh_tests/6/cython=yes,label=ctool-lab/testReport/junit/cqlshlib.test.test_cqlsh_completion/TestCqlshCompletion/test_complete_in_create_columnfamily/]. > For the error message I suspect this may have to do with something like the > test comparing the completion output to what DESCRIBE shows and the later now > doesn't include the {{cdc}} option by default. > Anyway, I'm not really familiar with cqlsh completion nor it's test so I'm > not sure what's the best option. I don't think we want to remove {{cdc}} from > completion so I suspect we want to either special case the test somehow (no > clue how to do that), or make the test run with cdc enabled so it doesn't > complain (which I think mostly apply a change to the CI environment since it > seems the tests themselves don't spin up the cluster). > Anyway, pushing that fix to someone else as I'm not competent here and I > have't even be able to run those cqlsh test so far (getting stuck at the test > telling me that "No appropriate python interpreter found", even though I > totally have an appropriate interpreter and cqlsh works perfectly if I > execute it directly). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-11521: - Status: Open (was: Patch Available) Rebasing on CASSANDRA-10707 is a bit problematic, both patches modified select statement and the pagers in a way that is not necessarily compatible. So back to in progress. > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting, protocolv5 > Fix For: 3.x > > Attachments: final-patch-jfr-profiles-1.zip > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411195#comment-15411195 ] Shogo Hoshii commented on CASSANDRA-12403: -- Hello [~Stefania], Thank you very much for quick and very detailed comment. I will look into `Monitorable`, and `MonitoringTask`. > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11701) [windows] dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows
[ https://issues.apache.org/jira/browse/CASSANDRA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411185#comment-15411185 ] Stefania commented on CASSANDRA-11701: -- CI results on Linux are good. On Windows I multiplexed copy tests 5 times without problems, [build #8|http://cassci.datastax.com/job/stef1927-dtest-multiplex-win32/8/] but [build #9|http://cassci.datastax.com/job/stef1927-dtest-multiplex-win32/9/] was aborted. So I launched [build #10|http://cassci.datastax.com/job/stef1927-dtest-multiplex-win32/10/] with some additional debug information. > [windows] dtest failure in > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows > - > > Key: CASSANDRA-11701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11701 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: Stefania > Labels: dtest, windows > > looks to be an assertion problem, so could be test or cassandra related: > e.g.: > {noformat} > 1 != 331 > {noformat} > http://cassci.datastax.com/job/trunk_dtest_win32/404/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_skip_and_max_rows > Failed on CassCI build trunk_dtest_win32 #404 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11701) [windows] dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows
[ https://issues.apache.org/jira/browse/CASSANDRA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408997#comment-15408997 ] Stefania edited comment on CASSANDRA-11701 at 8/8/16 2:10 AM: -- I rewrote the patch to avoid using a thread lock, since it cannot be pickl-ed on Windows. I extracted the pipe out of the channels, so we only transfer pipes and not channels to the child processes, which then create the channels after forking. To avoid redundant threads, which are only required for sending, I've separated receiving from sending channels. Only sending channels create a feeding thread, right in the constructor so the lock is no longer required. I'm not sure if we want this fix in 2.1 or not, it is not a critical bug but it is a regression compared to the old cqlsh COPY functionality. It is a rare failure but it can occur if the main thread of a child process needs to send an error when the receiving thread is already sending results. ||2.1||2.2||3.0||3.9||trunk|| |[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-2.1]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-2.2]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-3.9]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh]| |[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-2.1-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-2.2-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-3.0-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-3.9-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-cqlsh-tests/]| The windows test results will be available [here|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-dtest-multiplex-win32/], build -#7 and #8- #8 and #9. was (Author: stefania): I rewrote the patch to avoid using a thread lock, since it cannot be pickl-ed on Windows. I extracted the pipe out of the channels, so we only transfer pipes and not channels to the child processes, which then create the channels after forking. To avoid redundant threads, which are only required for sending, I've separated receiving from sending channels. Only sending channels create a feeding thread, right in the constructor so the lock is no longer required. I'm not sure if we want this fix in 2.1 or not, it is not a critical bug but it is a regression compared to the old cqlsh COPY functionality. It is a rare failure but it can occur if the main thread of a child process needs to send an error when the receiving thread is already sending results. ||2.1||2.2||3.0||3.9||trunk|| |[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-2.1]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-2.2]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh-3.9]|[patch|https://github.com/stef1927/cassandra/commits/11701-cqlsh]| |[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-2.1-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-2.2-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-3.0-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-3.9-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11701-cqlsh-cqlsh-tests/]| The windows test results will be available [here|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-dtest-multiplex-win32/], build #7 and #8. > [windows] dtest failure in > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows > - > > Key: CASSANDRA-11701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11701 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: Stefania > Labels: dtest, windows > > looks to be an assertion problem, so could be test or cassandra related: > e.g.: > {noformat} > 1 != 331 > {noformat} > http://cassci.datastax.com/job/trunk_dtest_win32/404/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_skip_and_max_rows > Failed on CassCI build trunk_dtest_win32 #404 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12404) ColumnFamilyStore.getIfExists
[ https://issues.apache.org/jira/browse/CASSANDRA-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stone updated CASSANDRA-12404: -- Description: ColumnFamilyStore.getIfExists("ks","cf") when it does not exist ks.cf,it should return null,but now there is a Nullpointexception. see heartbeat.PNG was: ColumnFamilyStore.getIfExists("ks","cf") when it does not exist ks.cf,it should return null,but now there is a Nullpointexception. > ColumnFamilyStore.getIfExists > - > > Key: CASSANDRA-12404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12404 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: stone > Attachments: heartbeat.PNG > > > ColumnFamilyStore.getIfExists("ks","cf") > when it does not exist ks.cf,it should return null,but now there is a > Nullpointexception. > see heartbeat.PNG -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12404) ColumnFamilyStore.getIfExists
stone created CASSANDRA-12404: - Summary: ColumnFamilyStore.getIfExists Key: CASSANDRA-12404 URL: https://issues.apache.org/jira/browse/CASSANDRA-12404 Project: Cassandra Issue Type: Bug Components: Packaging Reporter: stone Attachments: heartbeat.PNG ColumnFamilyStore.getIfExists("ks","cf") when it does not exist ks.cf,it should return null,but now there is a Nullpointexception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-12403: - Reviewer: Stefania > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411168#comment-15411168 ] Stefania commented on CASSANDRA-12403: -- Thank you for the patch. I think this is definitely something that operators will find very useful. CASSANDRA-7392 introduced a poor man's "slow query log" for queries that timed out, the code is in [{{MonitoringTask}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/monitoring/MonitoringTask.java#L144] in the [db.monitoring|https://github.com/apache/cassandra/tree/trunk/src/java/org/apache/cassandra/db/monitoring] package. This is done replica side rather than coordinator side. We may want to build on that rather than log slow queries coordinator side. If we log coordinator side, it may be that some queries took a long time because a replica was overloaded. Replica side we can instead measure the accurate query time. Each read command is already monitored, we basically just need to add a new state to {{Monitorable}}, and notify {{MonitoringTask}} when this state is triggered. I would continue logging in aggregate and at debug level, like it is currently done for queries that timed out. In {{MonitoringTask}}, we would basically first log slow queries, if any, then queries that timed out, if any. We can do this with two separate queues or by generalizing (and renaming) {{FailedOperation}}. > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12274) mx4j not work in 3.0.8
[ https://issues.apache.org/jira/browse/CASSANDRA-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated CASSANDRA-12274: --- Attachment: mx4j-error-log.txt Hi [~snazy], I've discovered by accident that this regression was indirectly caused by CASSANDRA-9402. I am testing on C* 3.9 branch. I could check that by putting the mx4j-tools.jar in the lib directory then commenting/uncommenting this line: https://github.com/apache/cassandra/blob/143d16961a67cc9d5608605ee2561253de629d2c/src/java/org/apache/cassandra/service/CassandraDaemon.java#L189 Looks like, the addition of a {{SecurityManager}} makes {{mx4j}} throws an exception and show a blank page. I have attached the Runtime error exception. {{Mx4jTool}} is the class in charge of loading mx4j if it is in the classpath. Last but not least, I found a small typo ("panalty") here: https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/cql3/functions/ThreadAwareSecurityManager.java#L36 I am sorry for not being able to provide a ready solution, but this SecurityManager + ClassLoader + XSLT stuff is beyond me now. :( > mx4j not work in 3.0.8 > -- > > Key: CASSANDRA-12274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: suse 12 > java 1.8.0_60 > mx4j 3.0.2 >Reporter: Ilya > Attachments: mx4j-error-log.txt > > > After update from 2.1 to 3.x version mx4j page begin empty > {code} > $ curl -i cassandra1:8081 > HTTP/1.0 200 OK > expires: now > Server: MX4J-HTTPD/1.0 > Cache-Control: no-cache > pragma: no-cache > Content-Type: text/html > {code} > There are no errors in the log. > logs: > {code} > ~ $ grep -i mx4j /local/apache-cassandra/logs/system.log | tail -2 > INFO [main] 2016-07-22 13:48:00,352 CassandraDaemon.java:432 - JVM > Arguments: [-Xloggc:/local/apache-cassandra//logs/gc.log, > -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, > -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/local/tmp, -Xss256k, > -XX:StringTableSize=103, -XX:+AlwaysPreTouch, -XX:+UseTLAB, > -XX:+ResizeTLAB, -XX:+UseNUMA, -Djava.net.preferIPv4Stack=true, -Xms512M, > -Xmx1G, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, > -XX:MaxGCPauseMillis=500, -XX:InitiatingHeapOccupancyPercent=25, > -XX:G1HeapRegionSize=32m, -XX:ParallelGCThreads=16, -XX:+PrintGCDetails, > -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, > -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, > -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, > -XX:CompileCommandFile=/local/apache-cassandra//conf/hotspot_compiler, > -javaagent:/local/apache-cassandra//lib/jamm-0.3.0.jar, > -Djava.rmi.server.hostname=cassandra1.d3, > -Dcom.sun.management.jmxremote.port=7199, > -Dcom.sun.management.jmxremote.rmi.port=7199, > -Dcom.sun.management.jmxremote.ssl=false, > -Dcom.sun.management.jmxremote.authenticate=false, > -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, > -Djava.library.path=/local/apache-cassandra//lib/sigar-bin, -Dmx4jport=8081, > -Dlogback.configurationFile=logback.xml, > -Dcassandra.logdir=/local/apache-cassandra//logs, > -Dcassandra.storagedir=/local/apache-cassandra//data, > -Dcassandra-pidfile=/local/apache-cassandra/run/cassandra.pid] > INFO [main] 2016-07-22 13:48:04,045 Mx4jTool.java:63 - mx4j successfuly > loaded > {code} > {code} > ~ $ sudo lsof -i:8081 > COMMAND PID USER FD TYPEDEVICE SIZE/OFF NODE NAME > java14489 cassandra 86u IPv4 381043582 0t0 TCP > cassandra1.d3:sunproxyadmin (LISTEN) > {code} > I checked versions 3.0.8 and 3.5, result the same - not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1542#comment-1542 ] Shogo Hoshii commented on CASSANDRA-12403: -- Hello [~ifesdjeen] I appreciate your advice. It is very useful in the case cassandra operator can access logs in client server. But I cannot do that because in my company my team(operators) has a lot of users from different departments and operators do not have permission to client servers. And it is also difficult for my team to let all of users to use specific driver because the organisations are separated. (I guess some of other cassandra operators are in same circumstance as my team is.) So I would like to say we need to log slow query in server side. > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411024#comment-15411024 ] Alex Petrov commented on CASSANDRA-12403: - Java Driver has a {{QueryLogger.SLOW}} that might be helpful in such circumstances. The big advantage of having it on client side is that you can also catch queries that are caused by slowness of client and network latencies between client and the server. > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12356) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_2_2_x.bug_10652_test
[ https://issues.apache.org/jira/browse/CASSANDRA-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson resolved CASSANDRA-12356. - Resolution: Duplicate > dtest failure in > upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_2_2_x.bug_10652_test > - > > Key: CASSANDRA-12356 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12356 > Project: Cassandra > Issue Type: Test >Reporter: Craig Kodman >Assignee: DS Test Eng > Labels: dtest > Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, > node3.log > > > example failure: > http://cassci.datastax.com/job/cassandra-2.2_dtest_upgrade/7/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_2_2_x/bug_10652_test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shogo Hoshii updated CASSANDRA-12403: - Status: Patch Available (was: Open) I attached the patch. I hope someone would review it! > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12403) Slow query detecting
[ https://issues.apache.org/jira/browse/CASSANDRA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shogo Hoshii updated CASSANDRA-12403: - Attachment: sample.log This is a sample result with setting - enable_logging_slow_select_query: true - logging_slow_select_query_threshold_in_ms: 10 > Slow query detecting > > > Key: CASSANDRA-12403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 > Project: Cassandra > Issue Type: Improvement >Reporter: Shogo Hoshii >Assignee: Shogo Hoshii > Attachments: sample.log, slow_query.patch > > > Hello, > In cassandra production environment, users sometimes build anti-pattern > tables and throw queries in inefficient manners. > So I would like to suggest a feature that enables to log slow query. > The feature can help cassandra operators to identify bad query patterns. > Then operators can give advices about queries and data model to users who > don't know cassandra so much. > This ticket is related to CASSANDRA-6226, and I focus on detecting bad query > patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12403) Slow query detecting
Shogo Hoshii created CASSANDRA-12403: Summary: Slow query detecting Key: CASSANDRA-12403 URL: https://issues.apache.org/jira/browse/CASSANDRA-12403 Project: Cassandra Issue Type: Improvement Reporter: Shogo Hoshii Assignee: Shogo Hoshii Attachments: slow_query.patch Hello, In cassandra production environment, users sometimes build anti-pattern tables and throw queries in inefficient manners. So I would like to suggest a feature that enables to log slow query. The feature can help cassandra operators to identify bad query patterns. Then operators can give advices about queries and data model to users who don't know cassandra so much. This ticket is related to CASSANDRA-6226, and I focus on detecting bad query patterns, not aborting them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11906) Unstable JVM due too many files when anticompacting big LCS tables
[ https://issues.apache.org/jira/browse/CASSANDRA-11906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410918#comment-15410918 ] Stefano Ortolani commented on CASSANDRA-11906: -- Some more details: * I have many LCS cfs on that cluster * Some (2/3) cfs have 300 SStables * While repairing those 2/3 cfs I see the number of SStables rising to 1K (per cf) until repairs are done (then it drops back to normal values) > Unstable JVM due too many files when anticompacting big LCS tables > -- > > Key: CASSANDRA-11906 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11906 > Project: Cassandra > Issue Type: Bug >Reporter: Stefano Ortolani > > I have recently moved from C* 2.1.x to C* 3.0.6. The setup is quite > heavy: > - 13 nodes with spinning disks > - ~120 GB of data per node > - 50% of CFs are LCS and have quite wide rows. > - 2/3 CFs with LCS have more than 200 SStables > Incremental repairs do not seem to play really well with that. > I have been running some tests node by node by using the -pr option: > {code:xml} > nodetool -h localhost repair -pr keyscheme > {code} > and to my surprise the whole process takes quite some time (1 hour > minimum, 8 hours if I haven't been repairing for 5/6 days). > Yesterday I tried to run the command with the -seq option so to > decrease the number of simultanoues compactions. After a while > the node on which I was running the repair simply died during > the anticompaction phase with the following > exception in the logs. > {code:xml} > ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-25 21:54:21,868 > ScheduledReporter.java:119 - RuntimeException thrown from > GraphiteReporter#report. Exception was suppressed. > java.lang.RuntimeException: Failed to list files in > /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396 > at > org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:57) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:691) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$SSTableLister.listFiles(Directories.java:662) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$TrueFilesSizeVisitor.(Directories.java:981) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories.getTrueAllocatedSizeIn(Directories.java:893) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories.trueSnapshotsSize(Directories.java:883) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.ColumnFamilyStore.trueSnapshotsSize(ColumnFamilyStore.java:2332) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:637) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:634) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281) > ~[metrics-graphite-3.1.0.jar:3.1.0] > at > com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158) > ~[metrics-graphite-3.1.0.jar:3.1.0] > at > com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) > ~[metrics-core-3.1.0.jar:3.1.0] > at > com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) > ~[metrics-core-3.1.0.jar:3.1.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_91] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_91] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_91] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_91] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: > /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396/ma_txn_anticompactionafterrepair_f20b50d0-22bd-11e6-970f-6f22464f4624.log: > Too many open files > at
[jira] [Reopened] (CASSANDRA-11906) Unstable JVM due too many files when anticompacting big LCS tables
[ https://issues.apache.org/jira/browse/CASSANDRA-11906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Ortolani reopened CASSANDRA-11906: -- > Unstable JVM due too many files when anticompacting big LCS tables > -- > > Key: CASSANDRA-11906 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11906 > Project: Cassandra > Issue Type: Bug >Reporter: Stefano Ortolani > > I have recently moved from C* 2.1.x to C* 3.0.6. The setup is quite > heavy: > - 13 nodes with spinning disks > - ~120 GB of data per node > - 50% of CFs are LCS and have quite wide rows. > - 2/3 CFs with LCS have more than 200 SStables > Incremental repairs do not seem to play really well with that. > I have been running some tests node by node by using the -pr option: > {code:xml} > nodetool -h localhost repair -pr keyscheme > {code} > and to my surprise the whole process takes quite some time (1 hour > minimum, 8 hours if I haven't been repairing for 5/6 days). > Yesterday I tried to run the command with the -seq option so to > decrease the number of simultanoues compactions. After a while > the node on which I was running the repair simply died during > the anticompaction phase with the following > exception in the logs. > {code:xml} > ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-25 21:54:21,868 > ScheduledReporter.java:119 - RuntimeException thrown from > GraphiteReporter#report. Exception was suppressed. > java.lang.RuntimeException: Failed to list files in > /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396 > at > org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:57) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:691) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$SSTableLister.listFiles(Directories.java:662) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories$TrueFilesSizeVisitor.(Directories.java:981) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories.getTrueAllocatedSizeIn(Directories.java:893) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.Directories.trueSnapshotsSize(Directories.java:883) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.db.ColumnFamilyStore.trueSnapshotsSize(ColumnFamilyStore.java:2332) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:637) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:634) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at > com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281) > ~[metrics-graphite-3.1.0.jar:3.1.0] > at > com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158) > ~[metrics-graphite-3.1.0.jar:3.1.0] > at > com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) > ~[metrics-core-3.1.0.jar:3.1.0] > at > com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) > ~[metrics-core-3.1.0.jar:3.1.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_91] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_91] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_91] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_91] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: > /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396/ma_txn_anticompactionafterrepair_f20b50d0-22bd-11e6-970f-6f22464f4624.log: > Too many open files > at org.apache.cassandra.io.util.FileUtils.readLines(FileUtils.java:622) > ~[apache-cassandra-3.0.6.jar:3.0.6] > at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321) > ~[na:1.8.0_91] > at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) > ~[na:1.8.0_91] > at >
[jira] [Commented] (CASSANDRA-11906) Unstable JVM due too many files when anticompacting big LCS tables
[ https://issues.apache.org/jira/browse/CASSANDRA-11906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410907#comment-15410907 ] Stefano Ortolani commented on CASSANDRA-11906: -- Unfortunately I might need to reopen this issues since one node died last night while doing a sequential repair. {noformat} INFO [STREAM-IN-/10.12.81.6] 2016-08-07 00:07:11,145 StreamResultFuture.java:169 - [Stream #e276de60-5c32-11e6-b182-e548aa39c6eb ID#0] Prepare completed. Receiving 2 files(1974371 bytes), sending 2 files(2101094 bytes) INFO [AntiEntropyStage:1] 2016-08-07 00:07:11,895 RepairSession.java:181 - [repair #739ae1c1-5c2e-11e6-b182-e548aa39c6eb] Received merkle tree for llkgbsubject2sample from /10.12.33.6 INFO [AntiEntropyStage:1] 2016-08-07 00:07:11,895 RepairJob.java:213 - Validating /10.12.81.6 WARN [STREAM-IN-/10.12.81.6] 2016-08-07 00:07:11,970 CLibrary.java:280 - open(/data/cassandra/data/schema/data-cf-685a935014a311e69af3cdff32bdcb0d, O_RDONLY) failed, errno (24). WARN [STREAM-IN-/10.12.33.6] 2016-08-07 00:07:11,970 CLibrary.java:280 - open(/data/cassandra/data/schema/data-cf-685a935014a311e69af3cdff32bdcb0d, O_RDONLY) failed, errno (24). WARN [STREAM-IN-/10.12.81.6] 2016-08-07 00:07:11,970 CompressedStreamReader.java:118 - [Stream e276de60-5c32-11e6-b182-e548aa39c6eb] Error while reading partition null from stream on ks='schema' and table='data-cf'. ERROR [STREAM-IN-/10.12.81.6] 2016-08-07 00:07:11,984 StreamSession.java:528 - [Stream #e276de60-5c32-11e6-b182-e548aa39c6eb] Streaming error occurred java.nio.file.FileSystemException: /data/cassandra/data/schema/data-cf-685a935014a311e69af3cdff32bdcb0d/mb-4984-big-Index.db: Too many open files at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[na:1.8.0_91] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_91] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_91] at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) ~[na:1.8.0_91] at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[na:1.8.0_91] at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[na:1.8.0_91] at org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:114) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:135) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:150) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.(BigTableWriter.java:375) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:84) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:516) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:498) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:481) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:476) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:162) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:92) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59) ~[apache-cassandra-3.0.8.jar:3.0.8] at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287) ~[apache-cassandra-3.0.8.jar:3.0.8] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]