[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310458#comment-15310458
]
jean carlo rivera ura commented on CASSANDRA-9126:
--
hello
[~mambocab]I found the ticket that correspond to our problem
https://issues.apache.org/jira/browse/CASSANDRA-9935
Thank you I think we can close this ticket :)
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
>Reporter: srinivasu gottipati
>Priority: Critical
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308195#comment-15308195
]
Jim Witschey commented on CASSANDRA-9126:
-
[~krummas] I don't have the context anymore to quickly understand this new
information and determine if this needs immediate attention. Do you? I can put
this in TE's queue for triage if you like, but you may be able to do so faster
than us.
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
>Reporter: srinivasu gottipati
>Priority: Critical
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[ https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304173#comment-15304173 ] jean carlo rivera ura commented on CASSANDRA-9126: -- [~mambocab] We are using cassandra 2.1.12 In production we got a first error doing a repair (-pr -par) ERROR [ValidationExecutor:3283] 2016-05-26 09:37:37,911 Validator.java:245 - Failed creating a merkle tree for [repair #b16c1f30-2314-11e6-97fe-dd058ca99653 on pns_fr_prod/pig, (8699512854132214411,8702471448538509513]], /10.98.255.144 (see log for details) node021.cassandra.prod.pns.s1.p.fti.net: /var/opt/hosting/log/cassandra/system.log-ERROR [ValidationExecutor:3283] 2016-05-26 09:37:37,913 CassandraDaemon.java:227 - Exception in thread Thread[ValidationExecutor:3283,1,main] node021.cassandra.prod.pns.s1.p.fti.net: /var/opt/hosting/log/cassandra/system.log-java.lang.AssertionError: row DecoratedKey(8699513492008207074, 434c503031303030303030303036363632323233) received out of order wrt DecoratedKey(8702470199759211565, 4b454e4f42494a52432d435531303330313532323130 We decided to do another a repair over this token range 8699512854132214411,8702471448538509513 using the next command. root@node021[SPH][PROD][PnS3]:~$ nodetool repair pns_fr_prod -st 8699512854132214411 -et 8702471448538509513 [2016-05-27 14:19:20,768] Starting repair command #37, repairing 1 ranges for keyspace pns_fr_prod (parallelism=SEQUENTIAL, full=true) [2016-05-27 14:19:38,489] Repair session 3dd2f250-2405-11e6-b33f-ab1665ddb9c0 for range (8699512854132214411,8702471448538509513] failed with error org.apache.cassandra.exceptions.RepairException: [repair #3dd2f250-2405-11e6-b33f-ab1665ddb9c0 on pns_fr_prod/syndic, (8699512854132214411,8702471448538509513]] Validation failed in /10.234.72.137 [2016-05-27 14:19:38,489] Repair command #37 finished error: nodetool failed, check server logs -- StackTrace -- java.lang.RuntimeException: nodetool failed, check server logs at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:292) at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:204) Unlike the first time, this error comes from the node 10.234.72.137. Checking the log, the node failed creating a merkle tree for the table syndic. This table is LCS ERROR [ValidationExecutor:3427] 2016-05-27 14:19:38,466 Validator.java:245 - Failed creating a merkle tree for [repair #3dd2f250-2405-11e6-b33f-ab1665ddb9c0 on pns_fr_prod/syndic, (8699512854132214411,8702471448538509513]], /10.98.255.154 (see log for details) ERROR [ValidationExecutor:3427] 2016-05-27 14:19:38,467 CassandraDaemon.java:227 - Exception in thread Thread[ValidationExecutor:3427,1,main] java.lang.AssertionError: row DecoratedKey(8699513197702636918, 49442d5350502d3130302d54513944486f7333704e68492b387a6e786870653347614d766a683259776a5543446d4b397a393545) received out of order wrt DecoratedKey(8702471248515353880, 49442d5350502d3130302d704a3063676b4446714b5075644b68654273672f686c6a4759767264724f4b6e664d3176765a7a70416d6b) at org.apache.cassandra.repair.Validator.add(Validator.java:126) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1038) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:89) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:649) ~[apache-cassandra-2.1.12.jar:2.1.12] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] We tried to run again the same repair on the same node with the same tokenrange and it finished without error root@node021[SPH][PROD][PnS3]:~$ nodetool repair pns_fr_prod -st 8699512854132214411 -et 8702471448538509513 [2016-05-27 14:27:00,419] Starting repair command #38, repairing 1 ranges for keyspace pns_fr_prod (parallelism=SEQUENTIAL, full=true) [2016-05-27 14:36:18,239] Repair session 4fcda620-2406-11e6-b33f-ab1665ddb9c0 for range (8699512854132214411,8702471448538509513] finished [2016-05-27 14:36:18,239] Repair command #38 finished It seems this error comes up by hazard. We notice that our first repair -pr -par has finished repairing all the tokenranges, but that one we got the error. > java.lang.RuntimeException: Last written key DecoratedKey >= current key > DecoratedKey > - > > Key: CASSANDRA-9126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9126 >
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953225#comment-14953225
]
Jim Witschey commented on CASSANDRA-9126:
-
[~krummas] Ok, then, I'll close this -- you should reopen if there's more for
us to do, but it sounds like you're satisfied?
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946781#comment-14946781
]
Marcus Eriksson commented on CASSANDRA-9126:
[~mambocab] yeah, I think we want users to actually report this issue if it
happens - it could be a legitimate bug in LCS for example
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944705#comment-14944705
]
Marcus Eriksson commented on CASSANDRA-9126:
[~mambocab] it is likely this was due to either a corrupt sstable or overlap in
the LCS leveling, scrubbing and restarting solves both
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945115#comment-14945115
]
Jim Witschey commented on CASSANDRA-9126:
-
[~krummas] Sure -- could we add that advice to the error messages these users
received? Or are these errors raised in too many situations for that to be
appropriate?
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943601#comment-14943601
]
Jim Witschey commented on CASSANDRA-9126:
-
[~dkblinux98] and [~sgottipati] do you have any more details, INFO-level logs
in particular?
[~krummas], [~yukim], [~benedict] Any thoughts on next steps here? At the very
least, is it possible to guide people toward {{nodetool scrub}} with error
reporting?
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804427#comment-14804427
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
No. There were no such errors since I restarted affected node with INFO level.
So for me solution was 'nodetool scrub' + node restart.
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804325#comment-14804325
]
Jim Witschey commented on CASSANDRA-9126:
-
[~saprykin] [~sgottipati] Do either of you have INFO-level logs for this error?
[~krummas] Do you have any new ideas about what might be causing this, or know
of any other tickets this might be related to?
> java.lang.RuntimeException: Last written key DecoratedKey >= current key
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue
> was raised in earlier versions and marked as closed. However it still appears
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing
> with this error.:
> {code}java.lang.RuntimeException: Last written key
> DecoratedKey(3462767860784856708,
> 354038323137333038305f3330325f31355f474d4543454f) >= current key
> DecoratedKey(3462334604624154281,
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708008#comment-14708008
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
I also experience the same issue. I did 'nodetool scrub' for affected
columnfamily without any errors. And issue persists after scrub
We use:
Cassandra 2.0.15, LCS, HSHA.
Keyspace is 8 nodes, RF=3, ~300GB per node (~1TB of unique data without RF),
~10.000.000 keys
Before scrub we had:
{noformat}
ERROR [CompactionExecutor:499] 2015-08-21 20:26:24,438 CassandraDaemon.java
(line 258) Exception in thread Thread[CompactionExecutor:499,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(1928386083417163406,
0313ad7e) = current key DecoratedKey(1911131194341551311, 010b2fb4) writing
into /data/cassandra/storage/data/bigks/chunks/bigks-chunks-tmp-jb-26880-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:170)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR [CompactionExecutor:498] 2015-08-21 20:29:14,655 CassandraDaemon.java
(line 258) Exception in thread Thread[CompactionExecutor:498,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(1928386083417163406,
0313ad7e) = current key DecoratedKey(1911131194341551311, 010b2fb4) writing
into /data/cassandra/storage/data/bigks/chunks/bigks-chunks-tmp-jb-26954-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:170)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
After scrub I see following errors during compaction:
{noformat}
ERROR [CompactionExecutor:815] 2015-08-22 03:16:17,153 CassandraDaemon.java
(line 258) Exception in thread Thread[CompactionExecutor:815,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(1922197470632862656,
02c3a415) = current key DecoratedKey(1916775706087412870, 013a445b) writing
into /data/cassandra/storage/data/bigks/chunks/bigks-chunks-tmp-jb-28982-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:170)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708010#comment-14708010
]
Marcus Eriksson commented on CASSANDRA-9126:
[~saprykin] could you post full logs (ie, from restart until the exception
occurs?) and also a nodetool cfstats would be good for the affected column
family
[~dkblinux98] in 2.0.9 it is most likely a LCS overlap issue - upgrade to
latest 2.0.x to fix
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708014#comment-14708014
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
Thanks a lot for your reply!
cfstats output for affected keyspace:
{noformat}
Keyspace: bigks
Read Count: 50
Read Latency: 12.39796 ms.
Write Count: 228916
Write Latency: 0.031050756609411312 ms.
Pending Tasks: 0
Table: chunks
SSTable count: 1208
SSTables in each level: [5/4, 14/10, 105/100, 1016/1000, 68, 0,
0, 0, 0]
Space used (live), bytes: 311268548761
Space used (total), bytes: 311268548761
Off heap memory used (total), bytes: 353408
SSTable Compression Ratio: 0.0
Number of keys (estimate): 329472
Memtable cell count: 0
Memtable data size, bytes: 0
Memtable switch count: 1
Local read count: 0
Local read latency: 0.000 ms
Local write count: 11
Local write latency: 0.000 ms
Pending tasks: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.0
Bloom filter space used, bytes: 321888
Bloom filter off heap memory used, bytes: 312224
Index summary off heap memory used, bytes: 41184
Compression metadata off heap memory used, bytes: 0
Compacted partition minimum bytes: 61215
Compacted partition maximum bytes: 36157190
Compacted partition mean bytes: 1325042
Average live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0
{noformat}
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708020#comment-14708020
]
Marcus Eriksson commented on CASSANDRA-9126:
that log makes no sense - did you remove the INFO lines?
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708021#comment-14708021
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
Schema:
{noformat}
CREATE KEYSPACE bigks WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC1': '3',
'DC2': '3'
};
USE bigks;
CREATE TABLE chunks (
id int,
local_id int,
data blob,
PRIMARY KEY ((id), local_id)
) WITH
bloom_filter_fp_chance=0.10 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.10 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.00 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'sstable_size_in_mb': '250', 'class':
'LeveledCompactionStrategy'} AND
compression={};
{noformat}
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708022#comment-14708022
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
No, I have removed nothing. This is complete log since last restart.
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708026#comment-14708026
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
restarted with INFO level. will attached updated log in the evening
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708016#comment-14708016
]
Marcus Eriksson commented on CASSANDRA-9126:
[~saprykin] could you post your schema as well?
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708019#comment-14708019
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
Attached system.log since last restart. I did scrub between 2 lines:
{noformat}
ERROR [CompactionExecutor:498] 2015-08-21 20:29:14,655 CassandraDaemon.java
(line 258) Exception in thread Thread[CompactionExecutor:498,1,main]
...
ERROR [CompactionExecutor:598] 2015-08-21 23:20:49,252 CassandraDaemon.java
(line 258) Exception in thread Thread[CompactionExecutor:598,1,main]
{noformat}
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708023#comment-14708023
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
Sorry, I have log level WARN. If necessary I can change it and post update when
issue appears again.
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708025#comment-14708025
]
Marcus Eriksson commented on CASSANDRA-9126:
yes, that would be helpful
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708230#comment-14708230
]
Dmitrii Saprykin commented on CASSANDRA-9126:
-
I restarted 2 nodes with INFO log level and there were no errors since restart.
So for me solution (at least temporary) was to make 'nodetool scrub' for
affected keyspace and restart node after scrub. I will leave INFO log.level and
if issue reappears post full logs here. Can it be LCS overlap issue like in
Darla Baker case? Do you advise to update to latest 2.0.x to fix? I am asking
because I see nothing connected with LCS fixes in 2.0.16 and 2.0.17 changelogs.
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Attachments: cassandra-system.log
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600119#comment-14600119
]
Darla Baker commented on CASSANDRA-9126:
Another user reported this same Error in 2.0.9 and they are using hsha and lcs.
scrub fixes the issue but it returns on occasion. As I get more details, I'll
update.
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.x
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492122#comment-14492122
]
Benedict commented on CASSANDRA-9126:
-
[~yukim]?
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.15
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492278#comment-14492278
]
Marcus Eriksson commented on CASSANDRA-9126:
[~sgottipati] did you get any errors while scrubbing?
are you using LCS?
anything special with your workload?
java.lang.RuntimeException: Last written key DecoratedKey = current key
DecoratedKey
-
Key: CASSANDRA-9126
URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: srinivasu gottipati
Priority: Critical
Fix For: 2.0.15
Cassandra V: 2.0.14,
Getting the following exceptions while trying to compact (I see this issue
was raised in earlier versions and marked as closed. However it still appears
in 2.0.14). In our case, compaction is not getting succeeded and keep failing
with this error.:
{code}java.lang.RuntimeException: Last written key
DecoratedKey(3462767860784856708,
354038323137333038305f3330325f31355f474d4543454f) = current key
DecoratedKey(3462334604624154281,
354036333036353334315f3336315f31355f474d4543454f) writing into {code}
...
Stacktrace:{code}
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}
Any help is greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey
[ https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482824#comment-14482824 ] Benedict commented on CASSANDRA-9126: - The bug may have affected you in an earlier version, and the result of the bug persisting. You should run nodetool scrub on the affected node(s). If the problem reappears after this, it may be a new (or regressed) cause. java.lang.RuntimeException: Last written key DecoratedKey = current key DecoratedKey - Key: CASSANDRA-9126 URL: https://issues.apache.org/jira/browse/CASSANDRA-9126 Project: Cassandra Issue Type: Bug Components: Core Reporter: srinivasu gottipati Priority: Blocker Fix For: 2.0.14 Cassandra V: 2.0.14, Getting the following exceptions while trying to compact (I see this issue was raised in earlier versions and marked as closed. However it still appears in 2.0.14). In our case, compaction is not getting succeeded and keep failing with this error.: java.lang.RuntimeException: Last written key DecoratedKey(3462767860784856708, 354038323137333038305f3330325f31355f474d4543454f) = current key DecoratedKey(3462334604624154281, 354036333036353334315f3336315f31355f474d4543454f) writing into ... Stacktrace: at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166) at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Any help is greatly appreciated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
