Hello,
we are currently in the process of upgrading from 2.1.18 to 3.0.14. After
upgrading a few test environments, we start to see some suspicious log entries
regarding repair issues.
We have a cron job on all nodes basically executing the following repair call
on a daily basis:
nodetool repair -pr <list of CFs>
This gets started on all nodes at the same time. While this has worked with
2.1.18 (at least we haven't seen anything suspicious in Cassandra log), with
3.0.14 we get something similar like that on all nodes (see below; IP addresses
and KS/CF faked).
Any pointers are appreciated. Thanks.
Thomas
INFO [Thread-2941] 2017-09-15 03:00:28,036 RepairSession.java:224 - [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0] new session: will sync /FAKE.33.64,
/FAKE.35.153, /FAKE.34.171 on range [(8195393703879512303,8196334842725538685],
(8166975326273137878,8182604850967732931],
(-7246799942440641887,-7227869626613009045],
(-8371707510273823988,-8365977215604569699],
(-141862581573028594,-140310864869418908],
(3732113975108886193,3743105867152786342],
(4998127507903069087,5008922734235607550],
(-5115827291264930140,-5111054924035590372],
(-2475342271852943287,-2447285553369030332],
(-8318606053827235336,-8308721754886697230],
(-5208900659917654871,-5202385837264015269],
(6618737991399272130,6623100721269775102],
(-4650650128572424858,-4650260492494258461],
(1886545362164970333,1886646959491599822],
(-4511817721998311568,-4507491187192881115],
(8114903118676615937,8132992506844206601],
(6224957219376301858,6304379125732293904],
(-3460547504877234383,-3459262416082517136],
(-167838948111369123,-141862581573028594],
(481579232521229473,491242114841289497],
(4052464144722307684,4059745901618136723],
(1659668187498418295,1679582585970705122],
(-1118922763210109192,-1093766915505652874],
(7504365235878319341,7526188885210185292],
(-79866884352549492,-77667207866300333],
(8151204058820798561,8154760186218662205],
(-1040398370287131739,-1033770179677543189],
(3767057277953758442,3783780844370292025],
(-6491678058233994892,-6487797181789288329],
(-916868210769480248,-907141794196269524],
(-9005441616028750657,-9002220258513351832],
(8183526518331102304,8186908810225025483],
(-5685737903527826627,-5672136154194382932],
(4976122621177738811,4987871287137312689],
(6051670147160447042,6051686987147911650],
(-1161640137086921883,-1159172734746043158],
(6895951547735922309,6899152466544114890],
(-3357667382515377172,-3356304907368646189],
(-5370953856683870319,-5345971445444542485],
(3824272999898372667,3829315045986248983],
(8132992506844206601,8149858096109302285],
(3975126143101303723,3980729378827590597],
(-956691623200349709,-946602525018301692],
(-82499927325251331,-79866884352549492],
(3952144214544622998,3955602392726495936],
(8154760186218662205,8157079055586089583],
(3840595196718778916,3866458971850198755],
(-1066905024007783341,-1055954824488508260],
(-7252356975874511782,-7246799942440641887],
(-810612946397276081,-792189809286829222],
(4964519403172053705,4970446606512414858],
(-5380038118840759647,-5370953856683870319],
(-3221630728515706463,-3206856875356976885],
(-1193448110686154165,-1161640137086921883],
(-3356304907368646189,-3346460884208327912],
(3466596314109623830,3468144326691723333],
(-9050241313548454460,-9005441616028750657],
(402227699082311580,407458511300218383]] for XXX.[YYY, ZZZ]
INFO [Repair#1:1] 2017-09-15 03:00:28,419 RepairJob.java:172 - [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0] Requesting merkle trees for YYY (to
[/FAKE.35.153, /FAKE.34.171, /FAKE.33.64])
INFO [Thread-2941] 2017-09-15 03:00:28,434 RepairSession.java:224 - [repair
#075d2720-99c2-11e7-91dc-6132f5fe5fb0] new session: will sync /FAKE.33.64,
/FAKE.35.57, /FAKE.34.171 on range
[(-5410955131843184047,-5390722609201388849],
(-2429793939970389370,-2402273315769352748],
(8085575576842594575,8086965740279021106],
(-8802193901675845653,-8790472027607832351],
(-3900412470120874591,-3892641480459306647],
(5455804264750818305,5465037357825542970],
(4930767198829659527,4939587074207662799],
(8086965740279021106,8087442741329154201],
(-8933201045321260661,-8926445549049070674],
(-4841328524165418854,-4838895482794593338],
(628107265570603622,682509946926464280],
(7043245467621414187,7055126022831789025],
(624871765540463735,627374995781897409],
(9219228482330263660,9221294940422311559],
(-2335215188301493066,-2315034243278984017],
(-6216599212198827632,-6211460136507414133],
(-3276490559558850323,-3273110814046238767],
(7204991007334459472,7214826985711309418],
(1815809811279373566,1846961604192445001],
(8743912118048160970,8751518028513315549],
(-9204701745739426439,-9200185935622985719],
(7926527126882050773,7941554683778488797],
(-1307707180308444994,-1274682085495751899],
(8354147540115782875,8358523989614737607],
(-5418282332713406631,-5415077779309282099],
(2436459402559272117,2441988676982099299],
(3718536403335860823,3722612994041485455],
(613072756805503750,624871765540463735],
(3348338086790700840,3352211222405547568],
(7941554683778488797,7949493423692701952],
(-319434281654804937,-312029192727845573],
(6560852500852746428,6577389024948220270],
(3468872066705970665,3469611600245714638],
(-4691666557374653357,-4691655196509961176],
(1517867403379415135,1524487506534693145],
(-7422802504145504981,-7402342710717945820],
(1766173970230627147,1798177111825664424],
(-4402244320098942276,-4399211415161403124],
(-6024004998426219783,-6013031695922198269],
(-6930256177103656090,-6926607739769548590],
(-3578196517591043633,-3561058036097453011],
(-7923695406605667073,-7885733750925254770],
(-8063817427378208986,-8061632046535266293],
(-7341733690813664665,-7287847431444491429],
(-8327456680139422800,-8322944736494135341],
(885662136143255295,898069608029365525],
(-5555349213881020405,-5555145874347997903],
(-4895525453201761515,-4892963646364151947],
(4925572085497096406,4926057814710187377],
(7785700473508314561,7813617525484553901],
(-1598850926292630432,-1593204162764888475],
(-2583030464997858536,-2581717714589673598],
(7780476570449984013,7785700473508314561],
(-3558045406807923433,-3550343462935144866],
(7571721681971063633,7578715430118769516],
(8865905084309486257,8879414598472867800],
(-4984170160639031182,-4973560491294201055],
(-5469249030580263062,-5432196900285376733],
(-340636581410346966,-324449230085298889],
(9120471287581648922,9127588925404102522],
(8366987539912250171,8368772126617619299],
(-9125573241533654210,-9123870950381500195],
(1193377366975272040,1205407211929215537],
(8879414598472867800,8900714457425949896],
(7055126022831789025,7065799572034111661],
(-1588707445511226686,-1539910209795101605],
(6990922382177634221,7007948980474566617]] for XXX.[YYY, ZZZ]
INFO [Thread-2941] 2017-09-15 03:00:28,778 RepairSession.java:224 - [repair
#0791a4a0-99c2-11e7-91dc-6132f5fe5fb0] new session: will sync /FAKE.33.64,
/FAKE.35.57, /FAKE.34.90 on range [(-3078252446668567055,-3069777768104388887],
(3648099278098819362,3665843214181043507],
(5797678959887947223,5820452778888079981],
(-2642545928835445644,-2631494184435884123],
(-1612062874902452641,-1610692114449831135],
(-1707971415501754602,-1692411707392102715],
(291977720343160936,302780457185309086],
(-8155767539645304082,-8140296327332660530],
(5553475010776119070,5565820507856584918],
(3560286576304809625,3566372473455883512],
(5226123540640728306,5228400303590993685],
(3421729237821036215,3428685863718017921],
(1227377289103207735,1240348980976161342],
(3246531324407742399,3247845316596819400],
(5304633131413824443,5306083993155890840],
(-8619514398407319973,-8606439320645645548],
(5911017071945481872,5912872298340550901],
(-5637986723275926091,-5632042588860715397],
(-7132684921710202809,-7132365563419969085],
(8246342224027692193,8283424185055442516],
(-4044458382788374703,-4036186919446747968],
(-5613445605139257263,-5596969946550242654],
(1083135773045625799,1084813234330856986],
(-8140296327332660530,-8138240730730282332],
(2996936679493130945,3002752531484603869],
(-4490677154018906600,-4487029511866186581],
(8456154651893349894,8463606226270547769],
(-2656313914110802764,-2642545928835445644],
(7193156006022031982,7197068598281426786],
(8318481021259042556,8345872746994439720],
(-1692411707392102715,-1673006435564531865],
(1097191643473406518,1104630053604629743],
(-3613177001569108681,-3599878627600515791],
(-1662308889719802294,-1656482847672279831],
(-6811080679276087557,-6804940650782637386],
(7392275851743093948,7393304112887579707],
(4240513765329086484,4257716503015670881],
(-5036265422304136579,-5029426061972234042],
(-4230577892781670592,-4228053239714848467],
(-8159151100170546488,-8155767539645304082],
(-5492523660879132140,-5478464594772026508],
(3421125671176312710,3421729237821036215],
(5262611881402902860,5275876653180871852],
(1577722798587790899,1613215617842269344],
(6666156845317387896,6676801654642938276],
(5155814764198615180,5168412530555049843],
(-6824229014637940118,-6819748162411093540],
(-3666684985038092211,-3613177001569108681],
(-8874623295866980549,-8872063750661463000],
(-3599878627600515791,-3599733923716673512],
(-5960723066215554398,-5949787783010678970],
(258158907638948329,259127284829404565],
(6661762711488840391,6666156845317387896],
(-6367100347601878936,-6363911697345955296],
(-6250542414915159381,-6229550501272299394],
(-6917261891898915460,-6904807338337882298],
(-7449919126757571330,-7438527202807402113],
(-6904807338337882298,-6881332882762615208],
(-7942161058117423218,-7938222136384107977],
(781504901963705642,791302021433124535],
(6527741860207715339,6537050054260012793],
(7186200312153798494,7187207897161667549]] for XXX.[YYY, ZZZ]
INFO [Thread-2941] 2017-09-15 03:00:28,942 RepairSession.java:224 - [repair
#07aaaae0-99c2-11e7-91dc-6132f5fe5fb0] new session: will sync /FAKE.33.64,
/FAKE.35.153, /FAKE.34.90 on range [(3075097238958732453,3095612227388458535],
(-3791204388712221856,-3785200705970537637],
(8410714452405602873,8417924634749091961],
(5114137550939144346,5121098642749542165],
(2740032807677652551,2746972622905345611],
(2197312222134575375,2220347677927119914],
(4864988928009437565,4869276508135102594],
(-3451050463660365008,-3450091362870187260],
(7424782039380072835,7472858232992173041],
(-5338516343760774634,-5325637469358550877],
(2731359217194343571,2734544646630861913],
(382200642783712616,390184326306122373],
(430251086609083397,431821763463501369],
(-5338895526845396808,-5338516343760774634],
(-2677272836643942252,-2671813555202977248],
(-2132202487362637132,-2128425620095084071],
(2165546227633605752,2174360971273026818],
(2763720958753148061,2764848409944217454],
(3538741049060169597,3547917038270802802],
(-6741608677619654195,-6731111079555719429],
(2764848409944217454,2779298419000989854],
(374883901669627753,382200642783712616],
(-2706148319144852162,-2694312255222799319],
(7132646356350938192,7136352706932176819],
(194821142331236531,228318513249291325],
(-2047674613927860008,-2032759017860426681],
(-2032759017860426681,-2011077831196488705],
(7472858232992173041,7477981949405760270],
(431821763463501369,449911816694531926],
(4347942912351042626,4349624658068191606],
(311672118330363382,320380777495445406],
(3099877433733294763,3112419968835610433],
(-7182182373680571317,-7177421545891675761],
(-6742074278675667310,-6741608677619654195],
(-2770159943814078393,-2757633164368231752],
(3095612227388458535,3099877433733294763],
(-3741816028443553809,-3741630281022069399],
(-738951315633674297,-727109613092919188],
(4851195187346961355,4864988928009437565],
(-769229207399684543,-744901822064683937],
(-4143548408671617625,-4141029975015318194],
(-7957562451432593113,-7952882742232241654],
(-511680378144663402,-494227476671052554],
(-832115128813031544,-819699210830328959],
(6784361874104123161,6792435444931864417],
(-8284973647739186524,-8256379601422884439],
(-5321596150219749918,-5309942391653520721],
(4723799060589319973,4734570102349898748],
(5077208816402461873,5091797538025862046],
(7487792874019513006,7489186250794409815],
(-1647123563569159073,-1620470309759457440],
(2779298419000989854,2784870878309140225],
(320380777495445406,344020839234233467],
(3112419968835610433,3119057615014897057],
(-8506504972343997648,-8502433630545238869],
(-2810980092222625726,-2810271001960433176],
(344020839234233467,347093216302465566],
(-8545779729891984163,-8519832061775919840],
(2633165392330429557,2680549740474631393],
(6391794517057168460,6431615673158447973],
(6440766036098716417,6458083094779720657],
(-8735147357564473077,-8730522574908969951],
(7484355688408489651,7487792874019513006],
(4325444310827485194,4347942912351042626],
(2220347677927119914,2223568835084606157],
(6802149418704156481,6819713717509862346],
(4827439895247379322,4851195187346961355],
(-4016529712384875309,-4009393333251988044],
(2018021481024026660,2049270980733207626]] for XXX.[YYY, ZZZ]
ERROR [ValidationExecutor:3] 2017-09-15 03:00:29,471
ActiveRepairService.java:554 - Cannot start multiple repair sessions over the
same sstables
ERROR [ValidationExecutor:3] 2017-09-15 03:00:29,471 Validator.java:268 -
Failed creating a merkle tree for [repair #071f81e0-99c2-11e7-91dc-6132f5fe5fb0
on XXX/YYY, [(8195393703879512303,8196334842725538685],
(8166975326273137878,8182604850967732931],
(-7246799942440641887,-7227869626613009045],
(-8371707510273823988,-8365977215604569699],
(-141862581573028594,-140310864869418908],
(3732113975108886193,3743105867152786342],
(4998127507903069087,5008922734235607550],
(-5115827291264930140,-5111054924035590372],
(-2475342271852943287,-2447285553369030332],
(-8318606053827235336,-8308721754886697230],
(-5208900659917654871,-5202385837264015269],
(6618737991399272130,6623100721269775102],
(-4650650128572424858,-4650260492494258461],
(1886545362164970333,1886646959491599822],
(-4511817721998311568,-4507491187192881115],
(8114903118676615937,8132992506844206601],
(6224957219376301858,6304379125732293904],
(-3460547504877234383,-3459262416082517136],
(-167838948111369123,-141862581573028594],
(481579232521229473,491242114841289497],
(4052464144722307684,4059745901618136723],
(1659668187498418295,1679582585970705122],
(-1118922763210109192,-1093766915505652874],
(7504365235878319341,7526188885210185292],
(-79866884352549492,-77667207866300333],
(8151204058820798561,8154760186218662205],
(-1040398370287131739,-1033770179677543189],
(3767057277953758442,3783780844370292025],
(-6491678058233994892,-6487797181789288329],
(-916868210769480248,-907141794196269524],
(-9005441616028750657,-9002220258513351832],
(8183526518331102304,8186908810225025483],
(-5685737903527826627,-5672136154194382932],
(4976122621177738811,4987871287137312689],
(6051670147160447042,6051686987147911650],
(-1161640137086921883,-1159172734746043158],
(6895951547735922309,6899152466544114890],
(-3357667382515377172,-3356304907368646189],
(-5370953856683870319,-5345971445444542485],
(3824272999898372667,3829315045986248983],
(8132992506844206601,8149858096109302285],
(3975126143101303723,3980729378827590597],
(-956691623200349709,-946602525018301692],
(-82499927325251331,-79866884352549492],
(3952144214544622998,3955602392726495936],
(8154760186218662205,8157079055586089583],
(3840595196718778916,3866458971850198755],
(-1066905024007783341,-1055954824488508260],
(-7252356975874511782,-7246799942440641887],
(-810612946397276081,-792189809286829222],
(4964519403172053705,4970446606512414858],
(-5380038118840759647,-5370953856683870319],
(-3221630728515706463,-3206856875356976885],
(-1193448110686154165,-1161640137086921883],
(-3356304907368646189,-3346460884208327912],
(3466596314109623830,3468144326691723333],
(-9050241313548454460,-9005441616028750657],
(402227699082311580,407458511300218383]]], /FAKE.33.64 (see log for details)
INFO [AntiEntropyStage:1] 2017-09-15 03:00:29,473 RepairSession.java:176 -
[repair #071f81e0-99c2-11e7-91dc-6132f5fe5fb0] Received merkle tree for YYY
from /FAKE.33.64
ERROR [Repair#1:1] 2017-09-15 03:00:29,492 CassandraDaemon.java:207 - Exception
in thread Thread[Repair#1:1,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.cassandra.exceptions.RepairException: [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0 on XXX/YYY,
[(8195393703879512303,8196334842725538685],
(8166975326273137878,8182604850967732931],
(-7246799942440641887,-7227869626613009045],
(-8371707510273823988,-8365977215604569699],
(-141862581573028594,-140310864869418908],
(3732113975108886193,3743105867152786342],
(4998127507903069087,5008922734235607550],
(-5115827291264930140,-5111054924035590372],
(-2475342271852943287,-2447285553369030332],
(-8318606053827235336,-8308721754886697230],
(-5208900659917654871,-5202385837264015269],
(6618737991399272130,6623100721269775102],
(-4650650128572424858,-4650260492494258461],
(1886545362164970333,1886646959491599822],
(-4511817721998311568,-4507491187192881115],
(8114903118676615937,8132992506844206601],
(6224957219376301858,6304379125732293904],
(-3460547504877234383,-3459262416082517136],
(-167838948111369123,-141862581573028594],
(481579232521229473,491242114841289497],
(4052464144722307684,4059745901618136723],
(1659668187498418295,1679582585970705122],
(-1118922763210109192,-1093766915505652874],
(7504365235878319341,7526188885210185292],
(-79866884352549492,-77667207866300333],
(8151204058820798561,8154760186218662205],
(-1040398370287131739,-1033770179677543189],
(3767057277953758442,3783780844370292025],
(-6491678058233994892,-6487797181789288329],
(-916868210769480248,-907141794196269524],
(-9005441616028750657,-9002220258513351832],
(8183526518331102304,8186908810225025483],
(-5685737903527826627,-5672136154194382932],
(4976122621177738811,4987871287137312689],
(6051670147160447042,6051686987147911650],
(-1161640137086921883,-1159172734746043158],
(6895951547735922309,6899152466544114890],
(-3357667382515377172,-3356304907368646189],
(-5370953856683870319,-5345971445444542485],
(3824272999898372667,3829315045986248983],
(8132992506844206601,8149858096109302285],
(3975126143101303723,3980729378827590597],
(-956691623200349709,-946602525018301692],
(-82499927325251331,-79866884352549492],
(3952144214544622998,3955602392726495936],
(8154760186218662205,8157079055586089583],
(3840595196718778916,3866458971850198755],
(-1066905024007783341,-1055954824488508260],
(-7252356975874511782,-7246799942440641887],
(-810612946397276081,-792189809286829222],
(4964519403172053705,4970446606512414858],
(-5380038118840759647,-5370953856683870319],
(-3221630728515706463,-3206856875356976885],
(-1193448110686154165,-1161640137086921883],
(-3356304907368646189,-3346460884208327912],
(3466596314109623830,3468144326691723333],
(-9050241313548454460,-9005441616028750657],
(402227699082311580,407458511300218383]]] Validation failed in /FAKE.33.64
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_102]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_102]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
~[apache-cassandra-3.0.14.jar:3.0.14]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_102]
Caused by: org.apache.cassandra.exceptions.RepairException: [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0 on XXX/YYY,
[(8195393703879512303,8196334842725538685],
(8166975326273137878,8182604850967732931],
(-7246799942440641887,-7227869626613009045],
(-8371707510273823988,-8365977215604569699],
(-141862581573028594,-140310864869418908],
(3732113975108886193,3743105867152786342],
(4998127507903069087,5008922734235607550],
(-5115827291264930140,-5111054924035590372],
(-2475342271852943287,-2447285553369030332],
(-8318606053827235336,-8308721754886697230],
(-5208900659917654871,-5202385837264015269],
(6618737991399272130,6623100721269775102],
(-4650650128572424858,-4650260492494258461],
(1886545362164970333,1886646959491599822],
(-4511817721998311568,-4507491187192881115],
(8114903118676615937,8132992506844206601],
(6224957219376301858,6304379125732293904],
(-3460547504877234383,-3459262416082517136],
(-167838948111369123,-141862581573028594],
(481579232521229473,491242114841289497],
(4052464144722307684,4059745901618136723],
(1659668187498418295,1679582585970705122],
(-1118922763210109192,-1093766915505652874],
(7504365235878319341,7526188885210185292],
(-79866884352549492,-77667207866300333],
(8151204058820798561,8154760186218662205],
(-1040398370287131739,-1033770179677543189],
(3767057277953758442,3783780844370292025],
(-6491678058233994892,-6487797181789288329],
(-916868210769480248,-907141794196269524],
(-9005441616028750657,-9002220258513351832],
(8183526518331102304,8186908810225025483],
(-5685737903527826627,-5672136154194382932],
(4976122621177738811,4987871287137312689],
(6051670147160447042,6051686987147911650],
(-1161640137086921883,-1159172734746043158],
(6895951547735922309,6899152466544114890],
(-3357667382515377172,-3356304907368646189],
(-5370953856683870319,-5345971445444542485],
(3824272999898372667,3829315045986248983],
(8132992506844206601,8149858096109302285],
(3975126143101303723,3980729378827590597],
(-956691623200349709,-946602525018301692],
(-82499927325251331,-79866884352549492],
(3952144214544622998,3955602392726495936],
(8154760186218662205,8157079055586089583],
(3840595196718778916,3866458971850198755],
(-1066905024007783341,-1055954824488508260],
(-7252356975874511782,-7246799942440641887],
(-810612946397276081,-792189809286829222],
(4964519403172053705,4970446606512414858],
(-5380038118840759647,-5370953856683870319],
(-3221630728515706463,-3206856875356976885],
(-1193448110686154165,-1161640137086921883],
(-3356304907368646189,-3346460884208327912],
(3466596314109623830,3468144326691723333],
(-9050241313548454460,-9005441616028750657],
(402227699082311580,407458511300218383]]] Validation failed in /FAKE.33.64
at
org.apache.cassandra.repair.ValidationTask.treesReceived(ValidationTask.java:68)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:178)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:486)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_102]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_102]
... 4 common frames omitted
WARN [RepairJobTask:2] 2017-09-15 03:00:29,493 RepairJob.java:153 - [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0] YYY sync failed
ERROR [RepairJobTask:2] 2017-09-15 03:00:29,498 RepairSession.java:277 -
[repair #071f81e0-99c2-11e7-91dc-6132f5fe5fb0] Session completed with the
following error
org.apache.cassandra.exceptions.RepairException: [repair
#071f81e0-99c2-11e7-91dc-6132f5fe5fb0 on XXX/YYY,
[(8195393703879512303,8196334842725538685],
(8166975326273137878,8182604850967732931],
(-7246799942440641887,-7227869626613009045],
(-8371707510273823988,-8365977215604569699],
(-141862581573028594,-140310864869418908],
(3732113975108886193,3743105867152786342],
(4998127507903069087,5008922734235607550],
(-5115827291264930140,-5111054924035590372],
(-2475342271852943287,-2447285553369030332],
(-8318606053827235336,-8308721754886697230],
(-5208900659917654871,-5202385837264015269],
(6618737991399272130,6623100721269775102],
(-4650650128572424858,-4650260492494258461],
(1886545362164970333,1886646959491599822],
(-4511817721998311568,-4507491187192881115],
(8114903118676615937,8132992506844206601],
(6224957219376301858,6304379125732293904],
(-3460547504877234383,-3459262416082517136],
(-167838948111369123,-141862581573028594],
(481579232521229473,491242114841289497],
(4052464144722307684,4059745901618136723],
(1659668187498418295,1679582585970705122],
(-1118922763210109192,-1093766915505652874],
(7504365235878319341,7526188885210185292],
(-79866884352549492,-77667207866300333],
(8151204058820798561,8154760186218662205],
(-1040398370287131739,-1033770179677543189],
(3767057277953758442,3783780844370292025],
(-6491678058233994892,-6487797181789288329],
(-916868210769480248,-907141794196269524],
(-9005441616028750657,-9002220258513351832],
(8183526518331102304,8186908810225025483],
(-5685737903527826627,-5672136154194382932],
(4976122621177738811,4987871287137312689],
(6051670147160447042,6051686987147911650],
(-1161640137086921883,-1159172734746043158],
(6895951547735922309,6899152466544114890],
(-3357667382515377172,-3356304907368646189],
(-5370953856683870319,-5345971445444542485],
(3824272999898372667,3829315045986248983],
(8132992506844206601,8149858096109302285],
(3975126143101303723,3980729378827590597],
(-956691623200349709,-946602525018301692],
(-82499927325251331,-79866884352549492],
(3952144214544622998,3955602392726495936],
(8154760186218662205,8157079055586089583],
(3840595196718778916,3866458971850198755],
(-1066905024007783341,-1055954824488508260],
(-7252356975874511782,-7246799942440641887],
(-810612946397276081,-792189809286829222],
(4964519403172053705,4970446606512414858],
(-5380038118840759647,-5370953856683870319],
(-3221630728515706463,-3206856875356976885],
(-1193448110686154165,-1161640137086921883],
(-3356304907368646189,-3346460884208327912],
(3466596314109623830,3468144326691723333],
(-9050241313548454460,-9005441616028750657],
(402227699082311580,407458511300218383]]] Validation failed in /FAKE.33.64
at
org.apache.cassandra.repair.ValidationTask.treesReceived(ValidationTask.java:68)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:178)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:486)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
~[apache-cassandra-3.0.14.jar:3.0.14]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_102]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_102]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_102]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_102]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.0.14.jar:3.0.14]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_102]
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose it
to anyone else. If you received it in error please notify us immediately and
then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a
company registered in Linz whose registered office is at 4040 Linz, Austria,
Freist?dterstra?e 313
