| Mathew.onipe created this task. Mathew.onipe added projects: Wikidata-Query-Service, Discovery-Search (Current work), Operations. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. |
At about 1:10 UTC, We started seeing SSH errors on wdqs* servers. @Dzahn reported this error and tries to SSH into the servers but couldn't. Then we started getting a lot of wdqs-updater errors like: java.time.format.DateTimeParseException
@Dzahn power-cycled wdqs2005 and we could login into wdqs2005 afterwards. We discovered on wdqs2005, blazegraph seems to be working fine but wdqs-updater errors continued. This same issue was seen in all wdqs servers. Also, I tried to revert back some updates to solve these problems via the deploy host, but didn't get anywhere with this. I logged in to the autodeployment host (wdqs1009) to see to try and revert back update using git rebase but this did not stop the errors.
Then through all this, wdqs for users seems to be working fine (but obviously they are not getting updates as updater has failed). I will let people smarter than me jump on this when they are available.
Cc: Gehel, Smalyshev, Mathew.onipe, Aklapper, Dzahn, Nandana, thifranc, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
