https://bugzilla.wikimedia.org/show_bug.cgi?id=66953
Bug ID: 66953
Summary: JobQueueRedis locking up and throwing errors and
exceptions
Product: MediaWiki
Version: 1.23.0
Hardware: All
OS: All
Status: NEW
Severity: normal
Priority: Unprioritized
Component: JobQueue
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]
Web browser: ---
Mobile Platform: ---
I recently moved my job queue into Redis. I run my job queue only via cron, and
the runJobs.php script has started to occasionally lockup on me and require a
manual kill. I run a wiki farm and I've seen it lockup on random wikis in the
farm, including wikis that have no edit activity at all.
I’m attaching the logs for one of the wikis that has no edit activity. The only
jobs running are "34 periodic queue task(s)". You can see a number of
unserialize errors as well an exception. I cannot determine yet what is causing
the problem.
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
2014-06-22 12:24:02 Executed 34 periodic queue task(s).
[a1c261e9] [no req] Exception from line 849 of
/srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php: Redis
server error: protocol error, got 'a' as reply type byte
Backtrace:
#0 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php(608):
JobQueueRedis->throwRedisException(RedisConnRef, RedisException)
#1 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php(589):
JobQueueRedis->doGetSiblingQueueSizes(array)
#2 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueue.php(675):
JobQueueRedis->doGetSiblingQueuesWithJobs(array)
#3 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueGroup.php(290):
JobQueue->getSiblingQueuesWithJobs(array)
#4 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueGroup.php(169):
JobQueueGroup->getQueuesWithJobs()
#5 /srv/www/mediawiki/public_html/w/maintenance/runJobs.php(91):
JobQueueGroup->pop(integer, integer, array)
#6 /srv/www/mediawiki/public_html/w/maintenance/doMaintenance.php(104):
RunJobs->execute()
#7 /srv/www/mediawiki/public_html/w/maintenance/runJobs.php(273):
require_once(string)
#8 {main}
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
2014-06-22 12:24:02 Executed 34 periodic queue task(s).
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
2014-06-22 12:26:02 Executed 34 periodic queue task(s).
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
2014-06-22 12:26:02 Executed 34 periodic queue task(s).
PHP Notice: Redis::evalsha(): send of 528 bytes failed with errno=104
Connection reset by peer in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 422
PHP Notice: Redis::lSize(): send of 82 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 395
PHP Notice: Redis::evalsha(): send of 474 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 422
PHP Notice: Redis::lSize(): send of 73 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 395
PHP Notice: Redis::evalsha(): send of 540 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 422
PHP Notice: Redis::lSize(): send of 84 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 395
PHP Notice: Redis::evalsha(): send of 486 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 422
PHP Notice: Redis::lSize(): send of 75 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 395
PHP Notice: Redis::exec(): send of 2775 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 395
PHP Notice: Redis::get(): send of 69 bytes failed with errno=32 Broken pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: Redis::setnx(): send of 2686 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: Redis::get(): send of 69 bytes failed with errno=32 Broken pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: Redis::setnx(): send of 2686 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: Redis::get(): send of 69 bytes failed with errno=32 Broken pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: Redis::setnx(): send of 2686 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
PHP Notice: Redis::get(): send of 69 bytes failed with errno=32 Broken pipe in
/srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php on
line 395
[c471ff2e] [no req] Exception from line 849 of
/srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php: Redis
server error: protocol error, got 'o' as reply type byte
Backtrace:
#0 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php(608):
JobQueueRedis->throwRedisException(RedisConnRef, RedisException)
#1 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueRedis.php(589):
JobQueueRedis->doGetSiblingQueueSizes(array)
#2 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueue.php(675):
JobQueueRedis->doGetSiblingQueuesWithJobs(array)
#3 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueGroup.php(290):
JobQueue->getSiblingQueuesWithJobs(array)
#4 /srv/www/mediawiki/public_html/w/includes/jobqueue/JobQueueGroup.php(169):
JobQueueGroup->getQueuesWithJobs()
#5 /srv/www/mediawiki/public_html/w/maintenance/runJobs.php(91):
JobQueueGroup->pop(integer, integer, array)
#6 /srv/www/mediawiki/public_html/w/maintenance/doMaintenance.php(104):
RunJobs->execute()
#7 /srv/www/mediawiki/public_html/w/maintenance/runJobs.php(273):
require_once(string)
#8 {main}
PHP Notice: Unknown: send of 4 bytes failed with errno=32 Broken pipe in
Unknown on line 0
PHP Notice: RedisBagOStuff::get(): send of 4 bytes failed with errno=32 Broken
pipe in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 0
2014-06-22 12:26:02 Executed 34 periodic queue task(s).
PHP Notice: Redis::close(): send of 4 bytes failed with errno=32 Broken pipe
in /srv/www/mediawiki/public_html/w/includes/clientpool/RedisConnectionPool.php
on line 341
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
PHP Notice: unserialize(): Error at offset 0 of 2 bytes in
/srv/www/mediawiki/public_html/w/includes/objectcache/RedisBagOStuff.php on
line 284
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
