The newer kernel went about 16 runs and then popped this:

[ 2137.810559] md: md0: resync done.
[ 2296.795633] INFO: task python3:11639 blocked for more than 120 seconds.
[ 2296.800320]       Tainted: P           O     4.15.0-55-generic 
#60+lp1796292+1
[ 2296.805097] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[ 2296.810071] python3         D    0 11639   8301 0x00000000
[ 2296.810075] Call Trace:
[ 2296.810100]  __schedule+0x291/0x8a0
[ 2296.810102]  ? __switch_to_asm+0x34/0x70
[ 2296.810103]  ? __switch_to_asm+0x40/0x70
[ 2296.810105]  schedule+0x2c/0x80
[ 2296.810118]  bch_bucket_alloc+0xe5/0x370 [bcache]
[ 2296.810128]  ? wait_woken+0x80/0x80
[ 2296.810132]  __bch_bucket_alloc_set+0x10d/0x160 [bcache]
[ 2296.810137]  bch_bucket_alloc_set+0x4e/0x70 [bcache]
[ 2296.810143]  __uuid_write+0x59/0x170 [bcache]
[ 2296.810148]  ? __write_super+0x137/0x170 [bcache]
[ 2296.810153]  bch_uuid_write+0x16/0x40 [bcache]
[ 2296.810159]  __cached_dev_store+0x1e5/0x8b0 [bcache]
[ 2296.810160]  ? __switch_to_asm+0x34/0x70
[ 2296.810161]  ? __switch_to_asm+0x40/0x70
[ 2296.810163]  ? __switch_to_asm+0x34/0x70
[ 2296.810167]  bch_cached_dev_store+0x46/0x110 [bcache]
[ 2296.810181]  sysfs_kf_write+0x3c/0x50
[ 2296.810182]  kernfs_fop_write+0x125/0x1a0
[ 2296.810185]  __vfs_write+0x1b/0x40
[ 2296.810187]  vfs_write+0xb1/0x1a0
[ 2296.810189]  SyS_write+0x55/0xc0
[ 2296.810193]  do_syscall_64+0x73/0x130
[ 2296.810194]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[ 2296.810196] RIP: 0033:0x7f8e80077154
[ 2296.810197] RSP: 002b:00007fffc23855e8 EFLAGS: 00000246 ORIG_RAX: 
0000000000000001
[ 2296.810199] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8e80077154
[ 2296.810200] RDX: 0000000000000008 RSI: 0000000000d734b0 RDI: 0000000000000003
[ 2296.810201] RBP: 00007f8e805726c0 R08: 0000000000000000 R09: 0000000000000000
[ 2296.810202] R10: 0000000000000100 R11: 0000000000000246 R12: 0000000000000003
[ 2296.810203] R13: 0000000000000000 R14: 0000000000d734b0 R15: 0000000000aa4db0
[ 2417.627259] INFO: task python3:11639 blocked for more than 120 seconds.
[ 2417.632687]       Tainted: P           O     4.15.0-55-generic 
#60+lp1796292+1
[ 2417.638276] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1796292

Title:
  Tight timeout for bcache removal causes spurious failures

Status in curtin:
  Fix Released
Status in linux package in Ubuntu:
  Confirmed
Status in linux source package in Bionic:
  Confirmed
Status in linux source package in Cosmic:
  Confirmed
Status in linux source package in Disco:
  Confirmed
Status in linux source package in Eoan:
  Confirmed

Bug description:
  I've had a number of deployment faults where curtin would report
  Timeout exceeded for removal of /sys/fs/bcache/xxx when doing a mass-
  deployment of 30+ nodes. Upon retrying the node would usually deploy
  fine. Experimentally I've set the timeout ridiculously high, and it
  seems I'm getting no faults with this. I'm wondering if the timeout
  for removal is set too tight, or might need to be made configurable.

  --- curtin/util.py~     2018-05-18 18:40:48.000000000 +0000
  +++ curtin/util.py      2018-10-05 09:40:06.807390367 +0000
  @@ -263,7 +263,7 @@
       return _subp(*args, **kwargs)
   
   
  -def wait_for_removal(path, retries=[1, 3, 5, 7]):
  +def wait_for_removal(path, retries=[1, 3, 5, 7, 1200, 1200]):
       if not path:
           raise ValueError('wait_for_removal: missing path parameter')

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1796292/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to