[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
@Ryan,

Marking this as invalid for curtin again. I looked closely to the log
and saw this:

May 21 11:00:35 geodude cloud-init[1643]: --2018-05-21 11:00:35--  
http://10.244.40.33/MAAS/metadata/latest/by-id/gnbttp/
May 21 11:00:35 geodude cloud-init[1643]: Connecting to 10.244.40.33:80... 
connected.
May 21 11:00:35 geodude cloud-init[1643]: HTTP request sent, awaiting 
response... 200 OK
May 21 11:00:35 geodude cloud-init[1643]: Length: unspecified [text/plain]
May 21 11:00:35 geodude cloud-init[1643]: Saving to: ‘/dev/null’
May 21 11:00:35 geodude cloud-init[1643]:  0K   
  138K=0s
May 21 11:00:35 geodude cloud-init[1643]: 2018-05-21 11:00:35 (138 KB/s) - 
‘/dev/null’ saved [2]

That means curtin run the correct netboot_off command, which should have
told MAAS that the machine is to localboot on next reboot.

As such, I need the HAProxy logs to continue to be able to debug as it
was done against: 10.244.40.33:80

** Changed in: curtin
   Status: Incomplete => Invalid

** Summary changed:

- bcache: register_bcache() error
+ 'Deploying' timed out after 40 minutes / Failedbcache: register_bcache() error

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  'Deploying' timed out after 40 minutes / Failedbcache:
  register_bcache() error

Status in curtin:
  Invalid
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Re: [Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Ryan Harper 
On Mon, May 21, 2018 at 3:59 PM, Andres Rodriguez
 wrote:
> @Ryan,
>
> I'm marking this as incomplete for curtin provided that after further
> debugging, I can see that the late command that's supposed to send the
> "netboot_off" operation is not being sent.
>
> This could be because curtin failed but we are lacking logs to determine
> this.

What?

Late commands run before we report curtin installation success.

Do you have the actual curtin config sent?

Also, generally it would be good if the qa runs set curtin install to
verbose so more info is dumped into the rsyslog output.
In debug mode we dump the merge curtin config that's sent to curtin in syslog.

>
> ** Changed in: curtin
>Status: Invalid => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1772490
>
> Title:
>   bcache: register_bcache() error
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  'Deploying' timed out after 40 minutes / Failedbcache:
  register_bcache() error

Status in curtin:
  Invalid
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
@Ryan,

I'm marking this as incomplete for curtin provided that after further
debugging, I can see that the late command that's supposed to send the
"netboot_off" operation is not being sent.

This could be because curtin failed but we are lacking logs to determine
this.

** Changed in: curtin
   Status: Invalid => Incomplete

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Incomplete
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
@Ashley,

Could you please start gathering the logs from HAProxy running for the
MAAS servers?

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Incomplete
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Ryan Harper 
May 21 11:00:42 geodude cloud-init[1643]: curtin: Installation finished.

>From the rsyslog, curtin finished the install without error.

** Changed in: curtin
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Invalid
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Re: [Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Ryan Harper 
Thanks for the log.  Curtin installed without error, I'll mark invalid.

AFAICT, it booted fine and was instructed to power off.

May 21 13:18:51 geodude cloud-init[1676]: Powering node off.
May 21 13:18:51 geodude ec2:
May 21 13:18:51 geodude ec2:
#
May 21 13:18:51 geodude ec2: -BEGIN SSH HOST KEY FINGERPRINTS-
May 21 13:18:51 geodude ec2: 1024
SHA256:O2WkpYqVPV8G7pumPrb/sAi8F8pBY3ay3jF+Ymfko1Q root@geodude (DSA)
May 21 13:18:51 geodude ec2: 256
SHA256:KILySo9Cbqs70KPsyV16HZpWueeHqiBzOzPFSGxXl1M root@geodude
(ECDSA)
May 21 13:18:51 geodude ec2: 256
SHA256:C4clHtaNL6GpwIdlJwyZXq23NfbqK0s3YWzof0Eu7CY root@geodude
(ED25519)
May 21 13:18:51 geodude ec2: 2048
SHA256:LFGGivHhyNdrN5AXu5mj5eBENjk2tWNDj41K1VsP6Z0 root@geodude (RSA)
May 21 13:18:51 geodude ec2: -END SSH HOST KEY FINGERPRINTS-
May 21 13:18:51 geodude ec2:
#
May 21 13:18:51 geodude cloud-init[1676]: Cloud-init v. 18.2 running
'modules:final' at Mon, 21 May 2018 13:18:50 +. Up 27.71 seconds.
May 21 13:18:51 geodude cloud-init[1676]: Cloud-init v. 18.2 finished
at Mon, 21 May 2018 13:18:51 +. Datasource DataSourceMAAS
[http://10.244.40.33/MAAS/metadata/].  Up 28.40 seconds
May 21 13:18:51 geodude systemd[1]: Started Execute cloud user/final scripts.
May 21 13:18:51 geodude systemd[1]: Reached target Cloud-init target.
May 21 13:18:51 geodude systemd[1]: Startup finished in 15.521s
(kernel) + 13.137s (userspace) = 28.659s.


On Mon, May 21, 2018 at 3:15 PM, Andres Rodriguez
 wrote:
> Attached the rsyslog showing the error. It indeed doesn't seem like
> there were any curtin failures, but I wonder that, while curtin
> successfully process, the machine actually doesn't actually boot onto
> the filesystem due to the kernel issue?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1772490
>
> Title:
>   bcache: register_bcache() error
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Invalid
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
** Attachment added: "rsyslog-bcache-failure"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1772490/+attachment/5142552/+files/rsyslog-bcache-failure

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Incomplete
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
Attached the rsyslog showing the error. It indeed doesn't seem like
there were any curtin failures, but I wonder that, while curtin
successfully process, the machine actually doesn't actually boot onto
the filesystem due to the kernel issue?

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Incomplete
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Ryan Harper 
ay 21 10:57:05 geodude kernel: [ 49.126408] bcache: register_bcache()
error /dev/sda3: device already registered (emitting change event)

These are not curtin or kernel errors but expected output.

I looked at the qa link but I didn't find the install.log debug output.

** Changed in: curtin
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  Incomplete
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
Hi Ashley,

Marking this incomplete for MAAS (although I think it is invalid).
Opening a task for curtin and for the kernel team. The error in curtin
implies is the same issue as [1]. Judging from [2], it seems that it
should already be fixed:

May 21 10:57:03 geodude cloud-init[1643]: Processing triggers for libc-bin 
(2.27-3ubuntu1) ...
May 21 10:57:04 geodude cloud-init[1643]: curtin: Installation started. 
(18.1-623-gae48e86-0ubuntu1~ubuntu16.04.1)
May 21 10:57:04 geodude cloud-init[1643]: third party drivers not installed or 
necessary.
May 21 10:57:05 geodude kernel: [   49.126408] bcache: register_bcache() error 
/dev/sda3: device already registered (emitting change event)
May 21 10:57:05 geodude kernel: [   49.166935] bcache: register_bcache() error 
/dev/sda3: device already registered (emitting change event)
May 21 10:57:05 geodude kernel: [   49.209233] bcache: register_bcache() error 
/dev/sda3: device already registered (emitting change event)
May 21 10:57:05 geodude kernel: [   49.254763] bcache: register_bcache() error 
/dev/sda3: device already registered (emitting change event)
May 21 10:57:05 geodude kernel: [   49.319986] bcache: register_bcache() error 
/dev/sda3: device already registered (emitting change event)


The ephemeral environment kernel seems to be:

May 21 10:56:43 geodude kernel: [0.00] Linux version
4.15.0-20-generic (buildd@lgw01-amd64-039) (gcc version 7.3.0 (Ubuntu
7.3.0-16ubuntu3)) #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 (Ubuntu
4.15.0-20.21-generic 4.15.17)


[1]: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1729145
[2]: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1729145/comments/54

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  New
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

[Kernel-packages] [Bug 1772490] Re: bcache: register_bcache() error

2018-05-21 Thread Andres Rodriguez 
** Changed in: maas
   Status: Invalid => Incomplete

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1772490

Title:
  bcache: register_bcache() error

Status in curtin:
  New
Status in MAAS:
  Incomplete
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  We have a few runs over the weekend failed to deploy with maas 2.3.3.

  May 21 11:33:50 swoobat maas.node: [info] geodude: Status transition from 
DEPLOYING to FAILED_DEPLOYMENT
  May 21 11:33:50 swoobat maas.node: [error] geodude: Marking node failed: Node 
operation 'Deploying' timed out after 40 minutes.

  https://solutions.qa.canonical.com/#/qa/testRun/67dae845-b22e-
  4de1-9b30-0ecb28eb3c35

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/1772490/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp