Re: [openstack-dev] [magnum] magnum cluster-create for kubernetes-template was failed.

2017-05-12 Thread Mark Goddard 
Hi,

I also hit the loopingcall error while running magnum 4.1.1 (ocata). It is
tracked by this bug: https://bugs.launchpad.net/magnum/+bug/1666790. I
cherry picked the fix to ocata locally, but this needs to be done upstream
as well.

I think that the heat stack create timeout is unrelated to that issue
though. Try the following to debug the issue:
- Check the cluster's heat stack and its component resources.
- If created, SSH to the master and slave nodes, checking systemd services
are up and cloud-init succeeded.

Regards,
Mark

On 12 May 2017 at 05:57, KiYoun Sung  wrote:

> Hello,
> Magnum Team.
>
> I installed magnum on Openstack Ocata(by fuel 11.0).
> I referred to this guide.(https://docs.openstack.
> org/project-install-guide/container-infrastructure-managemen
> t/ocata/install.html)
>
> Below is my installation information.
> root@controller:~# dpkg -l | grep magnum
> magnum-api  4.1.0-0ubuntu1~cloud0
>  all  OpenStack containers as a service
> magnum-common   4.1.0-0ubuntu1~cloud0
>  all  OpenStack containers as a service - API server
> magnum-conductor4.1.0-0ubuntu1~cloud0
>  all  OpenStack containers as a service - conductor
> python-magnum   4.1.0-0ubuntu1~cloud0
>  all  OpenStack containers as a service - Python library
> python-magnumclient 2.5.0-0ubuntu1~cloud0
>  all  client library for Magnum API - Python 2.x
>
> After installation,
> I created cluster-template for kubernetes like this.
> (magnum cluster-template-create --name k8s-cluster-template \ --image
> fedora-atomic-latest \ --keypair testkey \ --external-network
> admin_floating_net \ --dns-nameserver 8.8.8.8 \ --flavor m1.small \
> --docker-volume-size 5 \ --network-driver flannel \ --coe kubernetes )
>
> and I create cluster,
> but "magnum clutser-create' command was failed.
> (magnum cluster-create --name k8s-cluster \ --cluster-template
> k8s-cluster-template \ --node-count 1 \ --timeout 10 )
>
> After 10 minutes(option "--timeout 10"),
> creation was failed, and the status is "CREATE_FAILED"
>
> I executed "openstack server list" command,
> there is a only kube-master instance.
> (root@controller:~# openstack server list
> +--+
> ---++---
> -+--+
> | ID   | Name
>  | Status | Networks   | Image Name   |
> +--+
> ---++---
> -+--+
> | bf9c5097-74fd-4457-a8a2-4feae76d4111 | k8-i27fw72w5t-0-i6lg6mzpzrl6-kube-
>| ACTIVE | private=10.0.0.9, 172.16.1.135 | fedora-atomic-latest |
> |  | master-ekjrg2v6ztss
> |||  |
> +--+
> ---+++--+
>
> )
>
> I think kube-master instance create is successful.
> I can connect that instance
> and docker container was running normally.
>
> Why this command was failed?
>
> Here is my /var/log/magnum/magnum-conductor.log and /var/log/nova-all.log.
> magnum-conductor.log have a ERROR.
> ===
> 2017-05-12 04:05:00.684 756 ERROR magnum.common.keystone
> [req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Keystone API
> connection failed: no password, trust_id or token found.
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception
> [req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Exception in string
> format operation, kwargs: {'code': 500}
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception Traceback (most
> recent call last):
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception   File
> "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 92,
> in __init__
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception self.message
> = self.message % kwargs
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception KeyError:
> u'client'
> 2017-05-12 04:05:00.686 756 ERROR magnum.common.exception
> 2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall
> [req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Fixed interval
> looping call 'magnum.service.periodic.ClusterUpdateJob.update_status'
> failed
> 2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall Traceback (most
> recent call last):
> 2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall   File
> "/usr/lib/python2.7/dist-packages/oslo_service/loopingcall.py", line 137,
> in _run_loop
> 2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall result =
> func(*self.args, **self.kw)
> 2017-05-12 04:05:00.687 756 ERROR

[openstack-dev] [magnum] magnum cluster-create for kubernetes-template was failed.

2017-05-11 Thread KiYoun Sung 
Hello,
Magnum Team.

I installed magnum on Openstack Ocata(by fuel 11.0).
I referred to this guide.(https://docs.openstack.org/project-install-guide/
container-infrastructure-management/ocata/install.html)

Below is my installation information.
root@controller:~# dpkg -l | grep magnum
magnum-api  4.1.0-0ubuntu1~cloud0
   all  OpenStack containers as a service
magnum-common   4.1.0-0ubuntu1~cloud0
   all  OpenStack containers as a service - API server
magnum-conductor4.1.0-0ubuntu1~cloud0
   all  OpenStack containers as a service - conductor
python-magnum   4.1.0-0ubuntu1~cloud0
   all  OpenStack containers as a service - Python library
python-magnumclient 2.5.0-0ubuntu1~cloud0
   all  client library for Magnum API - Python 2.x

After installation,
I created cluster-template for kubernetes like this.
(magnum cluster-template-create --name k8s-cluster-template \ --image
fedora-atomic-latest \ --keypair testkey \ --external-network
admin_floating_net \ --dns-nameserver 8.8.8.8 \ --flavor m1.small \
--docker-volume-size 5 \ --network-driver flannel \ --coe kubernetes )

and I create cluster,
but "magnum clutser-create' command was failed.
(magnum cluster-create --name k8s-cluster \ --cluster-template
k8s-cluster-template \ --node-count 1 \ --timeout 10 )

After 10 minutes(option "--timeout 10"),
creation was failed, and the status is "CREATE_FAILED"

I executed "openstack server list" command,
there is a only kube-master instance.
(root@controller:~# openstack server list
+--+
---++---
-+--+
| ID   | Name
   | Status | Networks   | Image Name   |
+--+
---++---
-+--+
| bf9c5097-74fd-4457-a8a2-4feae76d4111 | k8-i27fw72w5t-0-i6lg6mzpzrl6-kube-
   | ACTIVE | private=10.0.0.9, 172.16.1.135 | fedora-atomic-latest |
|  | master-ekjrg2v6ztss
|||  |
+--+
---+++--+

)

I think kube-master instance create is successful.
I can connect that instance
and docker container was running normally.

Why this command was failed?

Here is my /var/log/magnum/magnum-conductor.log and /var/log/nova-all.log.
magnum-conductor.log have a ERROR.
===
2017-05-12 04:05:00.684 756 ERROR magnum.common.keystone
[req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Keystone API
connection failed: no password, trust_id or token found.
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception
[req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Exception in string
format operation, kwargs: {'code': 500}
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception Traceback (most
recent call last):
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception   File
"/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 92, in
__init__
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception self.message
= self.message % kwargs
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception KeyError:
u'client'
2017-05-12 04:05:00.686 756 ERROR magnum.common.exception
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall
[req-e2d4ea12-ec7a-4865-9eda-d272cc43a827 - - - - -] Fixed interval looping
call 'magnum.service.periodic.ClusterUpdateJob.update_status' failed
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall Traceback (most
recent call last):
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall   File
"/usr/lib/python2.7/dist-packages/oslo_service/loopingcall.py", line 137,
in _run_loop
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall result =
func(*self.args, **self.kw)
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall   File
"/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 71, in
update_status
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall
cdriver.update_cluster_status(self.ctx, self.cluster)
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall   File
"/usr/lib/python2.7/dist-packages/magnum/drivers/heat/driver.py", line 80,
in update_cluster_status
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall
poller.poll_and_check()
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall   File
"/usr/lib/python2.7/dist-packages/magnum/drivers/heat/driver.py", line 169,
in poll_and_check
2017-05-12 04:05:00.687 756 ERROR oslo.service.loopingcall stack =
self.openstack_client.heat().stacks.get(
2017-05-12 04:05:00.687 756 ERROR