>From log of POD
>k8s_controllers_master-controllers-host-10-1-241-74_kube-system, it seemed
>that master config wasn't compatible with images pullled so that etcd wasn't
>started.
The detailed log is as below.
E1030 14:26:51.590478 1 helpers.go:134] Encountered config error json:
unknown field "masterCount" in object *config.MasterConfig, raw JSON:
{"admissionConfig":{"pluginConfig":{"BuildDefaults":{"configuration":{"apiVersion":"v1","env":[],"kind":"BuildDefaultsConfig","resources":{"limits":{},"requests":{}}}},"BuildOverrides":{"configuration":{"apiVersion":"v1","kind":"BuildOverridesConfig"}},"openshift.io/ImagePolicy":{"configuration":{"apiVersion":"v1","executionRules":[{"matchImageAnnotations":[{"key":"images.openshift.io/deny-execution","value":"true"}],"name":"execution-denied","onResources":[{"resource":"pods"},{"resource":"builds"}],"reject":true,"skipOnResolutionFailure":true}],"kind":"ImagePolicyConfig"}}}},"aggregatorConfig":{"proxyClientInfo":{"certFile":"aggregator-front-proxy.crt","keyFile":"aggregator-front-proxy.key"}},"apiLevels":["v1"],"apiVersion":"v1","authConfig":{"requestHeader":{"clientCA":"front-proxy-ca.crt","clientCommonNames":["aggregator-front-proxy"],"extraHeaderPrefixes":["X-Remote-Extra-"],"groupHeaders":["X-Remote-Group"],"usernameHeaders":["X-Remote-User"]}},"controllerConfig":{"election":{"lockName":"openshift-master-controllers"},"serviceServingCert":{"signer":{"certFile":"service-signer.crt","keyFile":"service-signer.key"}}},"controllers":"*","corsAllowedOrigins":["(?i)//127\\.0\\.0\\.1(:|\\z)","(?i)//localhost(:|\\z)","(?i)//10\\.1\\.241\\.74(:|\\z)","(?i)//kubernetes\\.default(:|\\z)","(?i)//kubernetes\\.default\\.svc\\.cluster\\.local(:|\\z)","(?i)//kubernetes(:|\\z)","(?i)//openshift\\.default(:|\\z)","(?i)//openshift\\.default\\.svc(:|\\z)","(?i)//172\\.30\\.0\\.1(:|\\z)","(?i)//paas\\-dev\\.dataos\\.io(:|\\z)","(?i)//openshift\\.default\\.svc\\.cluster\\.local(:|\\z)","(?i)//host\\-10\\-1\\-241\\-74(:|\\z)","(?i)//kubernetes\\.default\\.svc(:|\\z)","(?i)//openshift(:|\\z)"],"dnsConfig":{"bindAddress":"0.0.0.0:8053","bindNetwork":"tcp4"},"etcdClientInfo":{"ca":"master.etcd-ca.crt","certFile":"master.etcd-client.crt","keyFile":"master.etcd-client.key","urls":["https://host-10-1-241-74:2379";<https://host-10-1-241-74:2379>]},"etcdStorageConfig":{"kubernetesStoragePrefix":"kubernetes.io","kubernetesStorageVersion":"v1","openShiftStoragePrefix":"openshift.io","openShiftStorageVersion":"v1"},"imageConfig":{"format":"docker.io/openshift/origin-${component}:${version}","latest":false},"imagePolicyConfig":{"internalRegistryHostname":"docker-registry.default.svc:5000"},"kind":"MasterConfig","kubeletClientInfo":{"ca":"ca-bundle.crt","certFile":"master.kubelet-client.crt","keyFile":"master.kubelet-client.key","port":10250},"kubernetesMasterConfig":{"apiServerArguments":{"storage-backend":["etcd3"],"storage-media-type":["application/vnd.kubernetes.protobuf"]},"controllerArguments":{"cluster-signing-cert-file":["/etc/origin/master/ca.crt"],"cluster-signing-key-file":["/etc/origin/master/ca.key"],"pv-recycler-pod-template-filepath-hostpath":["/etc/origin/master/recycler_pod.yaml"],"pv-recycler-pod-template-filepath-nfs":["/etc/origin/master/recycler_pod.yaml"]},"masterCount":1,"masterIP":"10.1.241.74","podEvictionTimeout":null,"proxyClientInfo":{"certFile":"master.proxy-client.crt","keyFile":"master.proxy-client.key"},"schedulerArguments":null,"schedulerConfigFile":"/etc/origin/master/scheduler.json","servicesNodePortRange":"","servicesSubnet":"172.30.0.0/16","staticNodeNames":[]},"masterClients":{"externalKubernetesClientConnectionOverrides":{"acceptContentTypes":"application/vnd.kubernetes.protobuf,application/json","burst":400,"contentType":"application/vnd.kubernetes.protobuf","qps":200},"externalKubernetesKubeConfig":"","openshiftLoopbackClientConnectionOverrides":{"acceptContentTypes":"application/vnd.kubernetes.protobuf,application/json","burst":600,"contentType":"application/vnd.kubernetes.protobuf","qps":300},"openshiftLoopbackKubeConfig":"openshift-master.kubeconfig"},"masterPublicURL":"https://paas-dev.dataos.io:8443";<https://paas-dev.dataos.io:8443>,"networkConfig":{"clusterNetworks":[{"cidr":"10.128.0.0/14","hostSubnetLength":9}],"externalIPNetworkCIDRs":["0.0.0.0/0"],"networkPluginName":"redhat/openshift-ovs-networkpolicy","serviceNetworkCIDR":"172.30.0.0/16"},"oauthConfig":{"assetPublicURL":"https://paas-dev.dataos.io:8443/console/";<https://paas-dev.dataos.io:8443/console/>,"grantConfig":{"method":"auto"},"identityProviders":[{"challenge":true,"login":true,"mappingMethod":"claim","name":"allow_all","provider":{"apiVersion":"v1","kind":"AllowAllPasswordIdentityProvider"}}],"masterCA":"ca-bundle.crt","masterPublicURL":"https://paas-dev.dataos.io:8443";<https://paas-dev.dataos.io:8443>,"masterURL":"https://paas-dev.dataos.io:8443";<https://paas-dev.dataos.io:8443>,"sessionConfig":{"sessionMaxAgeSeconds":3600,"sessionName":"ssn","sessionSecretsFile":"/etc/origin/master/session-secrets.yaml"},"tokenConfig":{"accessTokenMaxAgeSeconds":86400,"authorizeTokenMaxAgeSeconds":500}},"pauseControllers":false,"policyConfig":{"bootstrapPolicyFile":"/etc/origin/master/policy.json","openshiftInfrastructureNamespace":"openshift-infra","openshiftSharedResourcesNamespace":"openshift"},"projectConfig":{"defaultNodeSelector":"node-role.kubernetes.io/compute=true","projectRequestMessage":"","projectRequestTemplate":"","securityAllocator":{"mcsAllocatorRange":"s0:/2","mcsLabelsPerProject":5,"uidAllocatorRange":"1000000000-1999999999/10000"}},"routingConfig":{"subdomain":"paas-dev.dataos.io"},"serviceAccountConfig":{"limitSecretReferences":false,"managedNames":["default","builder","deployer"],"masterCA":"ca-bundle.crt","privateKeyFile":"serviceaccounts.private.key","publicKeyFiles":["serviceaccounts.public.key"]},"servingInfo":{"bindAddress":"0.0.0.0:8443","bindNetwork":"tcp4","certFile":"master.server.crt","clientCA":"ca.crt","keyFile":"master.server.key","maxRequestsInFlight":500,"requestTimeoutSeconds":3600},"volumeConfig":{"dynamicProvisioningEnabled":true}}
I1030 14:26:51.597367 1 plugins.go:84] Registered admission plugin
"NamespaceLifecycle"
I1030 14:26:51.597442 1 plugins.go:84] Registered admission plugin
"Initializers"
I1030 14:26:51.597504 1 plugins.go:84] Registered admission plugin
"ValidatingAdmissionWebhook"
I1030 14:26:51.597566 1 plugins.go:84] Registered admission plugin
"MutatingAdmissionWebhook"
I1030 14:26:51.597628 1 plugins.go:84] Registered admission plugin
"AlwaysAdmit"
I1030 14:26:51.597674 1 plugins.go:84] Registered admission plugin
"AlwaysPullImages"
I1030 14:26:51.597699 1 plugins.go:84] Registered admission plugin
"LimitPodHardAntiAffinityTopology"
I1030 14:26:51.597727 1 plugins.go:84] Registered admission plugin
"DefaultTolerationSeconds"
I1030 14:26:51.597815 1 plugins.go:84] Registered admission plugin
"AlwaysDeny"
I1030 14:26:51.597868 1 plugins.go:84] Registered admission plugin
"EventRateLimit"
I1030 14:26:51.597895 1 plugins.go:84] Registered admission plugin
"DenyEscalatingExec"
I1030 14:26:51.597912 1 plugins.go:84] Registered admission plugin
"DenyExecOnPrivileged"
I1030 14:26:51.597931 1 plugins.go:84] Registered admission plugin
"ExtendedResourceToleration"
I1030 14:26:51.597959 1 plugins.go:84] Registered admission plugin
"OwnerReferencesPermissionEnforcement"
I1030 14:26:51.597989 1 plugins.go:84] Registered admission plugin
"ImagePolicyWebhook"
I1030 14:26:51.598008 1 plugins.go:84] Registered admission plugin
"LimitRanger"
I1030 14:26:51.598041 1 plugins.go:84] Registered admission plugin
"NamespaceAutoProvision"
I1030 14:26:51.598062 1 plugins.go:84] Registered admission plugin
"NamespaceExists"
I1030 14:26:51.598084 1 plugins.go:84] Registered admission plugin
"NodeRestriction"
I1030 14:26:51.598111 1 plugins.go:84] Registered admission plugin
"PersistentVolumeLabel"
I1030 14:26:51.598135 1 plugins.go:84] Registered admission plugin
"PodNodeSelector"
I1030 14:26:51.598161 1 plugins.go:84] Registered admission plugin
"PodPreset"
I1030 14:26:51.598181 1 plugins.go:84] Registered admission plugin
"PodTolerationRestriction"
I1030 14:26:51.598200 1 plugins.go:84] Registered admission plugin
"ResourceQuota"
I1030 14:26:51.598228 1 plugins.go:84] Registered admission plugin
"PodSecurityPolicy"
I1030 14:26:51.598258 1 plugins.go:84] Registered admission plugin
"Priority"
I1030 14:26:51.598285 1 plugins.go:84] Registered admission plugin
"SecurityContextDeny"
I1030 14:26:51.598318 1 plugins.go:84] Registered admission plugin
"ServiceAccount"
I1030 14:26:51.598340 1 plugins.go:84] Registered admission plugin
"DefaultStorageClass"
I1030 14:26:51.598358 1 plugins.go:84] Registered admission plugin
"PersistentVolumeClaimResize"
I1030 14:26:51.598397 1 plugins.go:84] Registered admission plugin
"StorageObjectInUseProtection"
Invalid MasterConfig /etc/origin/master/master-config.yaml
etcdClientInfo.ca: Invalid value: "/etc/origin/master/master.etcd-ca.crt":
could not read file: stat /etc/origin/master/master.etcd-ca.crt: no such file
or directory
On 2018/10/30 17:23, Yu Wei wrote:
I met the same problem and found that etcd was skipped as below,
TASK [openshift_control_plane : Establish the default bootstrap kubeconfig for
masters] ******************************************************
changed: [host-10-1-241-74] => (item=/etc/origin/node/bootstrap.kubeconfig)
changed: [host-10-1-241-74] => (item=/etc/origin/node/node.kubeconfig)
TASK [openshift_control_plane : Check status of control plane image pre-pull]
****************************************************************
changed: [host-10-1-241-74]
TASK [openshift_control_plane : Check status of etcd image pre-pull]
*************************************************************************
skipping: [host-10-1-241-74]
TASK [openshift_control_plane : Start and enable self-hosting node]
**************************************************************************
changed: [host-10-1-241-74]
Is this playbooks issue?
Thanks,
Jared
Interested in big data, cloud computing
On 2018/10/30 15:47, [email protected]<mailto:[email protected]>
wrote:
Hello everyone
I am facing an issue with the installer for 3.10 (and 3.11 has the same problem)
It started around 2-3 weeks ago, since I wasnt able to run the Ansible
installer successfully...even when using a tag from 3.10 in the installer-repo
that worked before.
The control-plane is not starting and what I could figure out is, that etcd is
not started anywhere. The last time it was working, when running "docker ps" on
the master (single master, multi node system) I saw about 4 running
containers...one of them was the etcd.
Now, there are only 2 of them and no etcd anywhere.
https://github.com/lostiniceland/devops/tree/master/openshift<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flostiniceland%2Fdevops%2Ftree%2Fmaster%2Fopenshift&data=02%7C01%7C%7C4600687a2b3046a7ee3e08d63e496632%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636764882293088742&sdata=MLBiIBhjUfS4tACb2P8MqJ7WvG17zF2VfnIlR7%2FIgNQ%3D&reserved=0>
This is my current Vagrant-Setup which uses a simple script to check-out the
openshift-installer, prepare Vagrant and run the Ansible files.
I thought that I might have broken my inventory or script but I double checked
everything and I new that this setup was working before.
Now at work, the collegue who is maining our test-cluster has the same problem
when upgrading from 3.9 to 3.10...no etcd anywhere. It seems restarting the
docker-daemon fixes for our test-cluster.
If anyone could look into this would be very appreciated.
What I find odd is the fact that even a before working tag like
openshit-ansible-3.10.53-1 is now broken. The only reasons I can think of, is
the used Dockerimages have been updated or the installed Version of Docker is
somewhat broken.
best regards
Marc
_______________________________________________
users mailing list
[email protected]<mailto:[email protected]>
https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openshift.redhat.com%2Fopenshiftmm%2Flistinfo%2Fusers&data=02%7C01%7C%7Cc3b207c02bd24d56857008d63e3c1e7a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636764825256850099&sdata=FbRAeT9pwbbnhvmlO1WZ9cVHVbSuEKUOO9SBfTO83wk%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openshift.redhat.com%2Fopenshiftmm%2Flistinfo%2Fusers&data=02%7C01%7C%7C4600687a2b3046a7ee3e08d63e496632%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636764882293088742&sdata=zjv4PWqot7CM0Svwgst83JegpgTUX5GSI42hFiNVk9A%3D&reserved=0>
_______________________________________________
users mailing list
[email protected]<mailto:[email protected]>
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openshift.redhat.com%2Fopenshiftmm%2Flistinfo%2Fusers&data=02%7C01%7C%7C4600687a2b3046a7ee3e08d63e496632%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636764882293244988&sdata=AdV9y%2BewTgWX1%2F50UtU6sSaQsw5nFlpBsWGxCROwYgQ%3D&reserved=0
_______________________________________________
users mailing list
[email protected]
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
