Running Origin 3.6 on AWS, we’ve found that if our EC2 instances go down for
any length of time and come back up (as opposed to the EC2 instance getting
terminated) the nodes are automatically deleted from openshift then
re-registered after the ec2 instance is restarted. The activity is logged in
/var/log/messages
Dec 20 21:59:30 ip-172-21-21-30 origin-master-controllers: I1220
21:59:30.297638 26242 nodecontroller.go:761] Deleting node (no longer present
in cloud provider): ip-172-21-20-30.ec2.internal
Dec 20 21:59:30 ip-172-21-21-30 origin-master-controllers: I1220
21:59:30.297662 26242 controller_utils.go:273] Recording Deleting Node
ip-172-21-20-30.ec2.internal because it's not present according to cloud
provider event message for node ip-172-21-20-30.ec2.internal
Dec 20 21:59:30 ip-172-21-21-30 origin-master-controllers: I1220
21:59:30.297895 26242 event.go:217] Event(v1.ObjectReference{Kind:"Node",
Namespace:"", Name:"ip-172-21-20-30.ec2.internal",
UID:"36c8dca4-e5c9-11e7-b2ce-0e69b80c212e", APIVersion:"", ResourceVersion:"",
FieldPath:""}): type: 'Normal' reason: 'DeletingNode' Node
ip-172-21-20-30.ec2.internal event: Deleting Node ip-172-21-20-30.ec2.internal
because it's not present according to cloud provider
Dec 20 23:10:06 ip-172-21-21-30 origin-master-controllers: I1220
23:10:06.303567 26242 nodecontroller.go:616] NodeController observed a new
Node: "ip-172-21-22-30.ec2.internal"
Dec 20 23:10:06 ip-172-21-21-30 origin-master-controllers: I1220
23:10:06.303597 26242 controller_utils.go:273] Recording Registered Node
ip-172-21-22-30.ec2.internal in NodeController event message for node
ip-172-21-22-30.ec2.internal
Dec 20 23:10:06 ip-172-21-21-30 origin-master-controllers: I1220
23:10:06.303899 26242 event.go:217] Event(v1.ObjectReference{Kind:"Node",
Namespace:"", Name:"ip-172-21-22-30.ec2.internal",
UID:"e850129f-e5da-11e7-ac5e-027542a418ee", APIVersion:"", ResourceVersion:"",
FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node
ip-172-21-22-30.ec2.internal event: Registered Node
ip-172-21-22-30.ec2.internal in NodeController
The issue we are running into is that when the nodes come back they don’t have
all of our labels on them. They don’t get labelled to run the fluentd pods
“logging-infra-fluentd=true” and my masters aren’t set for “Scheduling
Disabled”.
Can anybody point me to the any doc regarding the automatic registration of the
node from the cloudprovider or knows how to adjust the behavior when a node is
re-registered so they can be tagged properly.
Thanks
Chad
########################################################################
The information contained in this message, and any attachments thereto,
is intended solely for the use of the addressee(s) and may contain
confidential and/or privileged material. Any review, retransmission,
dissemination, copying, or other use of the transmitted information is
prohibited. If you received this in error, please contact the sender
and delete the material from any computer. UNIGROUP.COM
########################################################################
_______________________________________________
users mailing list
[email protected]
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
