Hi Raj, This has been caused by an incorrect addition of the instanceClusterId from the received JSON object. I've submitted the fix at PR[1]. Please review and merge.
[1] - https://github.com/apache/stratos/pull/248 Regards, Chamila de Alwis Software Engineer | WSO2 | +94772207163 Blog: code.chamiladealwis.com On Mon, Mar 2, 2015 at 2:45 PM, Chamila De Alwis <[email protected]> wrote: > Hi Raj, > > Yes, this is wrong. I'll correct this and submit a PR soon. > > > Regards, > Chamila de Alwis > Software Engineer | WSO2 | +94772207163 > Blog: code.chamiladealwis.com > > > > On Mon, Mar 2, 2015 at 1:47 PM, Rajkumar Rajaratnam <[email protected]> > wrote: > >> Hi Devs, >> >> I am experiencing $Subject in kubernetes setup with >> stratos/php:4.1.0-beta docker image (probably outdated agent). >> >> Reason for this issue is that member termination cycle is broken at >> python agent. >> >> Agent is failing to process Instance Cleanup Cluster Event, so agent >> won't send instance ready to shutdown event CC, CC won't send member ready >> shutdown event to AS, AS won't ask CC to terminate the instance. So >> monitors and members are running even after we undeploy the application. >> >> Here is the agent log. >> >> [2015-03-02 07:51:29,971] DEBUG {agent.py:on_application_signup_removed} >> - Application signup removed event received: >> '{"applicationId":"single-cartridge-app","tenantId":-1234}' >> [2015-03-02 07:51:29,971] ERROR {agent.py:on_application_signup_removed} >> - Error processing tenant unSubscribed event >> Traceback (most recent call last): >> File "agent.py", line 347, in on_application_signup_removed >> >> CartridgeAgent.extension_handler.on_application_signup_removal_event(event_obj) >> File >> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py", >> line 362, in on_application_signup_removal_event >> " [application ID] " + application_signup_removal_event.applicationId >> TypeError: cannot concatenate 'str' and 'int' objects >> [2015-03-02 07:51:30,002] DEBUG {eventsubscriber.py:on_message} - Message >> received: topology/ClusterInstanceTerminatingEvent: >> >> {"serviceName":"php","clusterId":"my-php.php.domain","appId":"single-cartridge-app","instanceId":"single-cartridge-app-1"} >> [2015-03-02 07:51:30,002] DEBUG {eventsubscriber.py:run} - Event handler >> not found for event : 'ClusterInstanceTerminatingEvent' >> [2015-03-02 07:51:30,013] DEBUG {eventsubscriber.py:on_message} - Message >> received: instance/notifier/InstanceCleanupClusterEvent: >> >> {"clusterId":"my-php.php.domain","clusterInstanceId":"single-cartridge-app-1"} >> [2015-03-02 07:51:30,014] DEBUG {eventsubscriber.py:run} - Executing >> handler for event 'InstanceCleanupClusterEvent' >> [2015-03-02 07:51:30,014] ERROR {eventsubscriber.py:run} - Error >> processing 'InstanceCleanupClusterEvent' event >> Traceback (most recent call last): >> File >> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py", >> line 103, in run >> handler(event_msg) >> File "agent.py", line 195, in on_instance_cleanup_cluster >> event_obj = InstanceCleanupClusterEvent.create_from_json(msg.payload) >> File >> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/instance/notifier/events.py", >> line 67, in create_from_json >> instance_cluster_id = json_obj["instanceClusterId"] if "clusterId" in >> json_obj else None >> KeyError: 'instanceClusterId' >> >> >> @Chamila, do you know the reason for this? >> >> -- >> Rajkumar Rajaratnam >> Committer & PMC Member, Apache Stratos >> Software Engineer, WSO2 >> >> Mobile : +94777568639 >> Blog : rajkumarr.com >> > >
