Hi All,
I am currently testing on a ELB 211 fronted 1 manager/2 worker AS 521
cluster. Here when i am following below steps, i am getting below noted
errors on ELB stack traces.
*Steps:*
Setup a 1 manager/2 worker node ELB fronted setup
Invoke SimpleStockQuate service deployed on AS 521(http path) continuously
using jmeter script(with 1000 jmeter threads and 10 seconds ramp-up period)
Down the worker node using ./wso2server.sh stop command
Startup the downed worker node and check the ELB logs
*Issue:*
Intermittently observed following issue on ELB stack trace, but AS stack
traces does not show any thing except member left/member join messages
Note: I have tried above steps nearly 10 times and out of them 3 times
observed below error.[Attached ELB log for further reference]
TID: [0] [ELB] [2014-09-17 11:13:55,107] INFO
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} -
Member left [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:13:55,122] INFO
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} -
Member left [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:12,293] INFO
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} -
Member joined [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:12,297] INFO
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} -
Member joined [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:15,423] INFO
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} -
Application member Host:10.0.0.85, Remote Host:null, Port: 4300,
HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker,
Active:true joined application cluster
{org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:16,139] WARN
{org.apache.synapse.transport.passthru.TargetHandler} - Connection closed
by target host before receiving the request
{org.apache.synapse.transport.passthru.TargetHandler}
TID: [0] [ELB] [2014-09-17 11:16:16,169] INFO
{org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} -
Failed to send message to Member Host:10.0.0.85, Remote Host:null, Port:
4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker,
Active:true . Error Code: 101508
{org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 11:16:16,170] ERROR
{org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} -
Error sending request! Connection timeout after request is read by the
host as.wso2.com . Error code: 101508 Protocol State: RESPONSE HEAD
{org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
After analyzing AS and ELB stack-traces, i felt that AS worker nodes join
the cluster before it properly initialized. So when ELB handover the job to
AS worker nodes, It fails to do the job.
Does my observation make sense here or is there any other reason for this
behavior?
WDYT?
--
Asanka Vithanage
Senior Software Engineer -QA
Mobile: +94 0716286708
Email: [email protected]
WSO2 Inc. www.wso2.com
TID: [0] [ELB] [2014-09-17 10:46:24,517] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Starting WSO2 Carbon... {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Operating System : Linux 3.13.0-30-generic, amd64 {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Java Home : /usr/local/java/jdk1.7.0_67/jre {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Java Version : 1.7.0_67 {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Java VM : Java HotSpot(TM) 64-Bit Server VM 24.65-b04,Oracle Corporation {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Carbon Home : /home/ubuntu/ELBTest/wso2elb-2.1.1 {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,519] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - Java Temp Dir : /home/ubuntu/ELBTest/wso2elb-2.1.1/tmp {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,520] INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} - User : ubuntu, en-US, Etc/UTC {org.wso2.carbon.core.internal.CarbonCoreActivator}
TID: [0] [ELB] [2014-09-17 10:46:24,622] WARN {org.wso2.carbon.core.bootup.validator.SystemValidator} - Could not validate the system for configuration parameter : CPU {org.wso2.carbon.core.bootup.validator.SystemValidator}
TID: [0] [ELB] [2014-09-17 10:46:24,624] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - The default keystore (wso2carbon.jks) is currently being used. To maximize security when deploying to a production environment, configure a new keystore with a unique password in the production server profile. {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter}
TID: [0] [ELB] [2014-09-17 10:46:24,624] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - Swap Memory size (MB): 0 of the system is below the recommended minimum size :2048 {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter}
TID: [0] [ELB] [2014-09-17 10:46:24,633] INFO {org.wso2.carbon.databridge.agent.thrift.AgentHolder} - Agent created ! {org.wso2.carbon.databridge.agent.thrift.AgentHolder}
TID: [0] [ELB] [2014-09-17 10:46:24,666] INFO {org.wso2.carbon.databridge.agent.thrift.internal.AgentDS} - Successfully deployed Agent Client {org.wso2.carbon.databridge.agent.thrift.internal.AgentDS}
TID: [0] [ELB] [2014-09-17 10:46:26,109] INFO {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService} - Configured Registry in 62ms {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService}
TID: [0] [ELB] [2014-09-17 10:46:26,205] INFO {org.wso2.carbon.registry.core.internal.RegistryCoreServiceComponent} - Registry Mode : READ-WRITE {org.wso2.carbon.registry.core.internal.RegistryCoreServiceComponent}
TID: [0] [ELB] [2014-09-17 10:46:26,430] INFO {org.wso2.carbon.user.core.internal.UserStoreMgtDSComponent} - Carbon UserStoreMgtDSComponent activated successfully. {org.wso2.carbon.user.core.internal.UserStoreMgtDSComponent}
TID: [0] [ELB] [2014-09-17 10:46:31,616] INFO {org.apache.catalina.startup.TaglibUriRule} - TLD skipped. URI: http://tiles.apache.org/tags-tiles is already defined {org.apache.catalina.startup.TaglibUriRule}
TID: [0] [ELB] [2014-09-17 10:46:32,604] INFO {org.apache.axis2.deployment.ClusterBuilder} - Clustering has been enabled {org.apache.axis2.deployment.ClusterBuilder}
TID: [0] [ELB] [2014-09-17 10:46:32,610] INFO {org.apache.axis2.deployment.ClusterBuilder} - Clustering state management has been disabled {org.apache.axis2.deployment.ClusterBuilder}
TID: [0] [ELB] [2014-09-17 10:46:32,895] INFO {org.wso2.carbon.core.CarbonAxisConfigurator} - No modules directory was found under /home/ubuntu/ELBTest/wso2elb-2.1.1/repository/deployment/server. {org.wso2.carbon.core.CarbonAxisConfigurator}
TID: [0] [ELB] [2014-09-17 10:46:32,965] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender} - Initializing Pass-through HTTP/S Sender... {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender}
TID: [0] [ELB] [2014-09-17 10:46:32,985] INFO {org.apache.synapse.transport.nhttp.config.ClientConnFactoryBuilder} - HTTPS Loading Identity Keystore from : repository/resources/security/wso2carbon.jks {org.apache.synapse.transport.nhttp.config.ClientConnFactoryBuilder}
TID: [0] [ELB] [2014-09-17 10:46:32,992] INFO {org.apache.synapse.transport.nhttp.config.ClientConnFactoryBuilder} - HTTPS Loading Trust Keystore from : repository/resources/security/client-truststore.jks {org.apache.synapse.transport.nhttp.config.ClientConnFactoryBuilder}
TID: [0] [ELB] [2014-09-17 10:46:33,025] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender} - Pass-through HTTPS Sender started... {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender}
TID: [0] [ELB] [2014-09-17 10:46:33,025] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSender} - Initializing Pass-through HTTP/S Sender... {org.apache.synapse.transport.passthru.PassThroughHttpSender}
TID: [0] [ELB] [2014-09-17 10:46:33,032] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSender} - Pass-through HTTP Sender started... {org.apache.synapse.transport.passthru.PassThroughHttpSender}
TID: [0] [ELB] [2014-09-17 10:46:33,054] WARN {org.wso2.carbon.stratos.cloud.controller.axiom.AxiomXpathParser} - Unable to find a value for identity element from Secure Vault.Hence we will try to assign the plain text value (if specified). {org.wso2.carbon.stratos.cloud.controller.axiom.AxiomXpathParser}
TID: [0] [ELB] [2014-09-17 10:46:33,055] WARN {org.wso2.carbon.stratos.cloud.controller.axiom.AxiomXpathParser} - Unable to find a value for credential element from Secure Vault.Hence we will try to assign the plain text value (if specified). {org.wso2.carbon.stratos.cloud.controller.axiom.AxiomXpathParser}
TID: [0] [ELB] [2014-09-17 10:46:33,055] INFO {org.wso2.carbon.stratos.cloud.controller.deployers.CloudControllerDeployer} - Successfully deployed the cloud-controller XML file specified at /home/ubuntu/ELBTest/wso2elb-2.1.1/repository/deployment/server/../../conf/cloud-controller.xml {org.wso2.carbon.stratos.cloud.controller.deployers.CloudControllerDeployer}
TID: [0] [ELB] [2014-09-17 10:46:33,068] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener} - Initializing Pass-through HTTP/S Listener... {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener}
TID: [0] [ELB] [2014-09-17 10:46:33,092] INFO {org.apache.synapse.transport.passthru.PassThroughHttpListener} - Initializing Pass-through HTTP/S Listener... {org.apache.synapse.transport.passthru.PassThroughHttpListener}
TID: [0] [ELB] [2014-09-17 10:46:33,187] INFO {org.apache.axis2.deployment.ModuleDeployer} - Deploying module: addressing-1.6.1-wso2v10 - file:/home/ubuntu/ELBTest/wso2elb-2.1.1/repository/deployment/client/modules/addressing-1.6.1-wso2v10.mar {org.apache.axis2.deployment.ModuleDeployer}
TID: [0] [ELB] [2014-09-17 10:46:33,194] INFO {org.apache.axis2.deployment.ModuleDeployer} - Deploying module: rampart-1.6.1-wso2v1 - file:/home/ubuntu/ELBTest/wso2elb-2.1.1/repository/deployment/client/modules/rampart-1.6.1-wso2v1.mar {org.apache.axis2.deployment.ModuleDeployer}
TID: [0] [ELB] [2014-09-17 10:46:33,201] INFO {org.apache.axis2.transport.tcp.TCPTransportSender} - TCP Sender started {org.apache.axis2.transport.tcp.TCPTransportSender}
TID: [0] [ELB] [2014-09-17 10:46:33,830] INFO {org.apache.axis2.deployment.DeploymentEngine} - Deploying Web service: org.wso2.carbon.lb.common - {org.apache.axis2.deployment.DeploymentEngine}
TID: [0] [ELB] [2014-09-17 10:46:34,290] INFO {org.apache.axis2.deployment.DeploymentEngine} - Deploying Web service: org.wso2.carbon.stratos.cloud.controller - {org.apache.axis2.deployment.DeploymentEngine}
TID: [0] [ELB] [2014-09-17 10:46:34,458] INFO {org.wso2.carbon.core.init.CarbonServerManager} - Repository : /home/ubuntu/ELBTest/wso2elb-2.1.1/repository/deployment/server/ {org.wso2.carbon.core.init.CarbonServerManager}
TID: [0] [ELB] [2014-09-17 10:46:34,528] INFO {org.wso2.carbon.core.internal.permission.update.PermissionUpdater} - Permission cache updated for tenant -1234 {org.wso2.carbon.core.internal.permission.update.PermissionUpdater}
TID: [0] [ELB] [2014-09-17 10:46:34,689] INFO {org.wso2.carbon.mediation.initializer.ServiceBusInitializer} - Starting ESB... {org.wso2.carbon.mediation.initializer.ServiceBusInitializer}
TID: [0] [ELB] [2014-09-17 10:46:34,713] INFO {org.wso2.carbon.mediation.initializer.ServiceBusInitializer} - Initializing Apache Synapse... {org.wso2.carbon.mediation.initializer.ServiceBusInitializer}
TID: [0] [ELB] [2014-09-17 10:46:34,719] INFO {org.apache.synapse.SynapseControllerFactory} - Using Synapse home : /home/ubuntu/ELBTest/wso2elb-2.1.1/. {org.apache.synapse.SynapseControllerFactory}
TID: [0] [ELB] [2014-09-17 10:46:34,720] INFO {org.apache.synapse.SynapseControllerFactory} - Using synapse.xml location : /home/ubuntu/ELBTest/wso2elb-2.1.1/././repository/deployment/server/synapse-configs/default {org.apache.synapse.SynapseControllerFactory}
TID: [0] [ELB] [2014-09-17 10:46:34,720] INFO {org.apache.synapse.SynapseControllerFactory} - Using server name : localhost {org.apache.synapse.SynapseControllerFactory}
TID: [0] [ELB] [2014-09-17 10:46:34,726] INFO {org.apache.synapse.SynapseControllerFactory} - The timeout handler will run every : 15s {org.apache.synapse.SynapseControllerFactory}
TID: [0] [ELB] [2014-09-17 10:46:34,737] INFO {org.apache.synapse.Axis2SynapseController} - Initializing Synapse at : Wed Sep 17 10:46:34 UTC 2014 {org.apache.synapse.Axis2SynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,740] INFO {org.wso2.carbon.mediation.initializer.CarbonSynapseController} - Loading the mediation configuration from the file system {org.wso2.carbon.mediation.initializer.CarbonSynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,743] INFO {org.apache.synapse.config.xml.MultiXMLConfigurationBuilder} - Building synapse configuration from the synapse artifact repository at : ././repository/deployment/server/synapse-configs/default {org.apache.synapse.config.xml.MultiXMLConfigurationBuilder}
TID: [0] [ELB] [2014-09-17 10:46:34,748] INFO {org.apache.synapse.config.xml.XMLConfigurationBuilder} - Generating the Synapse configuration model by parsing the XML configuration {org.apache.synapse.config.xml.XMLConfigurationBuilder}
TID: [0] [ELB] [2014-09-17 10:46:34,865] INFO {org.apache.synapse.config.SynapseConfigurationBuilder} - Loaded Synapse configuration from the artifact repository at : ././repository/deployment/server/synapse-configs/default {org.apache.synapse.config.SynapseConfigurationBuilder}
TID: [0] [ELB] [2014-09-17 10:46:34,870] INFO {org.apache.synapse.Axis2SynapseController} - Loading mediator extensions... {org.apache.synapse.Axis2SynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,872] INFO {org.apache.synapse.Axis2SynapseController} - Deploying the Synapse service... {org.apache.synapse.Axis2SynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,874] INFO {org.apache.synapse.Axis2SynapseController} - Deploying Proxy services... {org.apache.synapse.Axis2SynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,874] INFO {org.apache.synapse.Axis2SynapseController} - Deploying EventSources... {org.apache.synapse.Axis2SynapseController}
TID: [0] [ELB] [2014-09-17 10:46:34,891] INFO {org.apache.synapse.ServerManager} - Server ready for processing... {org.apache.synapse.ServerManager}
TID: [0] [ELB] [2014-09-17 10:46:34,975] INFO {org.apache.synapse.endpoints.DynamicLoadbalanceEndpoint} - Dynamic load balance endpoint initialized {org.apache.synapse.endpoints.DynamicLoadbalanceEndpoint}
TID: [0] [ELB] [2014-09-17 10:46:34,978] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Managing group application domain:wso2.as.domain, sub-domain:mgt using agent class org.wso2.carbon.lb.common.group.mgt.SessionAwareGroupManagementAgent {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:34,979] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Managing group application domain:wso2.as.domain, sub-domain:worker using agent class org.wso2.carbon.lb.common.group.mgt.SessionAwareGroupManagementAgent {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:34,981] INFO {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Tenant Aware Load Balance Endpoint is initialized. {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 10:46:35,117] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener} - Starting Pass-through HTTPS Listener... {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener}
TID: [0] [ELB] [2014-09-17 10:46:35,132] INFO {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener} - Pass-through HTTPS Listener started on 0:0:0:0:0:0:0:0:8243 {org.apache.synapse.transport.passthru.PassThroughHttpSSLListener}
TID: [0] [ELB] [2014-09-17 10:46:35,133] INFO {org.apache.synapse.transport.passthru.PassThroughHttpListener} - Starting Pass-through HTTP Listener... {org.apache.synapse.transport.passthru.PassThroughHttpListener}
TID: [0] [ELB] [2014-09-17 10:46:35,137] INFO {org.apache.synapse.transport.passthru.PassThroughHttpListener} - Pass-through HTTP Listener started on 0:0:0:0:0:0:0:0:8280 {org.apache.synapse.transport.passthru.PassThroughHttpListener}
TID: [0] [ELB] [2014-09-17 10:46:35,149] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Cluster domain: wso2.carbon.lb.domain {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,149] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Using wka based membership management scheme {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,394] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Group management local member for domain [wso2.as.domain],sub-domain [worker] UUID: 6ac766ee-dfe2-450e-9c0d-6f763a1384b1. Host:10.0.0.83, Remote Host:null, Port: 4500, HTTP:8280, HTTPS:8243, Domain: wso2.as.domain, Sub-domain:null, Active:true {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,505] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Group management local member for domain [wso2.as.domain],sub-domain [mgt] UUID: 6ac766ee-dfe2-450e-9c0d-6f763a1384b1. Host:10.0.0.83, Remote Host:null, Port: 4500, HTTP:8280, HTTPS:8243, Domain: wso2.as.domain, Sub-domain:null, Active:true {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,550] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Hazelcast initialized in 16ms {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,588] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Local member: [ed1ff737-34fb-485e-bbd9-9f45c91baa91] - Host:10.0.0.83, Remote Host:null, Port: 4000, HTTP:8280, HTTPS:8243, Domain: wso2.carbon.lb.domain, Sub-domain:null, Active:true {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,590] INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} - Added member: Host:10.0.0.83, Remote Host:null, Port: 4000, HTTP:8280, HTTPS:8243, Domain: wso2.carbon.lb.domain, Sub-domain:__$default, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}
TID: [0] [ELB] [2014-09-17 10:46:35,664] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Cluster initialization completed {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:35,667] INFO {org.apache.tomcat.util.net.NioSelectorPool} - Using a shared selector for servlet write/read {org.apache.tomcat.util.net.NioSelectorPool}
TID: [0] [ELB] [2014-09-17 10:46:35,669] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - Elected this member [ed1ff737-34fb-485e-bbd9-9f45c91baa91] as the Coordinator for the cluster [wso2.carbon.lb.domain] {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent}
TID: [0] [ELB] [2014-09-17 10:46:36,000] INFO {org.apache.tomcat.util.net.NioSelectorPool} - Using a shared selector for servlet write/read {org.apache.tomcat.util.net.NioSelectorPool}
TID: [0] [ELB] [2014-09-17 10:46:36,073] INFO {org.wso2.carbon.ntask.core.service.impl.TaskServiceImpl} - Task service starting in CLUSTERED mode... {org.wso2.carbon.ntask.core.service.impl.TaskServiceImpl}
TID: [0] [ELB] [2014-09-17 10:46:36,088] WARN {org.wso2.andes.jndi.PropertiesFileInitialContextFactory} - No Provider URL specified. {org.wso2.andes.jndi.PropertiesFileInitialContextFactory}
TID: [0] [ELB] [2014-09-17 10:46:36,123] INFO {org.wso2.carbon.mediator.autoscale.lbautoscale.internal.AutoscalerTaskServiceComponent} - Autoscaling is disabled. {org.wso2.carbon.mediator.autoscale.lbautoscale.internal.AutoscalerTaskServiceComponent}
TID: [0] [ELB] [2014-09-17 10:46:36,542] INFO {org.wso2.carbon.core.init.JMXServerManager} - JMX Service URL : service:jmx:rmi://localhost:11111/jndi/rmi://localhost:9999/jmxrmi {org.wso2.carbon.core.init.JMXServerManager}
TID: [0] [ELB] [2014-09-17 10:46:36,542] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - Server : WSO2 ELB-2.1.1 {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent}
TID: [0] [ELB] [2014-09-17 10:46:36,542] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - WSO2 Carbon started in 16 sec {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent}
TID: [0] [ELB] [2014-09-17 10:46:36,773] INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} - Mgt Console URL : https://10.0.0.83:9443/carbon/ {org.wso2.carbon.ui.internal.CarbonUIServiceComponent}
TID: [0] [ELB] [2014-09-17 10:47:27,707] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [94f1353e-5431-4bda-b76a-9f8d21bf1688]: /10.0.0.85:4100 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:47:27,709] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [94f1353e-5431-4bda-b76a-9f8d21bf1688]: /10.0.0.85:4100 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:47:30,796] INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} - Added member: Host:10.0.0.85, Remote Host:null, Port: 4100, HTTP:9764, HTTPS:9444, Domain: wso2.as.domain, Sub-domain:mgt, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}
TID: [0] [ELB] [2014-09-17 10:47:30,797] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4100, HTTP:9764, HTTPS:9444, Domain: wso2.as.domain, Sub-domain:mgt, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:48:23,127] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [548980bb-3373-4e0c-9a8c-62f2c7a39b48]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:48:23,130] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [548980bb-3373-4e0c-9a8c-62f2c7a39b48]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:48:26,229] INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} - Added member: Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}
TID: [0] [ELB] [2014-09-17 10:48:26,230] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:49:56,725] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:49:56,725] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:49:59,853] INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} - Added member: Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}
TID: [0] [ELB] [2014-09-17 10:49:59,854] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 10:57:09,868] INFO {org.apache.synapse.core.axis2.TimeoutHandler} - This engine will expire all callbacks after : 86400 seconds, irrespective of the timeout action, after the specified or optional timeout {org.apache.synapse.core.axis2.TimeoutHandler}
TID: [0] [ELB] [2014-09-17 11:10:14,063] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [548980bb-3373-4e0c-9a8c-62f2c7a39b48]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:10:14,079] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [548980bb-3373-4e0c-9a8c-62f2c7a39b48]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:12:37,837] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [c855d373-1544-410d-9080-14a6440ee98e]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:12:37,854] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [c855d373-1544-410d-9080-14a6440ee98e]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:12:40,964] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:13:55,107] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:13:55,122] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [aa7c330e-eb71-488c-a905-79c3a07ba671]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:12,293] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:12,297] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:15,423] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 11:16:16,139] WARN {org.apache.synapse.transport.passthru.TargetHandler} - Connection closed by target host before receiving the request {org.apache.synapse.transport.passthru.TargetHandler}
TID: [0] [ELB] [2014-09-17 11:16:16,169] INFO {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Failed to send message to Member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true . Error Code: 101508 {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 11:16:16,170] ERROR {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Error sending request! Connection timeout after request is read by the host as.wso2.com . Error code: 101508 Protocol State: RESPONSE HEAD {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 12:13:48,082] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 12:13:48,089] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [cd2f1123-f734-456f-bc3f-7ae03ec0142a]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 12:15:21,203] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [485c6c84-fc5d-4a7f-ad6b-c7a769b4643d]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 12:15:21,205] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [485c6c84-fc5d-4a7f-ad6b-c7a769b4643d]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 12:15:24,324] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:08:10,976] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [485c6c84-fc5d-4a7f-ad6b-c7a769b4643d]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:08:10,999] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [485c6c84-fc5d-4a7f-ad6b-c7a769b4643d]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:09:16,987] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [4b0868e5-7c0d-4f8e-97c4-78f54662d60f]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:09:16,990] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [4b0868e5-7c0d-4f8e-97c4-78f54662d60f]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:09:20,123] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:05,100] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [4b0868e5-7c0d-4f8e-97c4-78f54662d60f]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:05,129] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [4b0868e5-7c0d-4f8e-97c4-78f54662d60f]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:48,835] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [02c387c3-bbae-4f59-9ab2-22c0deea40f3]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:48,835] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [02c387c3-bbae-4f59-9ab2-22c0deea40f3]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:51,958] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:15:52,185] WARN {org.apache.synapse.transport.passthru.TargetHandler} - Connection closed by target host before receiving the request {org.apache.synapse.transport.passthru.TargetHandler}
TID: [0] [ELB] [2014-09-17 13:15:52,191] INFO {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Failed to send message to Member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true . Error Code: 101508 {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 13:15:52,192] ERROR {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Error sending request! Connection timeout after request is read by the host as.wso2.com . Error code: 101508 Protocol State: RESPONSE HEAD {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 13:20:14,180] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [02c387c3-bbae-4f59-9ab2-22c0deea40f3]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:20:14,192] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [02c387c3-bbae-4f59-9ab2-22c0deea40f3]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:21:13,569] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [2ee171b5-1ab7-468b-aef3-b6b9130910b8]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:21:13,571] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [2ee171b5-1ab7-468b-aef3-b6b9130910b8]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:21:16,692] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:24:16,252] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [2ee171b5-1ab7-468b-aef3-b6b9130910b8]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:24:16,267] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [2ee171b5-1ab7-468b-aef3-b6b9130910b8]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:25:19,201] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [b3ef54bd-d214-4feb-a9bd-c5257ef65deb]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:25:19,203] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [b3ef54bd-d214-4feb-a9bd-c5257ef65deb]: /10.0.0.85:4300 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:25:22,344] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.85, Remote Host:null, Port: 4300, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:28:02,303] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [c855d373-1544-410d-9080-14a6440ee98e]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:28:02,324] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [c855d373-1544-410d-9080-14a6440ee98e]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:28:55,210] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [729b1409-c178-46ca-8ebc-c652dbae53eb]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:28:55,210] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [729b1409-c178-46ca-8ebc-c652dbae53eb]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:28:58,345] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:30:17,368] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [729b1409-c178-46ca-8ebc-c652dbae53eb]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:30:17,373] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member left [729b1409-c178-46ca-8ebc-c652dbae53eb]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:33:05,656] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [0a744a28-4fbc-4b4a-b687-6f69d43acb17]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:33:05,660] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Member joined [0a744a28-4fbc-4b4a-b687-6f69d43acb17]: /10.0.0.86:4200 {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:33:08,788] INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent} - Application member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true joined application cluster {org.wso2.carbon.core.clustering.hazelcast.HazelcastGroupManagementAgent}
TID: [0] [ELB] [2014-09-17 13:33:09,229] WARN {org.apache.synapse.transport.passthru.TargetHandler} - Connection closed by target host before receiving the request {org.apache.synapse.transport.passthru.TargetHandler}
TID: [0] [ELB] [2014-09-17 13:33:09,230] INFO {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Failed to send message to Member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true . Error Code: 101508 {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 13:33:09,230] ERROR {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Error sending request! Connection timeout after request is read by the host as.wso2.com . Error code: 101508 Protocol State: RESPONSE HEAD {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 13:33:09,635] WARN {org.apache.synapse.transport.passthru.TargetHandler} - Connection closed by target host before receiving the request {org.apache.synapse.transport.passthru.TargetHandler}
TID: [0] [ELB] [2014-09-17 13:33:09,636] INFO {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Failed to send message to Member Host:10.0.0.86, Remote Host:null, Port: 4200, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true . Error Code: 101508 {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
TID: [0] [ELB] [2014-09-17 13:33:09,636] ERROR {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint} - Error sending request! Connection timeout after request is read by the host as.wso2.com . Error code: 101508 Protocol State: RESPONSE HEAD {org.wso2.carbon.lb.endpoint.endpoint.TenantAwareLoadBalanceEndpoint}
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
