If all of the agents lost contact, this looks like a network-level drop. See: https://github.com/apache/httpcomponents-client/blob/3a68e74ec839263ae6c2cf770b6aa1e8f849ce19/httpclient/src/main/java/org/apache/http/impl/conn/DefaultHttpResponseParser.java#L141
which refers to apache HTTPcomponents http client 4.5.12 (See: https://github.com/gocd/gocd/blob/2c822492f3efeb5c9b6a9eed661f55243d6b7db9/dependencies.gradle#L31 ) Are you able to reach that GoCD service from your laptop, from the server itself (via a curl and via a temporary GoCD agent)? Do you know what changed (Network maintenance, OS hardening, patches applied)? -- Ram On Sun, Feb 7, 2021 at 8:59 AM Prabha <[email protected]> wrote: > Hi All, > > We have been using gocd 20.4 for a while now and we have around 20 agents > directly reporting to go-server(no proxy or loadbalancers in between). All > these agents were working well till yesterday but all of a sudden all these > agents "*Lost Contact*" for no reason. > > I have been digging around this issue for hours but its just dead ends > everywhere. > > Seen few threads similar to this issue but not of great help, any > suggestions would be great help. > > 2021-02-06 23:06:00,105 ERROR [scheduler-1] AgentHTTPClientController:103 > - Error occurred when agent tried to ping server: > org.springframework.remoting.RemoteAccessException: Could not access HTTP > invoker remote service at > [*https://myserver.com:8154/go/remoting/remoteBuildRepository > <https://myserver.com:8154/go/remoting/remoteBuildRepository>*]; nested > exception is org.apache.http.NoHttpResponseException: myserver.com:8154 > failed to respond > at > org.springframework.remoting.httpinvoker.HttpInvokerClientInterceptor.convertHttpInvokerAccessException(HttpInvokerClientInterceptor.java:226) > at > org.springframework.remoting.httpinvoker.HttpInvokerClientInterceptor.invoke(HttpInvokerClientInterceptor.java:153) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) > at com.sun.proxy.$Proxy10.ping(Unknown Source) > at > com.thoughtworks.go.agent.AgentHTTPClientController.ping(AgentHTTPClientController.java:98) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:567) > at > org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) > at > org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at > java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) > at > java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:835) > Caused by: org.apache.http.NoHttpResponseException: myserver.com:8154 > failed to respond > at > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) > at > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) > at > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) > at > org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) > at > org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) > at > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) > at > org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) > at > org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) > at > org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) > at > org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) > at > org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) > at > org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) > at > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) > at > com.thoughtworks.go.agent.common.ssl.GoAgentServerHttpClient.execute(GoAgentServerHttpClient.java:50) > at > com.thoughtworks.go.agent.GoHttpClientHttpInvokerRequestExecutor.doExecuteRequest(GoHttpClientHttpInvokerRequestExecutor.java:65) > at > org.springframework.remoting.httpinvoker.AbstractHttpInvokerRequestExecutor.executeRequest(AbstractHttpInvokerRequestExecutor.java:137) > at > org.springframework.remoting.httpinvoker.HttpInvokerClientInterceptor.executeRequest(HttpInvokerClientInterceptor.java:202) > at > org.springframework.remoting.httpinvoker.HttpInvokerClientInterceptor.executeRequest(HttpInvokerClientInterceptor.java:184) > at > org.springframework.remoting.httpinvoker.HttpInvokerClientInterceptor.invoke(HttpInvokerClientInterceptor.java:150) > ... 16 common frames omitted > > Thanks! > Prabha > > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/8be9be9e-c6e5-41b2-9af2-010c90aa41c0n%40googlegroups.com > <https://groups.google.com/d/msgid/go-cd/8be9be9e-c6e5-41b2-9af2-010c90aa41c0n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CANiY96aZVyCKsYpBptxq3iT2puQykgOTkheO4dB7gzXgtuTE6w%40mail.gmail.com.
