[google-appengine] Re: Too many 502 from LoadBalance

[Vinay Dahiya]

Exactly the same error been happening with us as well since yesterday.

And the log viewer doesn't really give a verbose options in detailing the 
error.

the logs don't show which ip was used to send the request with from the 
load balancer which makes it super difficult to debug any issues.



On Friday, March 23, 2018 at 6:36:00 PM UTC+5:30, Anu V wrote:
>
> I'am experiencing too many 502 errors from Load Balance and the instances 
> are working fine. (
>
>
> @type: "
> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
> statusDetails: "failed_to_connect_to_backend" 
>
> I've also found that this happen on my few projects starting about 12 
> hours ago, I've checked https://status.cloud.google.com/ and it said all 
> working fine.
> I cannot upgrade support level to create the ticket, so I am posting it 
> here to ask if anyone having the same problem as I am?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/07e1a699-3a5d-483f-823d-d4accd815d6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

[Jonas Klemming]

I have an open support case for this as well. It seems to be a known issue 
they are working on for 1+ months. Not sure when it will be fixed.


Den fredag 23 mars 2018 kl. 20:24:30 UTC+1 skrev Anu V:
>
> Backends are healthy, I found this 502 errors across my few projects using 
> AppEngine.
>
> All instances are able to access and work normally, I've tested using curl 
> to send a request directly to instance's private ip and it works all the 
> time.
>
>
> On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google Cloud 
> Support) wrote:
>>
>> Hello Anu,
>>
>> The "*failed_to_connect_to_backend" *error message indicates that load 
>> balancer is failing to connect to backends. There could be many possible 
>> reasons for it. For example: Unhealthy instance if reported by health 
>> check, incorrect URL mapping 
>> , 
>> high backend utilization causing backend not to respond to the load 
>> balancer requests. You can refer to this thread 
>> for 
>> more information and check instance system/error and access logs. 
>>
>> On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:
>>>
>>> I'am experiencing too many 502 errors from Load Balance and the 
>>> instances are working fine. (
>>>
>>>
>>> @type: "
>>> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
>>>
>>> statusDetails: "failed_to_connect_to_backend" 
>>>
>>> I've also found that this happen on my few projects starting about 12 
>>> hours ago, I've checked https://status.cloud.google.com/ and it said 
>>> all working fine.
>>> I cannot upgrade support level to create the ticket, so I am posting it 
>>> here to ask if anyone having the same problem as I am?
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/eb4b98ea-68a7-4c1d-bc41-9af7ee8e30be%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

[Suvodeep Pyne]

+1 Anu
This is the second time in the past 4 months we faced a P1 issue which took 
2 days to get resolved.

@Google team, this is perhaps not the best customer experience. I am aware 
of your support tiers but there should be a better interaction model with 
your customer. We are paying for the service and you are answerable to us 
in case of outages / service disruption.

On Saturday, March 24, 2018 at 10:32:14 PM UTC+5:30, Suvodeep Pyne wrote:
>
> We are also facing the same issue. The LB is throwing 502's 
> intermittently. 
>
> A simple parallel curl with 30 requests is able to choke up the LB 5 - 8 
> times. Our App servers are healthy and are not receiving the requests.
>
>
> On Saturday, March 24, 2018 at 2:14:21 AM UTC+5:30, Navi Aujla (Google 
> Cloud Support) wrote:
>>
>> Hello Anu
>>
>> As mentioned in my last message there are multiple factors which may 
>> cause this error. However, If you believe this to an issue with the GCP 
>> load balancer, I would suggest you to open a private issue report using this 
>> link 
>> [1]
>>  
>> providing the relevant detailed information for the investigation. 
>>
>>
>> On Friday, March 23, 2018 at 3:24:30 PM UTC-4, Anu V wrote:
>>>
>>> Backends are healthy, I found this 502 errors across my few projects 
>>> using AppEngine.
>>>
>>> All instances are able to access and work normally, I've tested using 
>>> curl to send a request directly to instance's private ip and it works all 
>>> the time.
>>>
>>>
>>> On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google 
>>> Cloud Support) wrote:

 Hello Anu,

 The "*failed_to_connect_to_backend" *error message indicates that load 
 balancer is failing to connect to backends. There could be many possible 
 reasons for it. For example: Unhealthy instance if reported by health 
 check, incorrect URL mapping 
 , 
 high backend utilization causing backend not to respond to the load 
 balancer requests. You can refer to this thread 
 for 
 more information and check instance system/error and access logs. 

 On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:
>
> I'am experiencing too many 502 errors from Load Balance and the 
> instances are working fine. (
>
>
> @type: "
> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
>
> statusDetails: "failed_to_connect_to_backend" 
>
> I've also found that this happen on my few projects starting about 12 
> hours ago, I've checked https://status.cloud.google.com/ and it said 
> all working fine.
> I cannot upgrade support level to create the ticket, so I am posting 
> it here to ask if anyone having the same problem as I am?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8b30e604-9db0-478f-8e5e-cce4448f3e33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

[Suvodeep Pyne]

We are also facing the same issue. The LB is throwing 502's intermittently. 

A simple parallel curl with 30 requests is able to choke up the LB 5 - 8 
times. Our App servers are healthy and are not receiving the requests.


On Saturday, March 24, 2018 at 2:14:21 AM UTC+5:30, Navi Aujla (Google 
Cloud Support) wrote:
>
> Hello Anu
>
> As mentioned in my last message there are multiple factors which may cause 
> this error. However, If you believe this to an issue with the GCP load 
> balancer, I would suggest you to open a private issue report using this 
> link 
> [1]
>  
> providing the relevant detailed information for the investigation. 
>
>
> On Friday, March 23, 2018 at 3:24:30 PM UTC-4, Anu V wrote:
>>
>> Backends are healthy, I found this 502 errors across my few projects 
>> using AppEngine.
>>
>> All instances are able to access and work normally, I've tested using 
>> curl to send a request directly to instance's private ip and it works all 
>> the time.
>>
>>
>> On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google Cloud 
>> Support) wrote:
>>>
>>> Hello Anu,
>>>
>>> The "*failed_to_connect_to_backend" *error message indicates that load 
>>> balancer is failing to connect to backends. There could be many possible 
>>> reasons for it. For example: Unhealthy instance if reported by health 
>>> check, incorrect URL mapping 
>>> , 
>>> high backend utilization causing backend not to respond to the load 
>>> balancer requests. You can refer to this thread 
>>> for 
>>> more information and check instance system/error and access logs. 
>>>
>>> On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:

 I'am experiencing too many 502 errors from Load Balance and the 
 instances are working fine. (


 @type: "
 type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 

 statusDetails: "failed_to_connect_to_backend" 

 I've also found that this happen on my few projects starting about 12 
 hours ago, I've checked https://status.cloud.google.com/ and it said 
 all working fine.
 I cannot upgrade support level to create the ticket, so I am posting it 
 here to ask if anyone having the same problem as I am?



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/eeda88ee-b5ae-43cc-a8e2-d9905f94383c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

[Anu V]

20 hours after created a ticket support and post in issuetracker, the issue 
has been solved, still no explanation though.

Not to mention that I cannot upgrade my support plan for about 20 hours 
earlier until I need to call support team and hours laters I finally can 
upgrade support plan.

My frustration is it took so long and so hard to convince your engineers 
that it's problem on your end, not to mention how long it've been solve.

And yeah, google cloud status shown all green.

It took about 36 hours before P1 issue get fixed, really unacceptable.


On Saturday, March 24, 2018 at 3:44:21 AM UTC+7, Navi Aujla (Google Cloud 
Support) wrote:
>
> Hello Anu
>
> As mentioned in my last message there are multiple factors which may cause 
> this error. However, If you believe this to an issue with the GCP load 
> balancer, I would suggest you to open a private issue report using this 
> link 
> [1]
>  
> providing the relevant detailed information for the investigation. 
>
>
> On Friday, March 23, 2018 at 3:24:30 PM UTC-4, Anu V wrote:
>>
>> Backends are healthy, I found this 502 errors across my few projects 
>> using AppEngine.
>>
>> All instances are able to access and work normally, I've tested using 
>> curl to send a request directly to instance's private ip and it works all 
>> the time.
>>
>>
>> On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google Cloud 
>> Support) wrote:
>>>
>>> Hello Anu,
>>>
>>> The "*failed_to_connect_to_backend" *error message indicates that load 
>>> balancer is failing to connect to backends. There could be many possible 
>>> reasons for it. For example: Unhealthy instance if reported by health 
>>> check, incorrect URL mapping 
>>> , 
>>> high backend utilization causing backend not to respond to the load 
>>> balancer requests. You can refer to this thread 
>>> for 
>>> more information and check instance system/error and access logs. 
>>>
>>> On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:

 I'am experiencing too many 502 errors from Load Balance and the 
 instances are working fine. (


 @type: "
 type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 

 statusDetails: "failed_to_connect_to_backend" 

 I've also found that this happen on my few projects starting about 12 
 hours ago, I've checked https://status.cloud.google.com/ and it said 
 all working fine.
 I cannot upgrade support level to create the ticket, so I am posting it 
 here to ask if anyone having the same problem as I am?



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0f4ee81e-b724-4c7a-8240-f5a2514e751b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

['Navi Aujla (Google Cloud Support)' via Google App Engine]

Hello Anu

As mentioned in my last message there are multiple factors which may cause 
this error. However, If you believe this to an issue with the GCP load 
balancer, I would suggest you to open a private issue report using this link 
[1]
 
providing the relevant detailed information for the investigation. 


On Friday, March 23, 2018 at 3:24:30 PM UTC-4, Anu V wrote:
>
> Backends are healthy, I found this 502 errors across my few projects using 
> AppEngine.
>
> All instances are able to access and work normally, I've tested using curl 
> to send a request directly to instance's private ip and it works all the 
> time.
>
>
> On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google Cloud 
> Support) wrote:
>>
>> Hello Anu,
>>
>> The "*failed_to_connect_to_backend" *error message indicates that load 
>> balancer is failing to connect to backends. There could be many possible 
>> reasons for it. For example: Unhealthy instance if reported by health 
>> check, incorrect URL mapping 
>> , 
>> high backend utilization causing backend not to respond to the load 
>> balancer requests. You can refer to this thread 
>> for 
>> more information and check instance system/error and access logs. 
>>
>> On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:
>>>
>>> I'am experiencing too many 502 errors from Load Balance and the 
>>> instances are working fine. (
>>>
>>>
>>> @type: "
>>> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
>>>
>>> statusDetails: "failed_to_connect_to_backend" 
>>>
>>> I've also found that this happen on my few projects starting about 12 
>>> hours ago, I've checked https://status.cloud.google.com/ and it said 
>>> all working fine.
>>> I cannot upgrade support level to create the ticket, so I am posting it 
>>> here to ask if anyone having the same problem as I am?
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/62061c20-c398-4d84-9419-0890ddc43a4f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

[Anu V]

Backends are healthy, I found this 502 errors across my few projects using 
AppEngine.

All instances are able to access and work normally, I've tested using curl 
to send a request directly to instance's private ip and it works all the 
time.


On Saturday, March 24, 2018 at 2:07:30 AM UTC+7, Navi Aujla (Google Cloud 
Support) wrote:
>
> Hello Anu,
>
> The "*failed_to_connect_to_backend" *error message indicates that load 
> balancer is failing to connect to backends. There could be many possible 
> reasons for it. For example: Unhealthy instance if reported by health 
> check, incorrect URL mapping 
> , high 
> backend utilization causing backend not to respond to the load balancer 
> requests. You can refer to this thread 
> for 
> more information and check instance system/error and access logs. 
>
> On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:
>>
>> I'am experiencing too many 502 errors from Load Balance and the instances 
>> are working fine. (
>>
>>
>> @type: "
>> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
>>
>> statusDetails: "failed_to_connect_to_backend" 
>>
>> I've also found that this happen on my few projects starting about 12 
>> hours ago, I've checked https://status.cloud.google.com/ and it said all 
>> working fine.
>> I cannot upgrade support level to create the ticket, so I am posting it 
>> here to ask if anyone having the same problem as I am?
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/061e6ad1-3214-440f-9287-12b87b0c246d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Too many 502 from LoadBalance

['Navi Aujla (Google Cloud Support)' via Google App Engine]

Hello Anu,

The "*failed_to_connect_to_backend" *error message indicates that load 
balancer is failing to connect to backends. There could be many possible 
reasons for it. For example: Unhealthy instance if reported by health 
check, incorrect URL mapping 
, high 
backend utilization causing backend not to respond to the load balancer 
requests. You can refer to this thread 
for 
more information and check instance system/error and access logs. 

On Friday, March 23, 2018 at 9:06:00 AM UTC-4, Anu V wrote:
>
> I'am experiencing too many 502 errors from Load Balance and the instances 
> are working fine. (
>
>
> @type: "
> type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" 
> statusDetails: "failed_to_connect_to_backend" 
>
> I've also found that this happen on my few projects starting about 12 
> hours ago, I've checked https://status.cloud.google.com/ and it said all 
> working fine.
> I cannot upgrade support level to create the ticket, so I am posting it 
> here to ask if anyone having the same problem as I am?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/02e2a40e-08b2-4f53-a6e0-257595d3f762%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.