This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.8
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.8 by this push:
new 972ed268d [KYUUBI #6081] RESTful client should retry on
UnknownHostException
972ed268d is described below
commit 972ed268df1c5e99fe0a1ac05b4e10b66bd11527
Author: Cheng Pan <[email protected]>
AuthorDate: Sat Feb 24 17:37:26 2024 +0800
[KYUUBI #6081] RESTful client should retry on UnknownHostException
# :mag: Description
## Issue References ๐
When deploying Kyuubi as StatefulSet on Kubernetes, during the rolling
upgrade, the Pod stops and creates one by one, and there are a few durations
that the Pod is not available, then `UnknownHostException` is returned.
```
2024-02-23 23:04:45 [ERROR] [KyuubiRestFrontendService-122]
org.apache.kyuubi.client.RestClient#189 - Error:
java.net.UnknownHostException:
kyuubi-2.kyuubi-headless.spark.svc.cluster.local
at java.net.InetAddress$CachedAddresses.get(InetAddress.java:764)
~[?:1.8.0_382]
...
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
~[httpclient-4.5.14.jar:4.5.14]
at
org.apache.kyuubi.client.RestClient.doRequest(RestClient.java:181)
~[kyuubi-rest-client-1.8.0.17.jar:1.8.0.17]
at org.apache.kyuubi.client.RestClient.get(RestClient.java:80)
~[kyuubi-rest-client-1.8.0.17.jar:1.8.0.17]
...
at
org.apache.kyuubi.client.BatchRestApi.getBatchLocalLog(BatchRestApi.java:104)
~[kyuubi-rest-client-1.8.0.17.jar:1.8.0.17]
at
org.apache.kyuubi.server.api.v1.InternalRestClient.$anonfun$getBatchLocalLog$1(InternalRestClient.scala:57)
~[kyuubi-server_2.12-1.8.0.17.jar:1.8.0.17]
at
org.apache.kyuubi.server.api.v1.InternalRestClient.withAuthUser(InternalRestClient.scala:81)
~[kyuubi-server_2.12-1.8.0.17.jar:1.8.0.17]
at
org.apache.kyuubi.server.api.v1.InternalRestClient.getBatchLocalLog(InternalRestClient.scala:57)
~[kyuubi-server_2.12-1.8.0.17.jar:1.8.0.17]
at
org.apache.kyuubi.server.api.v1.BatchesResource.$anonfun$getBatchLocalLog$4(BatchesResource.scala:424)
~[kyuubi-server_2.12-1.8.0.17.jar:1.8.0.17]
```
## Describe Your Solution ๐ง
Treat `UnknownHostException` as a retriable exception to tolerant the
transient Pod FQDN not available cases during rolling upgrade on K8s.
## Types of changes :bookmark:
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
Pass GA.
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6081 from pan3793/unknown-host.
Closes #6081
0df3883e6 [Cheng Pan] RESTful client should retry on UnknownHostException
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit d51ad2f45b36a5fc6f1299285eb9ba6baed2772d)
Signed-off-by: Cheng Pan <[email protected]>
---
.../src/main/java/org/apache/kyuubi/client/RestClient.java | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git
a/kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/RestClient.java
b/kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/RestClient.java
index e6d1d9674..86af2c759 100644
--- a/kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/RestClient.java
+++ b/kyuubi-rest-client/src/main/java/org/apache/kyuubi/client/RestClient.java
@@ -21,6 +21,7 @@ import java.io.File;
import java.net.ConnectException;
import java.net.URI;
import java.net.URISyntaxException;
+import java.net.UnknownHostException;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import org.apache.commons.collections.MapUtils;
@@ -180,7 +181,10 @@ public class RestClient implements IRestClient {
response = httpclient.execute(httpRequest, responseHandler);
LOG.debug("Response: {}", response);
- } catch (ConnectException | ConnectTimeoutException |
NoHttpResponseException e) {
+ } catch (ConnectException
+ | UnknownHostException
+ | ConnectTimeoutException
+ | NoHttpResponseException e) {
// net exception can be retried by connecting to other Kyuubi server
throw new RetryableKyuubiRestException("Api request failed for " +
uri.toString(), e);
} catch (KyuubiRestException rethrow) {
