kfaraz commented on code in PR #18142:
URL: https://github.com/apache/druid/pull/18142#discussion_r2147409913
##########
server/src/test/java/org/apache/druid/client/coordinator/CoordinatorClientImplTest.java:
##########
@@ -450,4 +465,35 @@ public void test_getCoordinatorDynamicConfig() throws
Exception
coordinatorClient.getCoordinatorDynamicConfig().get()
);
}
+
+ @Test
+ public void test_fetchLookupsForTier_detailedEnabled() throws Exception
+ {
+ LookupExtractorFactory lookupData = new MapLookupExtractorFactory(
+ Map.of(
+ "77483", "United States",
+ "77484", "India"
+ ),
+ true
+ );
+ LookupExtractorFactoryContainer lookupDataContainer = new
LookupExtractorFactoryContainer("v0", lookupData);
+ Map<String, LookupExtractorFactoryContainer> lookups = Map.of(
+ "country_code", lookupDataContainer
+ );
+
+ serviceClient.expectAndRespond(
+ new RequestBuilder(
+ HttpMethod.GET,
+ "/druid/coordinator/v1/lookups/config/country_code?detailed=true"
+ ),
+ HttpResponseStatus.OK,
+ ImmutableMap.of(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON),
Review Comment:
Nit: use `Map.of()` as you are already using it elsewhere in this method.
##########
server/src/main/java/org/apache/druid/query/lookup/LookupReferencesManager.java:
##########
@@ -476,38 +469,20 @@ private List<LookupBean>
getLookupListFromCoordinator(String tier)
@Nullable
private Map<String, LookupExtractorFactoryContainer>
tryGetLookupListFromCoordinator(String tier)
- throws IOException, InterruptedException
{
- final StringFullResponseHolder response = fetchLookupsForTier(tier);
- if (response.getStatus().equals(HttpResponseStatus.NOT_FOUND)) {
- LOG.warn("No lookups found for tier [%s], response [%s]", tier,
response);
- return null;
- } else if (!response.getStatus().equals(HttpResponseStatus.OK)) {
- throw new IOE(
- "Error while fetching lookup code from Coordinator with status[%s]
and content[%s]",
- response.getStatus(),
- response.getContent()
- );
+ try {
+ return FutureUtils.getUnchecked(
+ coordinatorClient.fetchLookupsForTier(tier), true);
}
-
- // Older version of getSpecificTier returns a list of lookup names.
- // Lookup loading is performed via snapshot if older version is present.
- // This check is only for backward compatibility and should be removed in
a future release
- if (response.getContent().startsWith("[")) {
- LOG.info(
- "Failed to retrieve lookup information from coordinator, " +
- "because coordinator appears to be running on older Druid version. "
+
- "Attempting to load lookups using snapshot instead"
- );
- return null;
- } else {
- Map<String, Object> lookupNameToGenericConfig =
- jsonMapper.readValue(response.getContent(),
LOOKUPS_ALL_GENERIC_REFERENCE);
- return LookupUtils.tryConvertObjectMapToLookupConfigMap(
- lookupNameToGenericConfig,
- jsonMapper
+ catch (Exception e) {
+ HttpResponseException httpEx = (HttpResponseException) e.getCause();
Review Comment:
We don't know if the cause will always be an `HttpResponseException`.
Use `Throwables.getRootCause()` and then check if it is an
`HttpResponseException`.
##########
server/src/test/java/org/apache/druid/client/coordinator/CoordinatorClientImplTest.java:
##########
@@ -450,4 +465,35 @@ public void test_getCoordinatorDynamicConfig() throws
Exception
coordinatorClient.getCoordinatorDynamicConfig().get()
);
}
+
+ @Test
+ public void test_fetchLookupsForTier_detailedEnabled() throws Exception
+ {
+ LookupExtractorFactory lookupData = new MapLookupExtractorFactory(
+ Map.of(
+ "77483", "United States",
+ "77484", "India"
+ ),
+ true
+ );
+ LookupExtractorFactoryContainer lookupDataContainer = new
LookupExtractorFactoryContainer("v0", lookupData);
+ Map<String, LookupExtractorFactoryContainer> lookups = Map.of(
+ "country_code", lookupDataContainer
+ );
+
+ serviceClient.expectAndRespond(
+ new RequestBuilder(
+ HttpMethod.GET,
+ "/druid/coordinator/v1/lookups/config/country_code?detailed=true"
+ ),
+ HttpResponseStatus.OK,
+ ImmutableMap.of(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON),
+ DefaultObjectMapper.INSTANCE.writeValueAsBytes(lookups)
+ );
+
+ Assert.assertEquals(
+ lookups,
+ coordinatorClient.fetchLookupsForTier("country_code").get()
Review Comment:
Nit: `country_code` doesn't sound like a tier name for historicals, use
`lookupTier`, `default_tier` or `cold_tier` instead.
##########
server/src/test/java/org/apache/druid/client/coordinator/CoordinatorClientImplTest.java:
##########
@@ -450,4 +465,35 @@ public void test_getCoordinatorDynamicConfig() throws
Exception
coordinatorClient.getCoordinatorDynamicConfig().get()
);
}
+
+ @Test
+ public void test_fetchLookupsForTier_detailedEnabled() throws Exception
+ {
+ LookupExtractorFactory lookupData = new MapLookupExtractorFactory(
+ Map.of(
+ "77483", "United States",
+ "77484", "India"
+ ),
+ true
+ );
+ LookupExtractorFactoryContainer lookupDataContainer = new
LookupExtractorFactoryContainer("v0", lookupData);
+ Map<String, LookupExtractorFactoryContainer> lookups = Map.of(
+ "country_code", lookupDataContainer
+ );
+
+ serviceClient.expectAndRespond(
+ new RequestBuilder(
+ HttpMethod.GET,
+ "/druid/coordinator/v1/lookups/config/country_code?detailed=true"
+ ),
+ HttpResponseStatus.OK,
+ ImmutableMap.of(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON),
+ DefaultObjectMapper.INSTANCE.writeValueAsBytes(lookups)
+ );
+
+ Assert.assertEquals(
+ lookups,
+ coordinatorClient.fetchLookupsForTier("country_code").get()
Review Comment:
Maybe use `FutureUtils.getUnchecked()` for consistency.
##########
server/src/main/java/org/apache/druid/client/coordinator/CoordinatorClientImpl.java:
##########
@@ -46,10 +50,12 @@
import javax.annotation.Nullable;
import java.util.ArrayList;
import java.util.List;
+import java.util.Map;
import java.util.Set;
public class CoordinatorClientImpl implements CoordinatorClient
{
+ private static final Logger LOG = new Logger(CoordinatorClientImpl.class);
Review Comment:
Not needed anymore.
##########
server/src/main/java/org/apache/druid/query/lookup/LookupReferencesManager.java:
##########
@@ -476,38 +469,20 @@ private List<LookupBean>
getLookupListFromCoordinator(String tier)
@Nullable
private Map<String, LookupExtractorFactoryContainer>
tryGetLookupListFromCoordinator(String tier)
- throws IOException, InterruptedException
{
- final StringFullResponseHolder response = fetchLookupsForTier(tier);
- if (response.getStatus().equals(HttpResponseStatus.NOT_FOUND)) {
- LOG.warn("No lookups found for tier [%s], response [%s]", tier,
response);
- return null;
- } else if (!response.getStatus().equals(HttpResponseStatus.OK)) {
- throw new IOE(
- "Error while fetching lookup code from Coordinator with status[%s]
and content[%s]",
- response.getStatus(),
- response.getContent()
- );
+ try {
+ return FutureUtils.getUnchecked(
+ coordinatorClient.fetchLookupsForTier(tier), true);
}
-
- // Older version of getSpecificTier returns a list of lookup names.
- // Lookup loading is performed via snapshot if older version is present.
- // This check is only for backward compatibility and should be removed in
a future release
- if (response.getContent().startsWith("[")) {
- LOG.info(
- "Failed to retrieve lookup information from coordinator, " +
- "because coordinator appears to be running on older Druid version. "
+
- "Attempting to load lookups using snapshot instead"
- );
- return null;
- } else {
- Map<String, Object> lookupNameToGenericConfig =
- jsonMapper.readValue(response.getContent(),
LOOKUPS_ALL_GENERIC_REFERENCE);
- return LookupUtils.tryConvertObjectMapToLookupConfigMap(
- lookupNameToGenericConfig,
- jsonMapper
+ catch (Exception e) {
+ HttpResponseException httpEx = (HttpResponseException) e.getCause();
+ LOG.warn(
Review Comment:
If the error is 404, no found, we need to return `null` to retain original
code behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
