yuanbenson commented on code in PR #9295:
URL: https://github.com/apache/pinot/pull/9295#discussion_r959048302
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java:
##########
@@ -747,6 +751,54 @@ public SimpleHttpResponse uploadSegment(URI uri, String
segmentName, InputStream
return uploadSegment(uri, segmentName, inputStream, null, parameters,
HttpClient.DEFAULT_SOCKET_TIMEOUT_MS);
}
+ /**
+ * Returns a map from a given tableType to a list of segments for that given
tableType (OFFLINE or REALTIME)
+ * If tableType is left unspecified, both OFFLINE and REALTIME segments will
be returned in the map.
+ */
+ public Map<String, List<String>> getSegments(URI uri, String rawTableName,
@Nullable String tableType,
+ boolean excludeReplacedSegments)
+ throws URISyntaxException, IOException {
+ List<String> tableTypes;
+ if (tableType == null || tableType.isEmpty()) {
+ tableTypes = Arrays.asList(TableType.OFFLINE.toString(),
TableType.REALTIME.toString());
+ } else {
+ tableTypes = Arrays.asList(tableType);
+ }
+ ControllerRequestURLBuilder controllerRequestURLBuilder =
ControllerRequestURLBuilder.baseUrl(uri.toString());
+ Map<String, List<String>> tableTypeToSegments = new HashMap<>();
+ for (String tableTypeToFilter : tableTypes) {
+ List<String> segments = new ArrayList<>();
+ RequestBuilder requestBuilder = RequestBuilder.get(
+
controllerRequestURLBuilder.forSegmentListAPIWithTableTypeAndExcludeReplacedSegments(rawTableName,
+ tableTypeToFilter,
excludeReplacedSegments)).setVersion(HttpVersion.HTTP_1_1);
+ HttpClient.setTimeout(requestBuilder,
HttpClient.DEFAULT_SOCKET_TIMEOUT_MS);
+ SimpleHttpResponse response;
+ try {
+ response =
HttpClient.wrapAndThrowHttpException(_httpClient.sendRequest(requestBuilder.build()));
+ } catch (HttpErrorStatusException e) {
+ tableTypeToSegments.put(tableTypeToFilter, segments);
+ continue;
+ }
+ String responseString = response.getResponse();
+ JsonNode responseJsonNode = JsonUtils.stringToJsonNode(responseString);
+ Iterator<JsonNode> responseElements = responseJsonNode.elements();
+ while (responseElements.hasNext()) {
Review Comment:
Could you clarify a bit more on the input parameters/ask for this method we
are talking about? I'll be glad to extract that out if it does not already
exist.
Ohh I see, I'll look into possibilities of reusing `getSelectedSegments` API
that you previously brought in. That may be used when we support APPEND tables
anyways! Thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
