devmadhuu commented on code in PR #6969:
URL: https://github.com/apache/ozone/pull/6969#discussion_r1795832759
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconUtils.java:
##########
@@ -596,6 +600,70 @@ public static long convertToEpochMillis(String dateString,
String dateFormat, Ti
}
}
+ /**
+ * Retrieves keys from the specified table based on pagination and prefix
filtering.
+ * This method handles different scenarios based on the presence of
startPrefix and prevKey,
+ * enabling efficient key retrieval from the table.
+ *
+ * The method handles the following cases:
+ *
+ * 1. prevKey provided, startPrefix empty:
+ * - Seeks to prevKey, skips it, and returns subsequent records up to the
limit.
+ *
+ * 2. prevKey empty, startPrefix empty:
+ * - Iterates from the beginning of the table, retrieving all records up to
the limit.
+ *
+ * 3. startPrefix provided, prevKey empty:
+ * - Seeks to the first key matching startPrefix and returns all matching
keys up to the limit.
+ *
+ * 4. startPrefix provided, prevKey provided:
+ * - Seeks to prevKey, skips it, and returns subsequent keys that match
startPrefix, up to the limit.
+ *
+ * If limit is 0, all matching keys are retrieved. If both startPrefix and
prevKey are empty, the method starts
+ * from the beginning of the table.
+ */
+ public static <T> Map<String, T> extractKeysFromTable(
+ Table<String, T> table, String startPrefix, int limit, String prevKey)
+ throws IOException {
+
+ Map<String, T> matchedKeys = new LinkedHashMap<>();
+ try (TableIterator<String, ? extends Table.KeyValue<String, T>> keyIter =
table.iterator()) {
+
+ // Scenario 1 & 4: prevKey is provided (whether startPrefix is empty or
not)
+ if (!prevKey.isEmpty()) {
+ keyIter.seek(prevKey);
+ if (keyIter.hasNext()) {
+ // Skip the previous key record
+ keyIter.next();
+ }
+ } else if (!startPrefix.isEmpty()) {
+ // Scenario 3: startPrefix is provided but prevKey is empty, so seek
to startPrefix
+ keyIter.seek(startPrefix);
+ }
+ // Scenario 2: Both startPrefix and prevKey are empty (iterate from the
start of the table)
+ // No seeking needed; just start iterating from the first record in the
table
+ // This is implicit in the following loop, as the iterator will start
from the beginning
+
+ // Iterate through the keys while adhering to the limit (if the limit is
not zero)
+ while (keyIter.hasNext() && (limit == 0 || matchedKeys.size() < limit)) {
Review Comment:
IMO, for `limit == 0`, number of records returned should be zero
irrespective of startPrefix and prevKey...We should keep consistency across
ozone for limit meaning. For limit == -1 , we should return all records.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/OMDBInsightSearchEndpoint.java:
##########
@@ -386,4 +343,20 @@ private KeyEntityInfo
createKeyEntityInfoFromOmKeyInfo(String dbKey,
return keyEntityInfo;
}
+ private boolean validateStartPrefixAndLimit(String startPrefix) {
Review Comment:
1. We are not validating limit here as method name says.
2. Similar method is duplicated in OMDBInsightEndPoint class also. Can we
optimize here ?
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/OMDBInsightEndpoint.java:
##########
@@ -476,17 +427,95 @@ public Response getDeletedKeySummary() {
@GET
@Path("/deletePending")
public Response getDeletedKeyInfo(
- @DefaultValue(DEFAULT_FETCH_COUNT) @QueryParam(RECON_QUERY_LIMIT)
- int limit,
- @DefaultValue(StringUtils.EMPTY) @QueryParam(RECON_QUERY_PREVKEY)
- String prevKey) {
- KeyInsightInfoResponse
- deletedKeyInsightInfo = new KeyInsightInfoResponse();
- getPendingForDeletionKeyInfo(limit, prevKey,
- deletedKeyInsightInfo);
+ @DefaultValue(DEFAULT_FETCH_COUNT) @QueryParam(RECON_QUERY_LIMIT) int
limit,
+ @DefaultValue(StringUtils.EMPTY) @QueryParam(RECON_QUERY_PREVKEY) String
prevKey,
+ @DefaultValue(StringUtils.EMPTY) @QueryParam(RECON_QUERY_START_PREFIX)
String startPrefix) {
+
+ // Initialize the response object to hold the key information
+ KeyInsightInfoResponse deletedKeyInsightInfo = new
KeyInsightInfoResponse();
+
+ // Ensure the limit is non-negative
+ limit = Math.max(0, limit);
+
+ boolean keysFound = false;
+
+ try {
+ if (isNotBlank(startPrefix)) {
+ // Validate and apply prefix-based search
+ if (!validateStartPrefix(startPrefix)) {
+ return createBadRequestResponse("Invalid startPrefix: Path must be
at the bucket level or deeper.");
+ }
+ keysFound = getPendingForDeletionKeyInfo(limit, prevKey, startPrefix,
deletedKeyInsightInfo);
+ } else {
Review Comment:
We can do some optimization here. We may not need else here and just pass
startPrefix as it is after validation if startPrefix is valid or not. Because
whether startPrefix is empty or not, those cases are being handled in
`extractKeysFromTable` method already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
