cgivre commented on a change in pull request #2270:
URL: https://github.com/apache/drill/pull/2270#discussion_r692953886
##########
File path:
contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/util/SimpleHttp.java
##########
@@ -322,6 +336,99 @@ private void setupCache(Builder builder) {
return formBodyBuilder;
}
+ /**
+ * Returns the URL-decoded URL
+ *
+ * @return Returns the URL-decoded URL
+ */
+ public static String decodedURL(HttpUrl url) {
+ try {
+ return URLDecoder.decode(url.toString(), "UTF-8");
+ } catch (UnsupportedEncodingException e) {
+ return url.toString();
+ }
+ }
+
+ /**
+ * Returns true if the url has url parameters, as indicated by the presence
of
+ * {param} in a url.
+ *
+ * @return True if there are URL params, false if not
+ */
+ public static boolean hasURLParameters(HttpUrl url) {
+ String decodedUrl = SimpleHttp.decodedURL(url);
+ Matcher matcher = URL_PARAM_REGEX.matcher(decodedUrl);
+ return matcher.find();
+ }
+
+ /**
+ * APIs sometimes are structured with parameters in the URL itself. For
instance, to request a list of
+ * an organization's repositories in github, the URL is:
https://api.github.com/orgs/{org}/repos, where
+ * you can replace the org with the actual organization name.
+ *
+ * @return A list of URL parameters enclosed by curly braces.
+ */
+ public static List<String> getURLParameters(HttpUrl url) {
+ List<String> parameters = new ArrayList<>();
+ String decodedURL;
+ try {
+ decodedURL = URLDecoder.decode(url.toString(), "UTF-8");
+ } catch (UnsupportedEncodingException e) {
+ return null;
+ }
+ Matcher matcher = URL_PARAM_REGEX.matcher(decodedURL);
+ String param;
+ while (matcher.find()) {
+ param = matcher.group();
+ param = param.replace("{", "");
+ param = param.replace("}", "");
+ parameters.add(param);
+ }
+ return parameters;
+ }
+
+ /**
+ * Used for APIs which have parameters in the URL. This function maps the
filters pushed down
+ * from the query into the URL. For example the API:
github.com/orgs/{org}/repos requires a user to
+ * specify an organization and replace {org} with an actual organization.
The filter is passed down from
+ * the query.
+ *
+ * Note that if a URL contains URL parameters and one is not provided in the
filters, Drill will throw
+ * a UserException.
+ *
+ * @param url The HttpUrl containing URL Parameters
+ * @param filters: A HashMap of filters
+ * @return A string of the URL with the URL parameters replaced by filter
values
+ */
+ public static String mapURLParameters(HttpUrl url, Map<String, String>
filters) {
+ if (! hasURLParameters(url)) {
+ return url.toString();
+ }
+
+ List<String> params = SimpleHttp.getURLParameters(url);
+ String tempUrl = SimpleHttp.decodedURL(url);
+ for (String param : params) {
+ if (filters == null) {
+ throw UserException
+ .parseError()
+ .message("API Query with URL Parameters must be populated. Parameter
" + param + " must be included in WHERE clause.")
+ .build(logger);
+ }
+
+ String value = filters.get(param);
+ // If the param is not populated, throw an exception
+ if (Strings.isNullOrEmpty(value)) {
Review comment:
Fixed in the context of adding defaults.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
