hansva commented on code in PR #9574:
URL: https://github.com/apache/hop/pull/9574#discussion_r4614856200


##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/RestConnectionEditor.java:
##########
@@ -1512,6 +1575,18 @@ public void setWidgetsContent() {
       wProxyPassword.setText(Const.NVL(metadata.getProxyPassword(), ""));
       wNonProxyHosts.setText(Const.NVL(metadata.getNonProxyHosts(), ""));
 
+      wHeaders.removeAll();
+      if (metadata.getHeaders() != null) {
+        for (RestConnectionHeader header : metadata.getHeaders()) {
+          TableItem item = new TableItem(wHeaders.table, SWT.NONE);
+          item.setText(1, Const.NVL(header.getName(), ""));

Review Comment:
   **NPE on a null header entry.** `resolveStaticHeaders` skips null entries, 
but this loop dereferences `header` directly. With `"headers":[null]` in a 
hand-edited or partially loaded JSON, the connection runs fine but the editor 
throws, so the connection can't be edited. Please skip nulls here too.



##########
plugins/transforms/httppost/src/main/java/org/apache/hop/pipeline/transforms/httppost/HttpPost.java:
##########
@@ -142,6 +142,10 @@ Object[] callHttpPOST(Object[] rowData) throws 
HopException {
       }
       // add HttpPost headers
       addHttpHeaders(post, rowData);
+      // The connection's static headers come after the transform's own: a 
header set here wins.
+      if (data.restAuthenticator != null) {
+        data.restAuthenticator.applyStaticHeaders(post, uri.toString());

Review Comment:
   **The same request-decoration code is copied into Http, HttpPost and 
ActionHttp.** Each copy does: apply the auth headers into a temp map, 
`setHeader` each entry, then `applyStaticHeaders`. In each copy the static 
headers are applied at a different point; here it's a separate block with its 
own null check, in the other two it's inside `addConnectionAuthentication`. A 
single `RestAuthenticator.applyTo(HttpMessage request, String uri)` would 
remove the copies, and would also be a natural place for the 401 renewal (see 
my comment on `Http.java`).



##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/client/RestAuthenticator.java:
##########
@@ -72,6 +113,100 @@ public void applyRequestHeaders(Map<String, String> 
headers, String requestUri)
     }
   }
 
+  /**
+   * Adds the connection's static headers (issue #9551) that the map does not 
already carry.
+   *
+   * @param headers the outbound header map, modified in place
+   * @param requestUri the URI this request is going to
+   */
+  public void applyStaticHeaders(Map<String, String> headers, String 
requestUri) {
+    if (headers == null || !hasStaticHeaders() || 
!targetMatchesOrigin(requestUri)) {
+      return;
+    }
+    for (Map.Entry<String, String> header : 
settings.getStaticHeaders().entrySet()) {
+      if (!headerAlreadySupplied(headers, header.getKey())) {

Review Comment:
   **A static `Accept` header on the connection is always dropped.** The REST 
transform puts its default `Accept` (from `data.mediaType`) into `headerMap` 
(`Rest.java` ~line 392), before the static headers are applied, so 
`headerAlreadySupplied` is true and the connection's `Accept: 
application/vnd.api+json` is skipped. HTTP POST does the same with its default 
`Content-Type: text/xml` via `addHeadersContentType`. The editor says only a 
header "set on the transform or by an input row" wins, so the user gets no 
indication. Please let a static header beat the transform's *default*: only a 
header the user set explicitly should win.



##########
plugins/transforms/http/src/main/java/org/apache/hop/pipeline/transforms/http/Http.java:
##########
@@ -312,17 +312,19 @@ private CloseableHttpClient httpClient(HttpHost target) {
   /**
    * Bearer, API-key and preemptive Basic authentication are request headers 
rather than answers to
    * a challenge, so the connection writes them onto every request itself. 
They stay scoped to the
-   * connection's base URL: an absolute URL on another host is sent without 
them.
+   * connection's base URL: an absolute URL on another host is sent without 
them. The connection's
+   * static headers follow the same rule, but a header this transform already 
set wins.
    */
   private void addConnectionAuthentication(HttpGet method, URI uri) {
     if (data.restAuthenticator == null) {
       return;
     }
     Map<String, String> authHeaders = new LinkedHashMap<>();
-    data.restAuthenticator.applyRequestHeaders(authHeaders, uri.toString());
+    data.restAuthenticator.applyAuthenticationHeaders(authHeaders, 
uri.toString());
     for (Map.Entry<String, String> authHeader : authHeaders.entrySet()) {
       method.setHeader(authHeader.getKey(), authHeader.getValue());
     }
+    data.restAuthenticator.applyStaticHeaders(method, uri.toString());

Review Comment:
   **The 401 token renewal only covers the REST transform and the connection 
test.** This transform, HTTP POST and the HTTP action now use the same OAuth 2 
REST connection, but none of them calls `refreshAfterUnauthorized` / 
`invalidate`. When the provider revokes the token early (another client fetched 
a new one), every row here gets a 401 until the cached expiry passes, which is 
exactly the #9550 bug. I'd prefer one fix in the shared client layer, e.g. an 
exec-chain interceptor in `RestClientFactory` or a shared `RestAuthenticator` 
execute helper, so all four callers get it.



##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/RestConnectionEditor.java:
##########
@@ -666,6 +686,49 @@ public void widgetSelected(SelectionEvent e) {
     enableControls(controls);
   }
 
+  /** Builds the table of headers sent on every request (issue #9551) on the 
Headers tab. */
+  private void buildHeadersWidgets(Composite p) {
+    Label wlInfo = new Label(p, SWT.LEFT | SWT.WRAP);
+    wlInfo.setText(BaseMessages.getString(PKG, 
"RestConnectionEditor.Headers.Info"));
+    PropsUi.setLook(wlInfo);
+    FormData fdlInfo = new FormData();
+    fdlInfo.left = new FormAttachment(0, 0);
+    fdlInfo.right = new FormAttachment(100, 0);
+    fdlInfo.top = new FormAttachment(0, margin);
+    wlInfo.setLayoutData(fdlInfo);
+
+    ColumnInfo nameColumn =
+        new ColumnInfo(
+            BaseMessages.getString(PKG, 
"RestConnectionEditor.Headers.Column.Name"),
+            ColumnInfo.COLUMN_TYPE_TEXT,
+            false,
+            false);
+    nameColumn.setUsingVariables(true);
+    ColumnInfo valueColumn =

Review Comment:
   **Header values are plain text.** The typical use for static headers is 
something credential-like (`x-api-key`, `Authorization: Basic …`). The values 
are shown unmasked and saved as typed into 
`metadata/rest-connection/<name>.json`, while the connection's other secrets 
use `PasswordTextVar` and are encrypted on save. `resolveStaticHeaders` already 
understands `Encrypted …` values. Could the editor encrypt values on save (or 
at least offer a "secret" flag per header and mask those)?



##########
plugins/transforms/rest/src/main/java/org/apache/hop/pipeline/transforms/rest/Rest.java:
##########
@@ -523,6 +532,82 @@ protected RestExchangeResult invokeRestExchange(
     // The client is not closed here: it is shared by every row and released 
on dispose().
   }
 
+  /**
+   * Sends the request, and when it comes back 401 on an OAuth 2 token Hop 
applied, sends it once
+   * more with a new token (issue #9550). Without this a token the server 
stopped accepting before
+   * its expiry time — revoked, rotated, or replaced because another client 
asked for one — keeps
+   * being sent until that time passes, and adding 401 to the retry status 
codes only resends it.
+   *
+   * <p>It runs inside {@link #executeWithRetry}, so the new token is in place 
before any configured
+   * retry. A streaming request reports a failed status as an exception rather 
than a response; a
+   * 401 there is just as safe to resend, because no record has gone 
downstream yet.
+   */
+  private HttpExchange sendRenewingRejectedToken(
+      Supplier<HttpExchange> send,
+      Map<String, String> headers,
+      String requestUri,
+      boolean[] tokenRefreshed) {
+    HttpExchange exchange;
+    try {
+      exchange = send.get();
+    } catch (HopRuntimeException e) {
+      if (isUnauthorizedStream(e) && renewRejectedToken(headers, requestUri, 
tokenRefreshed)) {
+        return send.get();
+      }
+      throw e;
+    }
+    if (exchange.status() == HttpStatus.SC_UNAUTHORIZED
+        && renewRejectedToken(headers, requestUri, tokenRefreshed)) {
+      return send.get();

Review Comment:
   **A resent request is counted twice.** `send.get()` goes through 
`executeRequest` again, which calls `trackRequestBytes` again. A 10 MB POST 
resent once after a 401 reports 20 MB of request bytes for one row. The 
counters should stay exact: count the resend separately, or don't count it.



##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/RestConnection.java:
##########
@@ -407,30 +439,61 @@ private boolean authTypeEquals(String canonicalLabel) {
         && authType.trim().equalsIgnoreCase(canonicalLabel.trim());
   }
 
-  /** Performs a GET against the URL with this connection's authentication, 
returning the body. */
+  /**
+   * Performs a GET against the URL with this connection's authentication and 
static headers,
+   * returning the body. An OAuth 2 token the server rejects with a 401 is 
replaced and the request
+   * sent once more (issue #9550).
+   */
   public String getResponse(String url) throws HopException {
     RestClientSettings settings = createClientSettings();
+    // Scoped to the URL rather than to the base URL: see 
applyAuthentication().
+    RestClientSettings requestSettings = createClientSettings();
+    requestSettings.setAuthOrigin(url);
+    RestAuthenticator authenticator = new RestAuthenticator(requestSettings);
     Map<String, String> headers = new LinkedHashMap<>();
-    applyAuthentication(headers, url);
-
-    HttpUriRequestBase request = new HttpUriRequestBase("GET", 
URI.create(url));
-    headers.forEach(request::addHeader);
+    authenticator.applyRequestHeaders(headers, url);
 
     try (CloseableHttpClient httpClient = 
RestClientFactory.createClient(settings)) {
-      return httpClient.execute(
-          request,
-          response -> {
-            if (response.getCode() != HttpStatus.SC_OK) {
-              throw new IOException("Error connecting to " + url + ": " + 
response.getCode());
-            }
-            HttpEntity entity = response.getEntity();
-            return entity == null ? "" : EntityUtils.toString(entity);
-          });
+      Response response = get(httpClient, url, headers);
+      if (response.status() == HttpStatus.SC_UNAUTHORIZED
+          && authenticator.refreshAfterUnauthorized(headers, url)) {
+        getLog().logBasic("The server rejected the OAuth 2 token; retrying 
with a new token.");

Review Comment:
   **Hard-coded English log messages.** This log line, and the `Error 
connecting to …` text below it, aren't translatable. `Rest.java` logs the same 
event through the `Rest.Log.OAuth2TokenRejectedRetrying` key. Please use 
`BaseMessages` here too.



##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/RestConnectionEditor.java:
##########
@@ -666,6 +686,49 @@ public void widgetSelected(SelectionEvent e) {
     enableControls(controls);
   }
 
+  /** Builds the table of headers sent on every request (issue #9551) on the 
Headers tab. */
+  private void buildHeadersWidgets(Composite p) {
+    Label wlInfo = new Label(p, SWT.LEFT | SWT.WRAP);
+    wlInfo.setText(BaseMessages.getString(PKG, 
"RestConnectionEditor.Headers.Info"));
+    PropsUi.setLook(wlInfo);
+    FormData fdlInfo = new FormData();
+    fdlInfo.left = new FormAttachment(0, 0);
+    fdlInfo.right = new FormAttachment(100, 0);
+    fdlInfo.top = new FormAttachment(0, margin);
+    wlInfo.setLayoutData(fdlInfo);
+
+    ColumnInfo nameColumn =
+        new ColumnInfo(
+            BaseMessages.getString(PKG, 
"RestConnectionEditor.Headers.Column.Name"),
+            ColumnInfo.COLUMN_TYPE_TEXT,
+            false,
+            false);
+    nameColumn.setUsingVariables(true);
+    ColumnInfo valueColumn =
+        new ColumnInfo(
+            BaseMessages.getString(PKG, 
"RestConnectionEditor.Headers.Column.Value"),
+            ColumnInfo.COLUMN_TYPE_TEXT,
+            false,
+            false);
+    valueColumn.setUsingVariables(true);
+
+    wHeaders =
+        new TableView(

Review Comment:
   **The Headers tab is hand-laid out.** It's built with FormAttachment and a 
free-standing `TableView`. Our dialog convention is annotated fields in a 
grouped `GuiCompositeWidgets` layout, with a `TableView` added through 
`GuiCompositeWidgets.registerExtraGroup(...)` rather than laid out by hand. The 
editor was already hand-built, so this can be a follow-up, but I'd rather not 
add new hand-laid tabs.



##########
plugins/misc/rest/src/main/java/org/apache/hop/metadata/rest/RestConnection.java:
##########
@@ -407,30 +439,61 @@ private boolean authTypeEquals(String canonicalLabel) {
         && authType.trim().equalsIgnoreCase(canonicalLabel.trim());
   }
 
-  /** Performs a GET against the URL with this connection's authentication, 
returning the body. */
+  /**
+   * Performs a GET against the URL with this connection's authentication and 
static headers,
+   * returning the body. An OAuth 2 token the server rejects with a 401 is 
replaced and the request
+   * sent once more (issue #9550).
+   */
   public String getResponse(String url) throws HopException {
     RestClientSettings settings = createClientSettings();
+    // Scoped to the URL rather than to the base URL: see 
applyAuthentication().
+    RestClientSettings requestSettings = createClientSettings();

Review Comment:
   **`createClientSettings()` runs twice per `getResponse`.** Each call 
resolves the variables again, resolves and decrypts the static headers, and 
rebuilds the SSL context, which loads the keystore and truststore from disk. So 
a connection test with client certificates loads both stores twice. Derive 
`requestSettings` from `settings` (copy + `setAuthOrigin(url)`) instead.
   
   Related: the public `applyAuthentication` now has no callers. `getResponse` 
re-implements this same sequence inline (`createClientSettings` + 
`setAuthOrigin(url)` + `new RestAuthenticator` + apply headers), so there are 
two copies of the URL-scoped auth logic. A future change to origin scoping has 
to be made in both. Please extract a private helper both use, or drop this 
method.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to