This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 08bffc38ed obfuscate marketplace repository passwords in
hop-config.json. fixes #7806 (#7828)
08bffc38ed is described below
commit 08bffc38ed50ebd402427cb781ed23ffae76f33a
Author: Bart Maertens <[email protected]>
AuthorDate: Fri Aug 7 13:57:08 2026 +0200
obfuscate marketplace repository passwords in hop-config.json. fixes #7806
(#7828)
---
.../ROOT/pages/hop-tools/hop-marketplace.adoc | 25 +++-
plugins/misc/marketplace/README.md | 24 ++++
.../marketplace/command/MarketplaceCommand.java | 4 +-
.../hop/marketplace/config/MarketplaceHttp.java | 12 ++
.../marketplace/config/MarketplaceRepository.java | 19 ++-
.../config/MarketplaceRepositoryDefinition.java | 8 +-
.../hop/marketplace/config/MarketplaceSecrets.java | 124 ++++++++++++++++
.../hop/marketplace/env/HopEnvironmentLoader.java | 3 +-
.../hop/marketplace/env/HopEnvironmentSpec.java | 10 ++
.../hop/marketplace/gui/HopEnvironmentDialog.java | 4 +
.../gui/MarketplaceRepositoriesPanel.java | 11 +-
.../gui/messages/messages_en_US.properties | 2 +
.../config/RepositoryPasswordStorageTest.java | 159 +++++++++++++++++++++
13 files changed, 391 insertions(+), 14 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
index 556c1dda09..93af602592 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
@@ -362,7 +362,7 @@ repositories:
# - id: corporate
# url: "https://nexus.example.com/repository/hop-plugins/";
# username: reader
- # password: "…" # prefer env HOP_MARKETPLACE_PASSWORD for secrets
+ # password: "${CORPORATE_TOKEN}" # stored obfuscated; prefer a variable
or env HOP_MARKETPLACE_PASSWORD
# Required optional plugins (installed into plugins/)
plugins:
@@ -457,6 +457,29 @@ Credentials are resolved most-specific first:
Use the repository-scoped variables when more than one private repository is
configured, since the global pair cannot express two different tokens.
+Username and password are resolved separately, so the two can come from
different sources: a repository entry that sets only `username` is combined
with `HOP_MARKETPLACE_PASSWORD` from the environment. Such a mixed pair counts
as deliberately configured — the anonymous retry described below applies only
when the repository entry supplies neither field.
+
+=== How stored passwords are protected
+
+A `password` on a repository entry is written to `hop-config.json` — and to
exported repository definitions and environment files — obfuscated with Hop's
two-way password encoder, the same `Encrypted 2be98afc…` form used for Hop
server and database passwords. Configurations written by older versions are
read as-is and re-encoded the next time they are saved.
+
+Obfuscation is not encryption: the encoder key is not a secret, so anyone
holding the file and a Hop install can recover the password. It stops
credentials being read over a shoulder, in a screenshot or in a diff — nothing
more.
+
+To keep the secret out of the file altogether, put a variable or a variable
resolver expression in `username` / `password` instead of the credential:
+
+[source,json]
+----
+{ "id": "acme", "url": "https://nexus.example.org/repository/hop/";,
"username": "reader", "password": "${ACME_TOKEN}" }
+----
+
+Variables are stored exactly as typed — there is nothing secret to obfuscate —
and are resolved when the repository is contacted, so the secret lives in the
variable rather than in the file.
+
+Marketplace repositories are client-wide rather than project-bound, so
`${...}` resolves against client-level variables only: JVM system properties
(`-DACME_TOKEN=…`, for example through `HOP_OPTIONS`) and variables described
in `hop-config.json`. Project and environment variables are not visible here,
and neither are plain shell environment variables — for those, use the
`HOP_MARKETPLACE_*` names above. A xref:variables.adoc[variable resolver]
expression such as `#{vault:hop/marketplace [...]
+
+A variable that is not set is sent to the server unresolved, and the resulting
401 says so.
+
+The `HOP_MARKETPLACE_*` environment variables above remain the simplest option
when nothing at all should be written to the configuration.
+
Because the global variables are offered to every repository — including
public ones such as the ASF repository and Maven Central, which reject unknown
credentials with HTTP 401 instead of ignoring them — a request that fails with
401 or 403 using credentials that came _only_ from the environment is retried
once anonymously. Credentials configured on the repository entry are treated as
deliberate and are never dropped, so those failures are reported as-is.
The same credentials apply to `hop marketplace repo import <https URL>`, so a
shared repository definition can be hosted in a private repository. No
repository entry exists at import time, so only the environment variables are
available there.
diff --git a/plugins/misc/marketplace/README.md
b/plugins/misc/marketplace/README.md
index cc2b709dec..78849ab41c 100644
--- a/plugins/misc/marketplace/README.md
+++ b/plugins/misc/marketplace/README.md
@@ -194,6 +194,30 @@ Credentials resolve most-specific first:
Use the scoped form when more than one private repository is configured; the
global pair cannot express two different tokens.
+Username and password resolve separately: an entry with only `username` set is
+combined with `HOP_MARKETPLACE_PASSWORD` from the environment. A mixed pair
+counts as deliberate — the anonymous retry below applies only when the entry
+supplies neither field.
+
+A `password` on a repository entry is stored obfuscated (`Encrypted 2be98afc…`,
+Hop's two-way password encoder) in `hop-config.json`, in exported repository
+definitions and in environment files. Older clear-text configurations are read
+as-is and re-encoded on the next save. Obfuscation is not encryption — the key
+is public — so to keep the secret out of the file use a variable instead:
+
+```json
+{ "id": "acme", "url": "https://nexus.example.org/repository/hop/";,
"password": "${ACME_TOKEN}" }
+```
+
+Variables and variable resolver expressions are stored as typed and resolved
+when the repository is contacted. Repositories are client-wide, so `${...}`
+resolves against client-level variables only: JVM system properties (`-D…`,
+e.g. via `HOP_OPTIONS`) and variables described in `hop-config.json` — not
+project/environment variables, and not plain shell exports (use the
+`HOP_MARKETPLACE_*` names for those). `#{resolver:…}` expressions go through
the
+active metadata. One that is not set reaches the server unresolved; the 401
+message says so.
+
The global pair is offered to **every** repository, including public ones, and
ASF/Central answer unknown credentials with 401 rather than ignoring them. So a
request that fails with 401/403 using credentials that came *only* from the
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/command/MarketplaceCommand.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/command/MarketplaceCommand.java
index fa5507f51f..2b12aab654 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/command/MarketplaceCommand.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/command/MarketplaceCommand.java
@@ -675,7 +675,9 @@ public class MarketplaceCommand implements Runnable,
IHopCommand, IHasHopMetadat
@Option(
names = {"--password"},
- description = "Optional Basic auth password (prefer env
HOP_MARKETPLACE_PASSWORD)")
+ description =
+ "Optional Basic auth password, stored obfuscated (prefer a
${VARIABLE} or env"
+ + " HOP_MARKETPLACE_PASSWORD)")
private String password;
@Option(
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
index 7e0d7e7df1..528a6778b8 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
@@ -156,11 +156,23 @@ public final class MarketplaceHttp {
+ repository.environmentIdPrefix()
+ "_USERNAME / _PASSWORD if this repository needs different ones.";
}
+ if (unresolved(repository.effectiveUsername()) ||
unresolved(repository.effectivePassword())) {
+ return ". The credentials configured for this repository use a variable
that is not set, so"
+ + " the expression itself was sent as the credential. Define the
variable, or set"
+ + " HOP_MARKETPLACE_"
+ + repository.environmentIdPrefix()
+ + "_USERNAME / _PASSWORD instead.";
+ }
return ". Basic auth was sent as user '"
+ repository.effectiveUsername()
+ "' from the repository configuration; check that password.";
}
+ /** A credential that still carries variable syntax never made it through
resolution. */
+ private static boolean unresolved(String credential) {
+ return credential != null && (credential.contains("${") ||
credential.contains("#{"));
+ }
+
private static void close(Object body) {
if (body instanceof Closeable closeable) {
try {
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
index 6d73d488be..300899fe42 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
@@ -17,6 +17,8 @@
package org.apache.hop.marketplace.config;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -56,10 +58,14 @@ public class MarketplaceRepository {
private String username;
/**
- * Optional HTTP Basic auth password. Prefer leaving this empty in
hop-config.json and setting
- * {@code HOP_MARKETPLACE_PASSWORD} instead for private repos. Do not set
for anonymous ASF /
- * Central / local Nexus.
+ * Optional HTTP Basic auth password, held in plain form in memory and
obfuscated in
+ * hop-config.json (see {@link MarketplaceSecrets}). Obfuscation is not
encryption, so for private
+ * repos prefer a variable ({@code ${MY_TOKEN}}), a variable resolver
expression, or {@code
+ * HOP_MARKETPLACE_PASSWORD} — those keep the secret out of the file
entirely. Do not set for
+ * anonymous ASF / Central / local Nexus.
*/
+ @JsonSerialize(using = MarketplaceSecrets.Serializer.class)
+ @JsonDeserialize(using = MarketplaceSecrets.Deserializer.class)
private String password;
/**
@@ -206,10 +212,13 @@ public class MarketplaceRepository {
* <p>Scoped variables exist so several private repositories can each have
their own token; the
* global pair applies to every repository, which is why {@link
#credentialsFromEnvironmentOnly()}
* exists to let a rejected global credential fall back to anonymous.
+ *
+ * <p>Credentials configured on the entry may be variables or variable
resolver expressions; they
+ * are resolved here, at use, so the config file only ever holds the
reference.
*/
public String effectiveUsername() {
if (StringUtils.isNotBlank(username)) {
- return username;
+ return MarketplaceSecrets.resolve(username);
}
return firstNonBlank(
scopedEnv("USERNAME"),
@@ -220,7 +229,7 @@ public class MarketplaceRepository {
public String effectivePassword() {
if (StringUtils.isNotBlank(password)) {
- return password;
+ return MarketplaceSecrets.resolve(password);
}
return firstNonBlank(scopedEnv("PASSWORD"),
env("HOP_MARKETPLACE_PASSWORD"));
}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
index fca58a11c7..950f0a320b 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
@@ -41,7 +41,8 @@ import org.yaml.snakeyaml.constructor.SafeConstructor;
/**
* Shareable Hop marketplace repository definition ({@code
hop-marketplace-repo.yaml}). Passwords
- * are never written on export.
+ * are never written on export unless explicitly asked for, and then only
obfuscated (see {@link
+ * MarketplaceSecrets}).
*/
public final class MarketplaceRepositoryDefinition {
@@ -160,7 +161,8 @@ public final class MarketplaceRepositoryDefinition {
// password intentionally ignored on import unless present (discouraged)
String password = stringVal(map.get("password"));
if (StringUtils.isNotBlank(password)) {
- repo.setPassword(password);
+ // Obfuscated when exported by Hop, clear text when hand-written: both
are accepted.
+ repo.setPassword(MarketplaceSecrets.decode(password));
}
repo.setBrowse(boolVal(map.get("browse"), false));
repo.setCatalogUrl(stringVal(map.get("catalogUrl")));
@@ -263,7 +265,7 @@ public final class MarketplaceRepositoryDefinition {
root.put("username", repo.getUsername());
}
if (includePassword && StringUtils.isNotBlank(repo.getPassword())) {
- root.put("password", repo.getPassword());
+ root.put("password", MarketplaceSecrets.encode(repo.getPassword()));
}
root.put("browse", repo.isBrowse());
if (StringUtils.isNotBlank(repo.getCatalogUrl())) {
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceSecrets.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceSecrets.java
new file mode 100644
index 0000000000..b78238b317
--- /dev/null
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceSecrets.java
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.DeserializationContext;
+import com.fasterxml.jackson.databind.JsonDeserializer;
+import com.fasterxml.jackson.databind.JsonSerializer;
+import com.fasterxml.jackson.databind.SerializerProvider;
+import java.io.IOException;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hop.core.encryption.Encr;
+import org.apache.hop.core.encryption.HopTwoWayPasswordEncoder;
+import org.apache.hop.core.encryption.ITwoWayPasswordEncoder;
+import org.apache.hop.core.variables.Variables;
+
+/**
+ * Password handling for marketplace repository credentials.
+ *
+ * <p>Repository passwords used to be written to hop-config.json in clear
text. They now pass
+ * through Hop's two-way password encoder on the way out ({@code Encrypted
2be98afc...}) and back
+ * through it on the way in, the same way Hop server and database passwords
are handled.
+ *
+ * <p>This is obfuscation, not encryption: the encoder key is not a secret, so
anyone with the
+ * config file and a Hop install can recover the password. It keeps
credentials from being readable
+ * over a shoulder, in a screenshot or in a diff — nothing more. Point the
password at a variable
+ * ({@code ${MY_TOKEN}}), a variable resolver expression, or one of the {@code
HOP_MARKETPLACE_*}
+ * environment variables when the secret must not live in the file at all.
+ */
+public final class MarketplaceSecrets {
+
+ private MarketplaceSecrets() {}
+
+ /**
+ * Used when no encoder is bound yet — {@link Encr} stays uninitialized
until {@code
+ * HopClientEnvironment.init()} runs, and config can be read before that.
Falling back keeps the
+ * value obfuscated instead of failing or silently writing clear text.
+ */
+ private static final ITwoWayPasswordEncoder FALLBACK = new
HopTwoWayPasswordEncoder();
+
+ /**
+ * Obfuscated form for storage. Values that use variables are stored as
typed: both shipped
+ * encoders leave them alone so the expression survives to be resolved at
use.
+ */
+ public static String encode(String password) {
+ if (StringUtils.isEmpty(password)) {
+ return password;
+ }
+ return encoder().encode(password, true);
+ }
+
+ /**
+ * Plain form of a stored value. Anything without the encoder's prefix is
returned unchanged, so
+ * configs written before this was added keep working.
+ */
+ public static String decode(String password) {
+ if (StringUtils.isEmpty(password)) {
+ return password;
+ }
+ return encoder().decode(password, true);
+ }
+
+ /**
+ * Resolve variables and variable resolver expressions in a credential.
Unresolvable expressions
+ * come back unchanged — that reaches the server as a bad password, which
{@link
+ * MarketplaceHttp#authHint} explains.
+ */
+ public static String resolve(String value) {
+ if (StringUtils.isEmpty(value) || !value.contains("{")) {
+ return value;
+ }
+ try {
+ return Variables.getADefaultVariableSpace().resolve(value);
+ } catch (Exception e) {
+ // Resolvers reach out to metadata and secret managers; a broken one
must not break the
+ // repository. Sending the expression unresolved fails visibly with an
auth hint.
+ return value;
+ }
+ }
+
+ private static ITwoWayPasswordEncoder encoder() {
+ ITwoWayPasswordEncoder active = Encr.getEncoder();
+ return active == null ? FALLBACK : active;
+ }
+
+ /** Writes the obfuscated form of a password field. */
+ public static class Serializer extends JsonSerializer<String> {
+ @Override
+ public void serialize(String value, JsonGenerator generator,
SerializerProvider provider)
+ throws IOException {
+ String encoded = encode(value);
+ if (encoded == null) {
+ generator.writeNull();
+ } else {
+ generator.writeString(encoded);
+ }
+ }
+ }
+
+ /** Reads a password field, obfuscated or (legacy) clear text. */
+ public static class Deserializer extends JsonDeserializer<String> {
+ @Override
+ public String deserialize(JsonParser parser, DeserializationContext
context)
+ throws IOException {
+ return decode(parser.getValueAsString());
+ }
+ }
+}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentLoader.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentLoader.java
index ee6cfa22a6..b4b83dcb64 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentLoader.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentLoader.java
@@ -31,6 +31,7 @@ import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.json.HopJson;
+import org.apache.hop.marketplace.config.MarketplaceSecrets;
import org.yaml.snakeyaml.DumperOptions;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
@@ -132,7 +133,7 @@ public final class HopEnvironmentLoader {
m.put("username", ref.getUsername());
}
if (StringUtils.isNotBlank(ref.getPassword())) {
- m.put("password", ref.getPassword());
+ m.put("password", MarketplaceSecrets.encode(ref.getPassword()));
}
repos.add(m);
}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentSpec.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentSpec.java
index 1efee5f64c..8e17e18c86 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentSpec.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/HopEnvironmentSpec.java
@@ -17,10 +17,13 @@
package org.apache.hop.marketplace.env;
+import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.util.ArrayList;
import java.util.List;
import lombok.Getter;
import lombok.Setter;
+import org.apache.hop.marketplace.config.MarketplaceSecrets;
/**
* Declarative Hop environment file ({@code hop-env.yaml} / {@code
hop-env.json}).
@@ -60,6 +63,13 @@ public class HopEnvironmentSpec {
/** Optional Basic auth username (prefer env HOP_MARKETPLACE_PASSWORD for
secrets). */
private String username;
+ /**
+ * Optional Basic auth password, obfuscated in the environment file the
same way as in
+ * hop-config.json. Environment files are meant to be shared, so prefer a
variable or {@code
+ * HOP_MARKETPLACE_PASSWORD} over a password here.
+ */
+ @JsonSerialize(using = MarketplaceSecrets.Serializer.class)
+ @JsonDeserialize(using = MarketplaceSecrets.Deserializer.class)
private String password;
}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/HopEnvironmentDialog.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/HopEnvironmentDialog.java
index dec6c41b1a..c5bec81e73 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/HopEnvironmentDialog.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/HopEnvironmentDialog.java
@@ -1212,6 +1212,10 @@ public class HopEnvironmentDialog extends Dialog {
Text wPass = new Text(dialog, SWT.SINGLE | SWT.LEFT | SWT.BORDER |
SWT.PASSWORD);
PropsUi.setLook(wPass);
wPass.setText(Const.NVL(ref.getPassword(), ""));
+ String passwordTooltip =
+ BaseMessages.getString(PKG,
"HopEnvironmentDialog.Repo.Column.Password.Tooltip");
+ wlPass.setToolTipText(passwordTooltip);
+ wPass.setToolTipText(passwordTooltip);
FormData fdPass = new FormData();
fdPass.left = new FormAttachment(middle, 0);
fdPass.top = new FormAttachment(wlPass, 0, SWT.CENTER);
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
index 61e00d4483..4fa7f4ceb2 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
@@ -447,6 +447,11 @@ public class MarketplaceRepositoriesPanel {
wlPass.setLayoutData(fdlPass);
Text wPass = new Text(general, SWT.SINGLE | SWT.LEFT | SWT.BORDER |
SWT.PASSWORD);
PropsUi.setLook(wPass);
+ wPass.setText(Const.NVL(repo.getPassword(), ""));
+ String passwordTooltip =
+ BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.Password.Tooltip");
+ wlPass.setToolTipText(passwordTooltip);
+ wPass.setToolTipText(passwordTooltip);
FormData fdPass = new FormData();
fdPass.left = new FormAttachment(middle, margin);
fdPass.top = new FormAttachment(wlPass, 0, SWT.CENTER);
@@ -725,9 +730,9 @@ public class MarketplaceRepositoriesPanel {
repo.setName(wName.getText().trim());
repo.setUrl(wUrl.getText().trim());
repo.setUsername(StringUtils.trimToNull(wUser.getText()));
- if (StringUtils.isNotBlank(wPass.getText())) {
- repo.setPassword(wPass.getText());
- }
+ // Empty means no password: clearing the field removes a stored
credential, which is the
+ // only way back to environment-only credentials for this repository.
+ repo.setPassword(StringUtils.trimToNull(wPass.getText()));
repo.setEnabled(wEnabled.getSelection());
repo.setPrimary(wPrimary.getSelection());
repo.setBrowse(wBrowse.getSelection());
diff --git
a/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
b/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
index 960f21ea8d..c5c2dca2ff 100644
---
a/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
+++
b/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
@@ -170,6 +170,7 @@ ManageRepositoriesDialog.Edit.Name=Name
ManageRepositoriesDialog.Edit.Url=URL
ManageRepositoriesDialog.Edit.Username=Username
ManageRepositoriesDialog.Edit.Password=Password
+ManageRepositoriesDialog.Edit.Password.Tooltip=Empty means no password: clear
the field to fall back to the HOP_MARKETPLACE_<ID>_PASSWORD environment
variable. Stored obfuscated in hop-config.json, which is not encryption: to
keep the secret out of the file use a variable like $'{'MY_TOKEN'}'.
ManageRepositoriesDialog.Edit.Enabled=Enabled (use in fallback chain)
ManageRepositoriesDialog.Edit.Primary=Primary (tried first)
ManageRepositoriesDialog.Edit.Browse=Browse (list zips in query / Plugins tab)
@@ -248,6 +249,7 @@ HopEnvironmentDialog.Repo.Column.Id=Id
HopEnvironmentDialog.Repo.Column.Url=URL
HopEnvironmentDialog.Repo.Column.Username=Username
HopEnvironmentDialog.Repo.Column.Password=Password
+HopEnvironmentDialog.Repo.Column.Password.Tooltip=Stored obfuscated in the
environment file, which is not encryption. Environment files are meant to be
shared: prefer a variable like $'{'MY_TOKEN'}' or the
HOP_MARKETPLACE_<ID>_PASSWORD environment variable.
HopEnvironmentDialog.Repo.Column.Auth=Auth
HopEnvironmentDialog.Repo.Edit.Add=Add repository
HopEnvironmentDialog.Repo.Edit.Edit=Edit repository
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryPasswordStorageTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryPasswordStorageTest.java
new file mode 100644
index 0000000000..6fb2aa7c26
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryPasswordStorageTest.java
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.hop.core.json.HopJson;
+import org.apache.hop.marketplace.env.HopEnvironmentSpec;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Passwords must not reach hop-config.json (or an exported definition) in
clear text, and whatever
+ * form they are stored in must survive a round trip.
+ */
+class RepositoryPasswordStorageTest {
+
+ private static final String PLAIN = "s3cret-token";
+
+ @AfterEach
+ void restoreEnvironment() {
+ MarketplaceRepository.setEnvironmentForTesting(null);
+ System.clearProperty("MARKETPLACE_TEST_TOKEN");
+ }
+
+ /** The password as it would appear in hop-config.json. */
+ @SuppressWarnings("unchecked")
+ private static String storedPassword(MarketplaceConfig config) {
+ Map<String, Object> map = HopJson.newMapper().convertValue(config,
Map.class);
+ List<Map<String, Object>> repos = (List<Map<String, Object>>)
map.get("repositories");
+ return (String) repos.get(0).get("password");
+ }
+
+ private static MarketplaceConfig configWithPassword(String password) {
+ MarketplaceConfig config = new MarketplaceConfig();
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme", "https://example.org/";, "user",
password);
+ config.setRepositories(new java.util.ArrayList<>(List.of(repo)));
+ return config;
+ }
+
+ @Test
+ void passwordIsObfuscatedInTheConfigFile() {
+ String stored = storedPassword(configWithPassword(PLAIN));
+ assertNotEquals(PLAIN, stored);
+ assertFalse(stored.contains(PLAIN), "clear text password found in stored
config: " + stored);
+ assertTrue(stored.startsWith("Encrypted "), "expected an encoded password,
got: " + stored);
+ }
+
+ @Test
+ void obfuscatedPasswordSurvivesASaveLoadRoundTrip() {
+ Map<String, Object> saved =
+ HopJson.newMapper().convertValue(configWithPassword(PLAIN), Map.class);
+ MarketplaceConfig reloaded = HopJson.newMapper().convertValue(saved,
MarketplaceConfig.class);
+
+ MarketplaceRepository repo = reloaded.findRepository("acme");
+ assertEquals(PLAIN, repo.getPassword());
+ assertEquals(PLAIN, repo.effectivePassword());
+ }
+
+ @Test
+ void clearTextPasswordsFromOlderConfigsStillLoad() {
+ // Written by a Hop version that had no obfuscation: read it as-is,
re-save it encoded.
+ Map<String, Object> legacy =
+ Map.of(
+ "repositories",
+ List.of(Map.of("id", "acme", "url", "https://example.org/";,
"password", PLAIN)));
+
+ MarketplaceConfig loaded = HopJson.newMapper().convertValue(legacy,
MarketplaceConfig.class);
+ assertEquals(PLAIN, loaded.findRepository("acme").getPassword());
+ assertTrue(storedPassword(loaded).startsWith("Encrypted "));
+ }
+
+ @Test
+ void variablesAreStoredAsTypedAndResolvedWhenUsed() {
+ MarketplaceConfig config = configWithPassword("${MARKETPLACE_TEST_TOKEN}");
+ // A variable is a reference, not a secret: it stays readable in the file.
+ assertEquals("${MARKETPLACE_TEST_TOKEN}", storedPassword(config));
+
+ MarketplaceRepository repo = config.findRepository("acme");
+ System.setProperty("MARKETPLACE_TEST_TOKEN", PLAIN);
+ assertEquals(PLAIN, repo.effectivePassword());
+ assertTrue(repo.hasCredentials());
+ }
+
+ @Test
+ void unresolvedVariablesAreExplainedOnAuthFailure() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme", "https://example.org/";, "user",
"${NO_SUCH_VARIABLE}");
+ // Nothing to resolve to, so the expression is passed through rather than
silently emptied.
+ assertEquals("${NO_SUCH_VARIABLE}", repo.effectivePassword());
+ assertTrue(MarketplaceHttp.authHint(401, repo).contains("variable that is
not set"));
+ }
+
+ @Test
+ void exportedDefinitionsCarryAnObfuscatedPassword() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme", "https://example.org/";, "user",
PLAIN);
+
+ Map<String, Object> withoutPassword =
MarketplaceRepositoryDefinition.toYamlMap(repo, false);
+ assertFalse(withoutPassword.containsKey("password"));
+
+ Map<String, Object> withPassword =
MarketplaceRepositoryDefinition.toYamlMap(repo, true);
+
assertTrue(String.valueOf(withPassword.get("password")).startsWith("Encrypted
"));
+ }
+
+ @Test
+ void importedDefinitionsAcceptBothForms() throws Exception {
+ Map<String, Object> encoded =
+ Map.of(
+ "id",
+ "acme",
+ "url",
+ "https://example.org/";,
+ "password",
+ MarketplaceSecrets.encode(PLAIN));
+ assertEquals(PLAIN,
MarketplaceRepositoryDefinition.fromMap(encoded).getPassword());
+
+ // Hand-written definitions are clear text; they must import unchanged.
+ Map<String, Object> plain =
+ Map.of("id", "acme", "url", "https://example.org/";, "password", PLAIN);
+ assertEquals(PLAIN,
MarketplaceRepositoryDefinition.fromMap(plain).getPassword());
+ }
+
+ @Test
+ void environmentFilesObfuscatePasswordsToo() {
+ HopEnvironmentSpec.RepositoryRef ref = new
HopEnvironmentSpec.RepositoryRef();
+ ref.setId("acme");
+ ref.setUrl("https://example.org/";);
+ ref.setPassword(PLAIN);
+
+ Map<String, Object> written = HopJson.newMapper().convertValue(ref,
Map.class);
+ assertTrue(String.valueOf(written.get("password")).startsWith("Encrypted
"));
+
+ HopEnvironmentSpec.RepositoryRef read =
+ HopJson.newMapper().convertValue(written,
HopEnvironmentSpec.RepositoryRef.class);
+ assertEquals(PLAIN, read.getPassword());
+ }
+}
