This is an automated email from the ASF dual-hosted git repository.
mattcasters pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new d46d56d9ad Scope stored marketplace credentials to their own
repository, fixes #8393 (#8398)
d46d56d9ad is described below
commit d46d56d9ad67fe3a6a889469149202d89885acf2
Author: Bart Maertens <[email protected]>
AuthorDate: Wed Sep 16 10:41:14 2026 +0200
Scope stored marketplace credentials to their own repository, fixes #8393
(#8398)
A project's install spec can declare its own plugin repository. When the
declaration omitted a username or password, EnvironmentApplier filled them
in
from the globally configured repository while taking the URL from the
project,
so enabling a project that named an unconfigured repository sent the
operator's
credentials to it.
Configured credentials are now only reused when the spec URL points at the
same
scheme, host and port as a configured repository, so a second path on the
same
repository manager keeps working. A spec repository with no configured
counterpart also opts out of the global HOP_MARKETPLACE_USERNAME/_PASSWORD
pair,
which reaches a project-chosen host by the same route; the repository-scoped
HOP_MARKETPLACE_<ID>_* variables still apply.
---
.../ROOT/pages/hop-tools/hop-marketplace.adoc | 11 +-
plugins/misc/marketplace/README.md | 8 +
.../marketplace/config/MarketplaceRepository.java | 74 ++++++++-
.../hop/marketplace/env/EnvironmentApplier.java | 72 +++++++--
.../marketplace/config/RepositoryOriginTest.java | 129 ++++++++++++++++
.../marketplace/env/EnvironmentApplierTest.java | 109 +++++++++++++
...stallSpecRepositoryCredentialScopeHttpTest.java | 172 +++++++++++++++++++++
7 files changed, 556 insertions(+), 19 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
index 0af25f8abe..ebb13f8e77 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
@@ -435,7 +435,7 @@ dependencies:
| `repositories`
| No
-| List of Maven base URLs used when *applying* this file. Order: first =
primary, rest = fallbacks. Each item: `id`, `url`, optional `username` /
`password`.
+| List of Maven base URLs used when *applying* this file. Order: first =
primary, rest = fallbacks. Each item: `id`, `url`, optional `username` /
`password`. A repository here only inherits configured credentials when it
points at a repository manager that is already configured — see
<<marketplace-spec-repository-credentials>>.
| `plugins`
| No
@@ -491,6 +491,15 @@ Use the repository-scoped variables when more than one
private repository is con
Username and password are resolved separately, so the two can come from
different sources: a repository entry that sets only `username` is combined
with `HOP_MARKETPLACE_PASSWORD` from the environment. Such a mixed pair counts
as deliberately configured — the anonymous retry described below applies only
when the repository entry supplies neither field.
+[[marketplace-spec-repository-credentials]]
+=== Repositories declared by an install spec
+
+An install spec carried by a project can declare its own `repositories`.
Because that URL comes from the project rather than from the operator, the
configured credentials are only reused for it when the project points at the
*same* repository manager they belong to — the same scheme, host and port as a
repository in `hop-config`. The path may differ, so a second repository on the
same Nexus or Artifactory keeps working.
+
+A spec repository on any other host gets only what the spec itself declares.
Neither the `username` / `password` stored on a configured repository nor the
global `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD` pair is sent to
it, so enabling a project cannot make Hop hand the operator's repository
credentials to an address of the project's choosing. The log says so at basic
level when credentials are withheld this way.
+
+To give such a repository credentials, set `username` / `password` on its
entry in the spec (prefer a variable over a literal secret), or set the
repository-scoped `HOP_MARKETPLACE_<ID>_USERNAME` / `_PASSWORD` variables for
the `id` the spec uses — those name one repository, so they still apply.
+
=== Authentication type
`authType` on a repository chooses how those credentials are sent:
diff --git a/plugins/misc/marketplace/README.md
b/plugins/misc/marketplace/README.md
index c7de8614ec..79b08d2209 100644
--- a/plugins/misc/marketplace/README.md
+++ b/plugins/misc/marketplace/README.md
@@ -199,6 +199,14 @@ combined with `HOP_MARKETPLACE_PASSWORD` from the
environment. A mixed pair
counts as deliberate — the anonymous retry below applies only when the entry
supplies neither field.
+Repositories declared by a hop-env install spec are scoped: because the URL
+comes from the project, the configured credentials are only reused when the
+spec points at the same scheme, host and port as a repository in
+`hop-config.json` (the path may differ). A spec repository on any other host
+gets only what the spec declares — neither a stored `username` / `password` nor
+the global `HOP_MARKETPLACE_*` pair reaches it. Give it credentials with its
own
+entry fields or the scoped `HOP_MARKETPLACE_<ID>_*` variables.
+
A `password` on a repository entry is stored obfuscated (`Encrypted 2be98afc…`,
Hop's two-way password encoder) in `hop-config.json`, in exported repository
definitions and in environment files. Older clear-text configurations are read
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
index 604f42b764..9eddec0fe9 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
@@ -17,8 +17,11 @@
package org.apache.hop.marketplace.config;
+import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import java.net.URI;
+import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -170,6 +173,18 @@ public class MarketplaceRepository {
/** Optional human description (documentation / export). */
private String description;
+ /**
+ * When false, the global {@code HOP_MARKETPLACE_USERNAME} / {@code
HOP_MARKETPLACE_PASSWORD}
+ * variables do not apply to this repository; only credentials set on the
entry itself or in the
+ * repository-scoped {@code HOP_MARKETPLACE_<ID>_*} variables are used.
+ *
+ * <p>Set false for a repository an install spec introduced that is not one
the operator
+ * configured: the global pair is meant for the operator's own repositories,
and a project must
+ * not be able to have it sent to a host of the project's choosing. Not
persisted — it follows
+ * from where the repository came from, not from configuration.
+ */
+ @JsonIgnore private boolean globalEnvironmentCredentials = true;
+
public MarketplaceRepository() {
// Jackson
}
@@ -213,6 +228,54 @@ public class MarketplaceRepository {
return url.endsWith("/") ? url : url + "/";
}
+ /**
+ * Scheme, host and port of {@link #normalizedUrl()} — the identity a
credential belongs to —
+ * lower-cased and with the default port of the scheme made explicit, or
{@code null} when the URL
+ * is not an absolute URL with a host. The path is deliberately left out:
several repositories on
+ * one repository manager share one set of credentials.
+ */
+ public String origin() {
+ return originOf(normalizedUrl());
+ }
+
+ /** See {@link #origin()}. Returns {@code null} for a blank, relative or
unparseable URL. */
+ public static String originOf(String repositoryUrl) {
+ if (StringUtils.isBlank(repositoryUrl)) {
+ return null;
+ }
+ try {
+ URI uri = new URI(repositoryUrl.trim());
+ String scheme = uri.getScheme();
+ String host = uri.getHost();
+ if (StringUtils.isAnyBlank(scheme, host)) {
+ return null;
+ }
+ scheme = scheme.toLowerCase(Locale.ROOT);
+ int port = uri.getPort() < 0 ? defaultPort(scheme) : uri.getPort();
+ return scheme + "://" + host.toLowerCase(Locale.ROOT) + (port < 0 ? "" :
":" + port);
+ } catch (URISyntaxException e) {
+ return null;
+ }
+ }
+
+ private static int defaultPort(String scheme) {
+ return switch (scheme) {
+ case "https" -> 443;
+ case "http" -> 80;
+ default -> -1;
+ };
+ }
+
+ /**
+ * True when {@code otherUrl} points at the same scheme, host and port as
this repository, which
+ * is the condition for this repository's credentials to be safe to use for
it. An unparseable URL
+ * on either side is never a match: no origin means no credentials rather
than a guess.
+ */
+ public boolean sameOriginAs(String otherUrl) {
+ String mine = origin();
+ return mine != null && mine.equals(originOf(otherUrl));
+ }
+
/**
* Environment lookup, replaceable in tests. Package-private on purpose:
credential resolution is
* otherwise untestable, and it is the part most likely to go subtly wrong.
@@ -238,6 +301,11 @@ public class MarketplaceRepository {
return id.trim().toUpperCase(Locale.ROOT).replaceAll("[^A-Z0-9]", "_");
}
+ /** The global {@code HOP_MARKETPLACE_*} pair, unless this repository opted
out of it. */
+ private String globalEnv(String name) {
+ return globalEnvironmentCredentials ? env(name) : null;
+ }
+
private String scopedEnv(String suffix) {
String prefix = environmentIdPrefix();
return StringUtils.isBlank(prefix) ? null : env("HOP_MARKETPLACE_" +
prefix + "_" + suffix);
@@ -262,8 +330,8 @@ public class MarketplaceRepository {
return firstNonBlank(
scopedEnv("USERNAME"),
scopedEnv("USER"),
- env("HOP_MARKETPLACE_USERNAME"),
- env("HOP_MARKETPLACE_USER"));
+ globalEnv("HOP_MARKETPLACE_USERNAME"),
+ globalEnv("HOP_MARKETPLACE_USER"));
}
public String effectivePassword() {
@@ -271,7 +339,7 @@ public class MarketplaceRepository {
return MarketplaceSecrets.resolve(password);
}
return firstNonBlank(
- scopedEnv("PASSWORD"), scopedEnv("TOKEN"),
env("HOP_MARKETPLACE_PASSWORD"));
+ scopedEnv("PASSWORD"), scopedEnv("TOKEN"),
globalEnv("HOP_MARKETPLACE_PASSWORD"));
}
/**
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/EnvironmentApplier.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/EnvironmentApplier.java
index 7320026067..d1a0bb7324 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/EnvironmentApplier.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/env/EnvironmentApplier.java
@@ -220,7 +220,8 @@ public class EnvironmentApplier {
}
}
- private MarketplaceConfig configFromEnv(HopInstallSpec env) {
+ /** Package-private so the credential scoping below can be asserted without
a live install. */
+ MarketplaceConfig configFromEnv(HopInstallSpec env) {
MarketplaceConfig config = new MarketplaceConfig();
config.setEnabled(baseConfig.isEnabled());
config.setGroupId(baseConfig.getGroupId());
@@ -229,24 +230,11 @@ public class EnvironmentApplier {
? env.getHopVersion()
: MarketplaceCommand.resolveDefaultVersion(baseConfig));
config.getRepositories().clear();
- MarketplaceRepository baseRepo = baseConfig.primaryRepository();
if (env.getRepositories() != null && !env.getRepositories().isEmpty()) {
boolean first = true;
for (HopInstallSpec.RepositoryRef ref : env.getRepositories()) {
if (StringUtils.isNotBlank(ref.getUrl())) {
- MarketplaceRepository repo =
- new MarketplaceRepository(
- StringUtils.defaultIfBlank(ref.getId(), "spec"),
- ref.getUrl(),
- StringUtils.isNotBlank(ref.getUsername())
- ? ref.getUsername()
- : baseRepo.getUsername(),
- StringUtils.isNotBlank(ref.getPassword())
- ? ref.getPassword()
- : baseRepo.getPassword());
- repo.setPrimary(first);
- repo.setEnabled(true);
- config.getRepositories().add(repo);
+ config.getRepositories().add(repositoryFromRef(ref, first));
first = false;
}
}
@@ -261,6 +249,60 @@ public class EnvironmentApplier {
return config;
}
+ /**
+ * Turn a repository the install spec declares into a marketplace
repository. The URL is the
+ * project's, so the configured credentials are only reused when the project
points at the same
+ * repository they belong to — same scheme, host and port. A project naming
a host the operator
+ * never configured gets what the project itself declared, or nothing.
+ */
+ private MarketplaceRepository repositoryFromRef(HopInstallSpec.RepositoryRef
ref, boolean first) {
+ MarketplaceRepository source = configuredCredentialSource(ref.getUrl());
+ MarketplaceRepository repo =
+ new MarketplaceRepository(
+ StringUtils.defaultIfBlank(ref.getId(), "spec"),
+ ref.getUrl(),
+ StringUtils.isNotBlank(ref.getUsername())
+ ? ref.getUsername()
+ : (source == null ? null : source.getUsername()),
+ StringUtils.isNotBlank(ref.getPassword())
+ ? ref.getPassword()
+ : (source == null ? null : source.getPassword()));
+ if (source == null) {
+ // The global HOP_MARKETPLACE_USERNAME / _PASSWORD pair belongs to the
operator's own
+ // repositories for the same reason; the repository-scoped variables
stay available so a
+ // project repository can still be given credentials without putting
them in the spec file.
+ repo.setGlobalEnvironmentCredentials(false);
+ }
+ repo.setPrimary(first);
+ repo.setEnabled(true);
+ return repo;
+ }
+
+ /**
+ * The configured repository whose credentials may be reused for {@code
url}, or null when none of
+ * them belongs to that origin. Install order, so the primary wins a tie.
+ */
+ private MarketplaceRepository configuredCredentialSource(String url) {
+ boolean anyStoredCredentials = false;
+ for (MarketplaceRepository repo : baseConfig.orderedRepositories()) {
+ if (StringUtils.isAllBlank(repo.getUsername(), repo.getPassword())) {
+ continue;
+ }
+ anyStoredCredentials = true;
+ if (repo.sameOriginAs(url)) {
+ return repo;
+ }
+ }
+ if (anyStoredCredentials) {
+ log.logBasic(
+ "Install spec repository "
+ + url
+ + " is not a configured marketplace repository: the configured
credentials are not"
+ + " sent to it");
+ }
+ return null;
+ }
+
private String resolveEnvVersion(HopInstallSpec env) {
if (StringUtils.isNotBlank(env.getHopVersion())) {
return env.getHopVersion();
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryOriginTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryOriginTest.java
new file mode 100644
index 0000000000..d287ad1ebd
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryOriginTest.java
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.Map;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+/**
+ * The origin comparison that decides whether stored credentials belong to a
repository URL, and the
+ * opt-out from the global environment credentials that goes with it.
+ */
+class RepositoryOriginTest {
+
+ @AfterEach
+ void restoreEnvironment() {
+ MarketplaceRepository.setEnvironmentForTesting(null);
+ }
+
+ @Test
+ void originIsSchemeHostAndPortWithDefaultPortMadeExplicit() {
+ assertEquals(
+ "https://nexus.example.org:443";,
+
MarketplaceRepository.originOf("https://nexus.example.org/repository/hop/";));
+ assertEquals(
+ "http://nexus.example.org:80";,
MarketplaceRepository.originOf("http://nexus.example.org/";));
+ assertEquals(
+ "https://nexus.example.org:8443";,
+
MarketplaceRepository.originOf("https://nexus.example.org:8443/repository/hop/";));
+ }
+
+ @Test
+ void originIsCaseInsensitiveOnSchemeAndHost() {
+ assertEquals(
+
MarketplaceRepository.originOf("https://nexus.example.org/repository/hop/";),
+
MarketplaceRepository.originOf("HTTPS://Nexus.Example.ORG/repository/hop/"));
+ }
+
+ @Test
+ void originIsNullWhenThereIsNoHostToCompare() {
+ assertNull(MarketplaceRepository.originOf(null));
+ assertNull(MarketplaceRepository.originOf(" "));
+ assertNull(MarketplaceRepository.originOf("repository/hop/"));
+ assertNull(MarketplaceRepository.originOf("file:/var/repo/"));
+ assertNull(MarketplaceRepository.originOf("https://nexus example.org/"));
+ }
+
+ @Test
+ void sameOriginIgnoresThePath() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("nexus",
"https://nexus.example.org/repository/hop/";);
+
assertTrue(repo.sameOriginAs("https://nexus.example.org/repository/hop-private/";));
+
assertTrue(repo.sameOriginAs("https://nexus.example.org:443/repository/hop/";));
+ }
+
+ @Test
+ void sameOriginRejectsADifferentHostSchemeOrPort() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("nexus",
"https://nexus.example.org/repository/hop/";);
+ assertFalse(repo.sameOriginAs("https://evil.example.com/repository/hop/";));
+ assertFalse(repo.sameOriginAs("http://nexus.example.org/repository/hop/";));
+
assertFalse(repo.sameOriginAs("https://nexus.example.org:8443/repository/hop/";));
+ // A subdomain is a different host, not a relative of it.
+
assertFalse(repo.sameOriginAs("https://nexus.example.org.evil.com/repository/hop/";));
+ }
+
+ @Test
+ void anUnparseableUrlOnEitherSideNeverMatches() {
+ assertFalse(new MarketplaceRepository("nexus", "not a
url").sameOriginAs("not a url"));
+ assertFalse(
+ new MarketplaceRepository("nexus",
"https://nexus.example.org/";).sameOriginAs(" "));
+ }
+
+ @Test
+ void globalEnvironmentCredentialsCanBeSuppressedWhileScopedOnesStillApply() {
+ MarketplaceRepository.setEnvironmentForTesting(
+ Map.of(
+ "HOP_MARKETPLACE_USERNAME", "global-user",
+ "HOP_MARKETPLACE_PASSWORD", "global-pass",
+ "HOP_MARKETPLACE_SPEC_USERNAME", "spec-user",
+ "HOP_MARKETPLACE_SPEC_PASSWORD", "spec-pass")
+ ::get);
+
+ MarketplaceRepository foreign =
+ new MarketplaceRepository("other", "https://other.example.org/";);
+ foreign.setGlobalEnvironmentCredentials(false);
+ assertNull(foreign.effectiveUsername());
+ assertNull(foreign.effectivePassword());
+ assertFalse(foreign.hasCredentials());
+
+ // The repository-scoped pair names one repository, so it is still
honoured.
+ MarketplaceRepository spec = new MarketplaceRepository("spec",
"https://other.example.org/";);
+ spec.setGlobalEnvironmentCredentials(false);
+ assertEquals("spec-user", spec.effectiveUsername());
+ assertEquals("spec-pass", spec.effectivePassword());
+ }
+
+ @Test
+ void suppressionIsNotPersisted() throws Exception {
+ MarketplaceRepository repo = new MarketplaceRepository("spec",
"https://other.example.org/";);
+ repo.setGlobalEnvironmentCredentials(false);
+ String json =
org.apache.hop.core.json.HopJson.newMapper().writeValueAsString(repo);
+ assertFalse(json.contains("globalEnvironmentCredentials"));
+ assertTrue(
+ org.apache.hop.core.json.HopJson.newMapper()
+ .readValue(json, MarketplaceRepository.class)
+ .isGlobalEnvironmentCredentials());
+ }
+}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/EnvironmentApplierTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/EnvironmentApplierTest.java
index 00d474a35e..871b9e807c 100644
---
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/EnvironmentApplierTest.java
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/EnvironmentApplierTest.java
@@ -19,6 +19,7 @@ package org.apache.hop.marketplace.env;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.nio.file.Files;
@@ -27,6 +28,7 @@ import org.apache.hop.core.json.HopJson;
import org.apache.hop.core.logging.HopLogStore;
import org.apache.hop.core.logging.LogChannel;
import org.apache.hop.marketplace.config.MarketplaceConfig;
+import org.apache.hop.marketplace.config.MarketplaceRepository;
import org.apache.hop.marketplace.install.InstallReceipt;
import org.apache.hop.marketplace.install.PluginInstaller;
import org.junit.jupiter.api.BeforeAll;
@@ -117,4 +119,111 @@ class EnvironmentApplierTest {
System.clearProperty("PROJECT_HOME");
}
}
+
+ private static MarketplaceConfig configuredWithPrivateRepository() {
+ MarketplaceConfig config = new MarketplaceConfig();
+ config.getRepositories().clear();
+ MarketplaceRepository repo =
+ new MarketplaceRepository(
+ "nexus", "https://nexus.example.org/repository/hop/";, "operator",
"operator-secret");
+ repo.setPrimary(true);
+ config.getRepositories().add(repo);
+ return config;
+ }
+
+ private static HopInstallSpec specWithRepository(String id, String url) {
+ HopInstallSpec env = new HopInstallSpec();
+ HopInstallSpec.RepositoryRef ref = new HopInstallSpec.RepositoryRef();
+ ref.setId(id);
+ ref.setUrl(url);
+ env.getRepositories().add(ref);
+ return env;
+ }
+
+ @Test
+ void projectRepositoryOnAnotherHostDoesNotGetTheConfiguredCredentials() {
+ MarketplaceConfig config =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
configuredWithPrivateRepository())
+ .configFromEnv(
+ specWithRepository("project",
"https://evil.example.com/repository/hop/";));
+
+ MarketplaceRepository applied = config.primaryRepository();
+ assertEquals("project", applied.getId());
+ assertNull(applied.getUsername());
+ assertNull(applied.getPassword());
+ assertFalse(applied.isGlobalEnvironmentCredentials());
+ }
+
+ @Test
+ void projectRepositoryOnTheSameHostStillInheritsThem() {
+ MarketplaceConfig config =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
configuredWithPrivateRepository())
+ .configFromEnv(
+ specWithRepository("project",
"https://nexus.example.org/repository/hop-extra/";));
+
+ MarketplaceRepository applied = config.primaryRepository();
+ assertEquals("operator", applied.getUsername());
+ assertEquals("operator-secret", applied.getPassword());
+ assertTrue(applied.isGlobalEnvironmentCredentials());
+ }
+
+ @Test
+ void aDifferentPortOrSchemeIsADifferentRepository() {
+ for (String url :
+ new String[] {
+ "https://nexus.example.org:8443/repository/hop/";,
+ "http://nexus.example.org/repository/hop/";
+ }) {
+ MarketplaceConfig config =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
configuredWithPrivateRepository())
+ .configFromEnv(specWithRepository("project", url));
+ assertNull(config.primaryRepository().getUsername(), url);
+ assertNull(config.primaryRepository().getPassword(), url);
+ }
+ }
+
+ @Test
+ void credentialsDeclaredByTheProjectAreAlwaysUsed() {
+ HopInstallSpec env = specWithRepository("project",
"https://other.example.com/repository/hop/";);
+ env.getRepositories().get(0).setUsername("project-user");
+ env.getRepositories().get(0).setPassword("project-secret");
+
+ MarketplaceRepository applied =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
configuredWithPrivateRepository())
+ .configFromEnv(env)
+ .primaryRepository();
+ assertEquals("project-user", applied.getUsername());
+ assertEquals("project-secret", applied.getPassword());
+ }
+
+ @Test
+ void credentialsComeFromTheMatchingRepositoryNotOnlyFromThePrimary() {
+ MarketplaceConfig config = new MarketplaceConfig();
+ config.getRepositories().clear();
+ MarketplaceRepository primary =
+ new MarketplaceRepository("asf",
"https://repository.apache.org/content/groups/public/";);
+ primary.setPrimary(true);
+ config.getRepositories().add(primary);
+ config
+ .getRepositories()
+ .add(
+ new MarketplaceRepository(
+ "acme", "https://nexus.example.org/repository/hop/";,
"acme-user", "acme-secret"));
+
+ MarketplaceRepository applied =
+ new EnvironmentApplier(new LogChannel("test"), tempDir, config)
+ .configFromEnv(
+ specWithRepository("project",
"https://nexus.example.org/repository/hop-extra/";))
+ .primaryRepository();
+ assertEquals("acme-user", applied.getUsername());
+ assertEquals("acme-secret", applied.getPassword());
+ }
+
+ @Test
+ void specWithoutRepositoriesKeepsTheConfiguredOnes() {
+ MarketplaceConfig config =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
configuredWithPrivateRepository())
+ .configFromEnv(new HopInstallSpec());
+ assertEquals("operator", config.primaryRepository().getUsername());
+ }
}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/InstallSpecRepositoryCredentialScopeHttpTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/InstallSpecRepositoryCredentialScopeHttpTest.java
new file mode 100644
index 0000000000..91c0b52549
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/env/InstallSpecRepositoryCredentialScopeHttpTest.java
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.env;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import com.sun.net.httpserver.HttpExchange;
+import com.sun.net.httpserver.HttpServer;
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.List;
+import org.apache.hop.core.logging.HopLogStore;
+import org.apache.hop.core.logging.LogChannel;
+import org.apache.hop.marketplace.config.MarketplaceConfig;
+import org.apache.hop.marketplace.config.MarketplaceRepository;
+import org.apache.hop.marketplace.resolve.MavenCoordinates;
+import org.apache.hop.marketplace.resolve.MavenRepositoryClient;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+/**
+ * End to end over real HTTP: what a download triggered by an install spec
actually puts on the
+ * wire. The unit tests assert which credentials are selected; these assert
that the selection is
+ * what reaches the server, which is the property issue #8393 is about.
+ *
+ * <p>Two loopback servers on different ports are two different origins, so
"the operator's
+ * repository" and "a repository the project names" can be told apart without
DNS.
+ */
+class InstallSpecRepositoryCredentialScopeHttpTest {
+
+ private static final MavenCoordinates COORDS =
+ new MavenCoordinates("org.apache.hop", "hop-tech-parquet", "2.19.0");
+
+ @TempDir private Path tempDir;
+
+ private HttpServer configured;
+ private HttpServer foreign;
+
+ /** Authorization headers seen by each server, in order. */
+ private final List<String> configuredAuth = new ArrayList<>();
+
+ private final List<String> foreignAuth = new ArrayList<>();
+
+ @BeforeAll
+ static void initLogging() {
+ HopLogStore.init();
+ }
+
+ @BeforeEach
+ void startServers() throws IOException {
+ configured = start(configuredAuth);
+ foreign = start(foreignAuth);
+ }
+
+ @AfterEach
+ void stopServers() {
+ configured.stop(0);
+ foreign.stop(0);
+ }
+
+ private static HttpServer start(List<String> authorizations) throws
IOException {
+ HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1",
0), 0);
+ server.createContext(
+ "/",
+ (HttpExchange exchange) -> {
+
authorizations.add(exchange.getRequestHeaders().getFirst("Authorization"));
+ byte[] body = "zip".getBytes(StandardCharsets.UTF_8);
+ exchange.sendResponseHeaders(200, body.length);
+ exchange.getResponseBody().write(body);
+ exchange.close();
+ });
+ server.start();
+ return server;
+ }
+
+ private static String url(HttpServer server, String path) {
+ return "http://127.0.0.1:"; + server.getAddress().getPort() + path;
+ }
+
+ /** hop-config with one private repository on the {@link #configured}
server. */
+ private MarketplaceConfig operatorConfig() {
+ MarketplaceConfig config = new MarketplaceConfig();
+ config.getRepositories().clear();
+ MarketplaceRepository repo =
+ new MarketplaceRepository(
+ "nexus", url(configured, "/repository/hop/"), "operator",
"operator-secret");
+ repo.setPrimary(true);
+ config.getRepositories().add(repo);
+ return config;
+ }
+
+ private HopInstallSpec specPointingAt(String repositoryUrl) {
+ HopInstallSpec spec = new HopInstallSpec();
+ HopInstallSpec.RepositoryRef ref = new HopInstallSpec.RepositoryRef();
+ ref.setId("project");
+ ref.setUrl(repositoryUrl);
+ spec.getRepositories().add(ref);
+ return spec;
+ }
+
+ private void downloadUsing(HopInstallSpec spec) throws Exception {
+ MarketplaceConfig applied =
+ new EnvironmentApplier(new LogChannel("test"), tempDir,
operatorConfig())
+ .configFromEnv(spec);
+ Path target = Files.createTempDirectory(tempDir,
"dl").resolve("plugin.zip");
+ new MavenRepositoryClient(new LogChannel("test"))
+ .downloadZip(applied.primaryRepository(), COORDS, target);
+ }
+
+ @Test
+ void aSpecPointingAtAnotherHostGetsNoAuthorizationHeader() throws Exception {
+ downloadUsing(specPointingAt(url(foreign, "/repository/hop/")));
+
+ assertEquals(1, foreignAuth.size(), "the foreign server must have served
the download");
+ assertNull(
+ foreignAuth.get(0), "operator credentials must not reach a repository
it never configured");
+ assertTrue(configuredAuth.isEmpty(), "the configured repository was not
contacted at all");
+ }
+
+ @Test
+ void aSpecPointingAtTheConfiguredRepositoryStillAuthenticates() throws
Exception {
+ downloadUsing(specPointingAt(url(configured, "/repository/hop-extra/")));
+
+ assertEquals(1, configuredAuth.size());
+ assertEquals(
+ "Basic "
+ + Base64.getEncoder()
+
.encodeToString("operator:operator-secret".getBytes(StandardCharsets.UTF_8)),
+ configuredAuth.get(0),
+ "a second path on the operator's own repository keeps working");
+ }
+
+ @Test
+ void aSpecMayCarryItsOwnCredentialsForAForeignHost() throws Exception {
+ HopInstallSpec spec = specPointingAt(url(foreign, "/repository/hop/"));
+ spec.getRepositories().get(0).setUsername("project-user");
+ spec.getRepositories().get(0).setPassword("project-secret");
+
+ downloadUsing(spec);
+
+ assertEquals(
+ "Basic "
+ + Base64.getEncoder()
+
.encodeToString("project-user:project-secret".getBytes(StandardCharsets.UTF_8)),
+ foreignAuth.get(0));
+ }
+}
