This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 80836bf749 marketplace: support Forgejo/Gitea registries and non-Maven
plugin hosts, fixes #7720 (#7724)
80836bf749 is described below
commit 80836bf74916bdee00d81c0c842e83db599ce4ca
Author: Bart Maertens <[email protected]>
AuthorDate: Mon Aug 3 08:54:49 2026 +0200
marketplace: support Forgejo/Gitea registries and non-Maven plugin hosts,
fixes #7720 (#7724)
Browsing only spoke the Sonatype Nexus search API and downloads only
understood
Maven layout, which ruled out Forgejo and Gitea package registries as well
as any
host serving plugin zips as release assets or static files. Private
repositories
were only half-supported: some requests authenticated, others did not.
Browsing
Adds ForgejoRepositoryBrowser against /api/v1/packages/{owner}?type=maven
and a
browserType discriminator on the repository: auto (default, detected from
the
URL), nexus or forgejo. Existing repositories are unaffected -- auto
resolves to
nexus for every URL that is not a package registry, and definitions written
before this change still load.
Forgejo names Maven packages groupId:artifactId, and its listing does not
say
whether a version ships a zip, so each candidate gets one /files call;
versions
without a zip are dropped rather than offered as installs that would 404.
That
lookup is capped and the cap is logged, never silent.
Downloads
Repositories can set a urlTemplate for hosts that are not Maven
repositories at
all -- git release assets, static file hosts, CDNs:
urlTemplate:
https://host/org/dist/releases/download/v${version}/${artifactId}-${version}.zip
Placeholders: ${groupId}, ${groupPath}, ${artifactId}, ${version}. An
unrecognised placeholder is reported as an error rather than producing a
confusing 404. When set, the repository url is not used for plugin
downloads;
staging, receipts and uninstall are unchanged because none of them depended
on
the layout. Such hosts serve no Maven metadata, so versions must be exact
and
pinned by a catalogUrl.
Credentials
Catalog fetches and definition imports built bare requests, so a catalog or
a
shared repository definition hosted in a private repository always failed
with
401 however credentials were configured. All four request paths -- catalog,
browse, definition import and artifact download -- now go through one
helper,
MarketplaceHttp, which also removes a duplicated Basic-auth implementation.
HOP_MARKETPLACE_USERNAME/PASSWORD apply to every repository, so credentials
for a
private repository were also offered to the ASF repository and Maven
Central,
which answer unknown credentials with 401 instead of ignoring them:
configuring
one private repository broke installs from the public ones. A request that
fails
with 401/403 using credentials that came only from the environment is now
retried
once anonymously. Credentials set on the repository entry are deliberate
and are
never dropped. Repository-scoped variables (HOP_MARKETPLACE_<ID>_USERNAME)
let
several private repositories carry different tokens, which the global pair
cannot
express.
Two silent failures found while configuring a real catalog-backed
repository:
applyToConfig upserts an existing entry field by field and dropped
browserType
and urlTemplate on re-import, and a repository with a catalogUrl but
browse=false
contributes no plugins and no error. Both fixed, the latter with a log hint
naming the repository.
Settings are carried through YAML import/export and the Repositories tab,
and
documented in the plugin README and the user manual. Module suite at 102
tests.
---
.../ROOT/pages/hop-tools/hop-marketplace.adoc | 39 ++-
plugins/misc/marketplace/README.md | 69 ++++-
.../catalog/ForgejoRepositoryBrowser.java | 331 +++++++++++++++++++++
.../catalog/NexusRepositoryBrowser.java | 44 +--
.../hop/marketplace/catalog/PluginDiscovery.java | 16 +
.../marketplace/catalog/RemotePluginCatalog.java | 40 ++-
.../hop/marketplace/config/MarketplaceHttp.java | 185 ++++++++++++
.../marketplace/config/MarketplaceRepository.java | 114 ++++++-
.../config/MarketplaceRepositoryDefinition.java | 46 ++-
.../gui/MarketplaceRepositoriesPanel.java | 50 +++-
.../marketplace/resolve/MavenRepositoryClient.java | 70 +++--
.../gui/messages/messages_en_US.properties | 4 +
.../catalog/ForgejoRepositoryBrowserTest.java | 150 ++++++++++
.../MarketplaceRepositoryBrowserTypeTest.java | 168 +++++++++++
.../config/RepositoryCredentialsTest.java | 124 ++++++++
.../hop/marketplace/gui/MessagesFormatTest.java | 80 +++++
.../hop/marketplace/resolve/UrlTemplateTest.java | 78 +++++
17 files changed, 1511 insertions(+), 97 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
index e22f7be2a1..f02b57bf8c 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/hop-tools/hop-marketplace.adoc
@@ -440,7 +440,44 @@ A longer sample ships with the marketplace plugin sources
(`plugins/misc/marketp
== Enterprise / air-gapped
-Point marketplace repositories at your internal Nexus or Artifactory mirror of
`org.apache.hop` plugin zips. Prefer anonymous read where possible; for private
repos set Basic auth on the repository entry (in hop-config or in hop-env
`repositories`) or use `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD`.
Wrong credentials can cause HTTP 401 even when anonymous access would work.
+Point marketplace repositories at your internal Nexus or Artifactory mirror of
`org.apache.hop` plugin zips. Prefer anonymous read where possible; for private
repos set Basic auth on the repository entry (in hop-config or in hop-env
`repositories`) or use `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD`.
+
+Credentials are resolved most-specific first:
+
+. `username` / `password` on the repository entry
+. `HOP_MARKETPLACE_<ID>_USERNAME` / `_PASSWORD`, where `<ID>` is the
repository id upper-cased with non-alphanumeric characters replaced by `_`
(repository `local-nexus` reads `HOP_MARKETPLACE_LOCAL_NEXUS_USERNAME`)
+. `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD`, applying to every
repository
+
+Use the repository-scoped variables when more than one private repository is
configured, since the global pair cannot express two different tokens.
+
+Because the global variables are offered to every repository — including
public ones such as the ASF repository and Maven Central, which reject unknown
credentials with HTTP 401 instead of ignoring them — a request that fails with
401 or 403 using credentials that came _only_ from the environment is retried
once anonymously. Credentials configured on the repository entry are treated as
deliberate and are never dropped, so those failures are reported as-is.
+
+The same credentials apply to `hop marketplace repo import <https URL>`, so a
shared repository definition can be hosted in a private repository. No
repository entry exists at import time, so only the environment variables are
available there.
+
+Installing works against any Maven repository, because downloads use plain
Maven layout. Listing plugins (`browse: true`) needs a repository-manager API,
and those differ per product. Set `browserType` on the repository to choose:
+
+[options="header"]
+|===
+|Value|Endpoint used|Repository manager
+|`auto` (default)|detected from the URL|any of the below
+|`nexus`|`/service/rest/v1/search`|Sonatype Nexus 3
+|`forgejo`|`/api/v1/packages/{owner}`|Forgejo, Gitea
+|===
+
+`auto` selects `forgejo` for URLs containing `/api/packages/` (for example
`https://forge.example.org/api/packages/acme/maven`) and `nexus` otherwise. On
Forgejo, packages are named `groupId:artifactId` and each candidate version is
checked for a plugin zip, so plain library jars are not listed as installable.
+
+For a repository manager with no supported search API, publish a catalog YAML
and set `catalogUrl` on the repository instead — discovery then reads that file
and never calls a registry API. The catalog is fetched with the repository's
Basic credentials when they are configured, so it can live in a private
repository.
+
+Plugin zips do not have to be served from a Maven repository at all. Set
`urlTemplate` on the repository when they come from git release assets, a
static file host, or a CDN:
+
+[source,yaml]
+----
+urlTemplate:
https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip
+----
+
+Supported placeholders are `${groupId}`, `${groupPath}` (groupId with dots
replaced by slashes), `${artifactId}` and `${version}`; an unrecognised
placeholder is reported as an error instead of producing a confusing 404. When
`urlTemplate` is set, the repository `url` is not used for plugin downloads.
+
+Such a host serves no Maven metadata, so versions must be exact: combine
`urlTemplate` with a `catalogUrl` that pins each version. SNAPSHOT resolution
does not apply, and live browsing cannot list release assets.
== For plugin authors
diff --git a/plugins/misc/marketplace/README.md
b/plugins/misc/marketplace/README.md
index 89e54c4a9e..cc2b709dec 100644
--- a/plugins/misc/marketplace/README.md
+++ b/plugins/misc/marketplace/README.md
@@ -184,6 +184,26 @@ export HOP_MARKETPLACE_USERNAME=reader
export HOP_MARKETPLACE_PASSWORD='…'
```
+Credentials resolve most-specific first:
+
+1. `username` / `password` on the repository entry
+2. `HOP_MARKETPLACE_<ID>_USERNAME` / `_PASSWORD` — per repository, id
upper-cased
+ with non-alphanumerics as `_` (`local-nexus` →
`HOP_MARKETPLACE_LOCAL_NEXUS_*`)
+3. `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD` — all repositories
+
+Use the scoped form when more than one private repository is configured; the
+global pair cannot express two different tokens.
+
+The global pair is offered to **every** repository, including public ones, and
+ASF/Central answer unknown credentials with 401 rather than ignoring them. So a
+request that fails with 401/403 using credentials that came *only* from the
+environment is retried once anonymously. Credentials set on the repository
entry
+are deliberate and are never dropped this way — those failures surface as-is.
+
+Credentials also apply to `marketplace repo import <https URL>`, so a shared
+definition can live in a private repository. There is no repository entry yet
at
+that point, so only the environment variables above are used.
+
## GUI
Hop GUI → **Tools → Marketplace…**, or the main toolbar icon after **Save As…**
@@ -209,7 +229,7 @@ Three steps:
**How listing works**
- Bundled Apache optional plugins: **version = running Hop version** (not
absolute latest on ASF)
-- Every enabled repo with **`browse: true`**: live Nexus search for `*.zip`
(one row per artifact, latest plugin version, last updated)
+- Every enabled repo with **`browse: true`**: live repository search for
`*.zip` (one row per artifact, latest plugin version, last updated)
- Optional YAML `plugins:` metadata **enriches** names/categories and can set:
- `minHopVersion` / `maxHopVersion` — hide if this Hop is outside the range
(e.g. datavault needs ≥ 2.18.1). A running `x.y.z-SNAPSHOT` counts as the
`x.y.z` line (so `2.19.0-SNAPSHOT` fulfills `minHopVersion: 2.19.0`)
- `version` — optional pin of the plugin artifact; otherwise latest from
browse
@@ -225,6 +245,53 @@ GUI **Plugins** tab uses the same discovery as `query`.
Sample: `src/main/samples/hop-marketplace-repo.community.example.yaml`
+**Which browse API** — set per repository with `browserType`:
+
+| Value | Endpoint | Use for |
+|-------|----------|---------|
+| `auto` (default) | detected from the URL | anything |
+| `nexus` | `/service/rest/v1/search` | Sonatype Nexus 3 |
+| `forgejo` | `/api/v1/packages/{owner}` | Forgejo / Gitea package registry |
+
+`auto` picks `forgejo` when the URL contains `/api/packages/`, else `nexus`.
+Forgejo lists packages named `groupId:artifactId`; a second call per version
+confirms a plugin zip exists, so library jars are not offered as installs.
+Downloads never use this setting — they are plain Maven layout, which works
+against both.
+
+```yaml
+id: acme
+url: https://forge.example.org/api/packages/acme/maven
+browse: true
+browserType: forgejo # or omit and let auto detect it
+groupIdFilter: com.acme.hop
+```
+
+A private Forgejo registry needs credentials (owner name as username, an access
+token as password) via `HOP_MARKETPLACE_USERNAME` / `HOP_MARKETPLACE_PASSWORD`.
+
+Serving a `catalogUrl` YAML instead skips the registry API altogether and works
+against any static host. The catalog is fetched with the repository's Basic
+credentials when configured, so it may live in a private repository.
+
+**Non-Maven hosts** — set `urlTemplate` when plugin zips are served from
+something that is not a Maven repository (git release assets, a static file
+host, a CDN). It replaces layout resolution for downloads:
+
+```yaml
+id: acme
+urlTemplate:
https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip
+catalogUrl:
https://forge.example.org/acme/dist/releases/download/v2026.09/catalog.yaml
+browse: false
+```
+
+Placeholders: `${groupId}`, `${groupPath}` (dots → slashes), `${artifactId}`,
+`${version}`. An unknown placeholder is an error rather than a silent 404.
+
+Such a host has no Maven metadata, so versions must be exact — pair
`urlTemplate`
+with a `catalogUrl` that pins them. SNAPSHOT resolution does not apply, and
+`browse` cannot list release assets.
+
**Environment file** path + **Browse…** / **Edit…** / **Validate** / **Apply**
manage
declarative `hop-env.yaml` files. **Edit…** opens a tabbed editor (General,
Repositories, Plugins, Dependencies) with New/Open/Save/Save As, catalog
multi-add,
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowser.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowser.java
new file mode 100644
index 0000000000..4ee1001bda
--- /dev/null
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowser.java
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.catalog;
+
+import java.net.http.HttpClient;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hop.core.exception.HopException;
+import org.apache.hop.core.json.HopJson;
+import org.apache.hop.core.logging.ILogChannel;
+import org.apache.hop.marketplace.config.MarketplaceHttp;
+import org.apache.hop.marketplace.config.MarketplaceRepository;
+import org.apache.hop.marketplace.resolve.SnapshotVersions;
+
+/**
+ * Best-effort browse of a Forgejo (or Gitea) Maven package registry.
+ *
+ * <p>Forgejo has no Nexus-style search endpoint. It lists an owner's packages
at {@code
+ * /api/v1/packages/{owner}?type=maven}, where each entry carries a package
{@code name} of the form
+ * {@code groupId:artifactId} plus a {@code version}. Whether a version ships
an installable plugin
+ * zip is a second call to {@code .../files}, so entries without a zip (plain
library jars) are
+ * dropped instead of being offered as installs that would 404.
+ *
+ * <p>Downloads never come through here — they use plain Maven layout via
{@code
+ * MavenRepositoryClient}, which already works against Forgejo unchanged.
+ */
+public final class ForgejoRepositoryBrowser {
+
+ /** Marker separating the host from the package registry path. */
+ private static final String PACKAGES_MARKER = "/api/packages/";
+
+ static final int PAGE_SIZE = 50;
+ static final int MAX_PAGES = 50;
+
+ /**
+ * Upper bound on the per-version {@code /files} calls used to confirm a zip
exists. Reaching it
+ * is logged rather than silently truncating the list.
+ */
+ static final int MAX_ZIP_LOOKUPS = 200;
+
+ private ForgejoRepositoryBrowser() {}
+
+ /** One candidate package version, before the zip check. */
+ record Candidate(String groupId, String artifactId, String version, String
createdAt) {
+ String gaKey() {
+ return groupId.toLowerCase(Locale.ROOT) + ":" +
artifactId.toLowerCase(Locale.ROOT);
+ }
+
+ String packageName() {
+ return groupId + ":" + artifactId;
+ }
+ }
+
+ public static List<OptionalPluginInfo> browse(
+ MarketplaceRepository repository, String textFilter, ILogChannel log)
throws HopException {
+ return browse(repository, textFilter, log, MarketplaceHttp.newClient());
+ }
+
+ static List<OptionalPluginInfo> browse(
+ MarketplaceRepository repository, String textFilter, ILogChannel log,
HttpClient client)
+ throws HopException {
+ if (repository == null || StringUtils.isBlank(repository.getUrl())) {
+ return List.of();
+ }
+ String apiBase = extractApiBase(repository.getUrl());
+ String owner = extractOwner(repository.getUrl());
+ if (StringUtils.isAnyBlank(apiBase, owner)) {
+ throw new HopException(
+ "Cannot derive Forgejo host/owner from URL: "
+ + repository.getUrl()
+ + " (expected .../api/packages/{owner}/maven)");
+ }
+
+ // One row per groupId:artifactId — keep the newest version, like the
Nexus browser.
+ Map<String, Candidate> byGa = new LinkedHashMap<>();
+ Map<String, String> updatedByGa = new LinkedHashMap<>();
+
+ String query = StringUtils.firstNonBlank(textFilter,
repository.getSearchQuery());
+ for (int page = 1; page <= MAX_PAGES; page++) {
+ StringBuilder url = new StringBuilder();
+ url.append(apiBase)
+ .append("/api/v1/packages/")
+ .append(MarketplaceHttp.enc(owner))
+ .append("?type=maven&limit=")
+ .append(PAGE_SIZE)
+ .append("&page=")
+ .append(page);
+ if (StringUtils.isNotBlank(query)) {
+ url.append("&q=").append(MarketplaceHttp.enc(query.trim()));
+ }
+
+ List<Candidate> candidates =
+ parsePackagesPage(
+ MarketplaceHttp.getText(
+ client, url.toString(), repository, "Forgejo package
search"));
+ for (Candidate candidate : candidates) {
+ if (StringUtils.isNotBlank(repository.getGroupIdFilter())
+ && !repository.getGroupIdFilter().equals(candidate.groupId())) {
+ continue;
+ }
+ if (!repository.isIncludeSnapshots() &&
SnapshotVersions.isSnapshot(candidate.version())) {
+ continue;
+ }
+ String key = candidate.gaKey();
+ OptionalPluginInfo existing = asInfo(byGa.get(key),
updatedByGa.get(key));
+ if (!NexusRepositoryBrowser.isPreferable(
+ candidate.version(), candidate.createdAt(), existing)) {
+ continue;
+ }
+ byGa.put(key, candidate);
+ updatedByGa.put(key, candidate.createdAt());
+ }
+
+ if (candidates.size() < PAGE_SIZE) {
+ break;
+ }
+ if (page == MAX_PAGES && log != null) {
+ log.logBasic(
+ "Forgejo package listing stopped after "
+ + MAX_PAGES
+ + " pages for owner "
+ + owner
+ + "; some plugins may not be listed.");
+ }
+ }
+
+ List<OptionalPluginInfo> out = new ArrayList<>();
+ int lookups = 0;
+ for (Candidate candidate : byGa.values()) {
+ if (lookups >= MAX_ZIP_LOOKUPS) {
+ if (log != null) {
+ log.logBasic(
+ "Forgejo browse checked only the first "
+ + MAX_ZIP_LOOKUPS
+ + " packages for plugin zips; "
+ + (byGa.size() - lookups)
+ + " more were skipped. Narrow the search or set a
groupIdFilter.");
+ }
+ break;
+ }
+ lookups++;
+
+ String filesUrl =
+ apiBase
+ + "/api/v1/packages/"
+ + MarketplaceHttp.enc(owner)
+ + "/maven/"
+ + MarketplaceHttp.enc(candidate.packageName())
+ + "/"
+ + MarketplaceHttp.enc(candidate.version())
+ + "/files";
+ String zipFile;
+ try {
+ zipFile =
+ extractZipFileName(
+ MarketplaceHttp.getText(client, filesUrl, repository, "Forgejo
package files"));
+ } catch (HopException e) {
+ if (log != null) {
+ log.logDetailed(
+ "Skipping "
+ + candidate.packageName()
+ + ":"
+ + candidate.version()
+ + " — file list unavailable: "
+ + e.getMessage());
+ }
+ continue;
+ }
+ if (zipFile == null) {
+ // A jar-only artifact (shared library), not an installable Hop plugin.
+ continue;
+ }
+
+ OptionalPluginInfo info = new OptionalPluginInfo();
+ info.setGroupId(candidate.groupId());
+ info.setArtifactId(candidate.artifactId());
+ info.setVersion(candidate.version());
+ info.setName(candidate.artifactId());
+ info.setCategory("auto-discovered");
+ String path =
+ NexusRepositoryBrowser.mavenPath(
+ candidate.groupId(), candidate.artifactId(),
candidate.version())
+ + zipFile;
+ info.setDescription(path);
+ info.setInstallPath(path);
+ info.setLastUpdated(candidate.createdAt());
+ info.setSource(repository.getId());
+ out.add(info);
+ }
+
+ if (StringUtils.isNotBlank(query)) {
+ // Server-side q= is a partial name match only; refine over all fields.
+ out = RemotePluginCatalog.filter(out, repository, query);
+ }
+ return out;
+ }
+
+ /** Adapt a stored candidate for {@link NexusRepositoryBrowser#isPreferable}
comparison. */
+ private static OptionalPluginInfo asInfo(Candidate candidate, String
lastUpdated) {
+ if (candidate == null) {
+ return null;
+ }
+ OptionalPluginInfo info = new OptionalPluginInfo();
+ info.setVersion(candidate.version());
+ info.setLastUpdated(lastUpdated);
+ return info;
+ }
+
+ /** Parse one page of {@code /api/v1/packages/{owner}} into candidates. */
+ static List<Candidate> parsePackagesPage(String json) throws HopException {
+ if (StringUtils.isBlank(json)) {
+ return List.of();
+ }
+ List<?> items;
+ try {
+ items = HopJson.newMapper().readValue(json, List.class);
+ } catch (Exception e) {
+ throw new HopException("Unable to parse Forgejo package list JSON", e);
+ }
+ List<Candidate> out = new ArrayList<>();
+ for (Object item : items) {
+ if (!(item instanceof Map<?, ?> map)) {
+ continue;
+ }
+ String type = MarketplaceHttp.str(map.get("type"));
+ if (type != null && !"maven".equalsIgnoreCase(type)) {
+ continue;
+ }
+ String[] ga = splitPackageName(MarketplaceHttp.str(map.get("name")));
+ String version = MarketplaceHttp.str(map.get("version"));
+ if (ga == null || version == null) {
+ continue;
+ }
+ out.add(new Candidate(ga[0], ga[1], version,
MarketplaceHttp.str(map.get("created_at"))));
+ }
+ return out;
+ }
+
+ /**
+ * Forgejo stores a Maven package under the name {@code groupId:artifactId}.
Returns {@code
+ * {groupId, artifactId}}, or null when the name has no colon.
+ */
+ static String[] splitPackageName(String packageName) {
+ if (StringUtils.isBlank(packageName)) {
+ return null;
+ }
+ String name = packageName.trim();
+ int colon = name.lastIndexOf(':');
+ if (colon <= 0 || colon == name.length() - 1) {
+ return null;
+ }
+ return new String[] {name.substring(0, colon), name.substring(colon + 1)};
+ }
+
+ /**
+ * File name of the first plugin zip in a {@code .../files} response, or
null when the version has
+ * no zip (checksum sidecars are not files of their own in Forgejo, but are
excluded defensively).
+ */
+ static String extractZipFileName(String json) throws HopException {
+ if (StringUtils.isBlank(json)) {
+ return null;
+ }
+ List<?> files;
+ try {
+ files = HopJson.newMapper().readValue(json, List.class);
+ } catch (Exception e) {
+ throw new HopException("Unable to parse Forgejo package file list JSON",
e);
+ }
+ for (Object file : files) {
+ if (!(file instanceof Map<?, ?> map)) {
+ continue;
+ }
+ String name = MarketplaceHttp.str(map.get("name"));
+ if (isPluginZip(name)) {
+ return name;
+ }
+ }
+ return null;
+ }
+
+ /** True when the file is a real zip artifact rather than a checksum
sidecar. */
+ static boolean isPluginZip(String fileName) {
+ return StringUtils.isNotBlank(fileName) &&
fileName.toLowerCase(Locale.ROOT).endsWith(".zip");
+ }
+
+ /** {@code https://forge.example.io/api/packages/acme/maven} → {@code
https://forge.example.io} */
+ static String extractApiBase(String repositoryUrl) {
+ if (StringUtils.isBlank(repositoryUrl)) {
+ return null;
+ }
+ int idx = repositoryUrl.indexOf(PACKAGES_MARKER);
+ return idx < 0 ? null : repositoryUrl.substring(0, idx);
+ }
+
+ /** {@code https://forge.example.io/api/packages/acme/maven} → {@code acme}
*/
+ static String extractOwner(String repositoryUrl) {
+ if (StringUtils.isBlank(repositoryUrl)) {
+ return null;
+ }
+ int idx = repositoryUrl.indexOf(PACKAGES_MARKER);
+ if (idx < 0) {
+ return null;
+ }
+ String rest = repositoryUrl.substring(idx + PACKAGES_MARKER.length());
+ while (rest.startsWith("/")) {
+ rest = rest.substring(1);
+ }
+ int slash = rest.indexOf('/');
+ String owner = slash < 0 ? rest : rest.substring(0, slash);
+ return StringUtils.trimToNull(owner);
+ }
+}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/NexusRepositoryBrowser.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/NexusRepositoryBrowser.java
index f1d829751f..bd2f5d3e29 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/NexusRepositoryBrowser.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/NexusRepositoryBrowser.java
@@ -17,15 +17,8 @@
package org.apache.hop.marketplace.catalog;
-import java.net.URI;
-import java.net.URLEncoder;
import java.net.http.HttpClient;
-import java.net.http.HttpRequest;
-import java.net.http.HttpResponse;
-import java.nio.charset.StandardCharsets;
-import java.time.Duration;
import java.util.ArrayList;
-import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
@@ -34,6 +27,7 @@ import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.json.HopJson;
import org.apache.hop.core.logging.ILogChannel;
+import org.apache.hop.marketplace.config.MarketplaceHttp;
import org.apache.hop.marketplace.config.MarketplaceRepository;
import org.apache.hop.marketplace.resolve.SnapshotVersions;
@@ -74,11 +68,7 @@ public final class NexusRepositoryBrowser {
return List.of();
}
- HttpClient client =
- HttpClient.newBuilder()
- .connectTimeout(Duration.ofSeconds(30))
- .followRedirects(HttpClient.Redirect.NORMAL)
- .build();
+ HttpClient client = MarketplaceHttp.newClient();
// One row per groupId:artifactId — keep the latest plugin version (not
every unique SNAPSHOT).
Map<String, OptionalPluginInfo> byGa = new LinkedHashMap<>();
@@ -228,39 +218,15 @@ public final class NexusRepositoryBrowser {
private static String getText(HttpClient client, String url,
MarketplaceRepository repository)
throws HopException {
- try {
- HttpRequest.Builder b =
-
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(60)).GET();
- if (repository.hasCredentials()) {
- String token =
- Base64.getEncoder()
- .encodeToString(
- (repository.effectiveUsername() + ":" +
repository.effectivePassword())
- .getBytes(StandardCharsets.UTF_8));
- b.header("Authorization", "Basic " + token);
- }
- HttpResponse<String> response = client.send(b.build(),
HttpResponse.BodyHandlers.ofString());
- if (response.statusCode() < 200 || response.statusCode() >= 300) {
- throw new HopException("Nexus search failed HTTP " +
response.statusCode() + " for " + url);
- }
- return response.body();
- } catch (HopException e) {
- throw e;
- } catch (Exception e) {
- throw new HopException("Nexus search failed for " + url, e);
- }
+ return MarketplaceHttp.getText(client, url, repository, "Nexus search");
}
private static String enc(String s) {
- return URLEncoder.encode(s, StandardCharsets.UTF_8);
+ return MarketplaceHttp.enc(s);
}
private static String str(Object o) {
- if (o == null) {
- return null;
- }
- String s = String.valueOf(o).trim();
- return s.isEmpty() ? null : s;
+ return MarketplaceHttp.str(o);
}
static String mavenPath(String group, String artifact, String version) {
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/PluginDiscovery.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/PluginDiscovery.java
index e9d20adc33..66a02d3da4 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/PluginDiscovery.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/PluginDiscovery.java
@@ -78,6 +78,19 @@ public final class PluginDiscovery {
if (config != null && config.getRepositories() != null) {
for (MarketplaceRepository repo : config.getRepositories()) {
if (repo == null || !repo.isEnabled() || !repo.isBrowse()) {
+ // A catalogUrl with browse=false lists nothing at all, which
otherwise looks
+ // identical to an unreachable catalog: no entries, no error.
+ if (repo != null
+ && repo.isEnabled()
+ && !repo.isBrowse()
+ && StringUtils.isNotBlank(repo.getCatalogUrl())
+ && log != null) {
+ log.logBasic(
+ "Repository '"
+ + repo.getId()
+ + "' has a catalogUrl but browse=false, so its plugins are
not listed."
+ + " Set browse=true to include them.");
+ }
continue;
}
if (StringUtils.isNotBlank(repoId) && !repoId.equals(repo.getId())) {
@@ -272,6 +285,9 @@ public final class PluginDiscovery {
if (StringUtils.isNotBlank(repo.getCatalogUrl())) {
live = RemotePluginCatalog.load(repo);
live = RemotePluginCatalog.filter(live, repo, filter);
+ } else if
(MarketplaceRepository.BROWSER_FORGEJO.equals(repo.effectiveBrowserType())) {
+ // Checked before the Nexus guard: a Forgejo registry URL has no
/repository/ segment.
+ live = ForgejoRepositoryBrowser.browse(repo, filter, log);
} else if (NexusRepositoryBrowser.isNexusBrowseUrl(repo.getUrl())) {
live = NexusRepositoryBrowser.browse(repo, filter, log);
} else {
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/RemotePluginCatalog.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/RemotePluginCatalog.java
index 61dc658859..eca7ce7805 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/RemotePluginCatalog.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/catalog/RemotePluginCatalog.java
@@ -32,6 +32,7 @@ import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.json.HopJson;
+import org.apache.hop.marketplace.config.MarketplaceHttp;
import org.apache.hop.marketplace.config.MarketplaceRepository;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
@@ -50,7 +51,7 @@ public final class RemotePluginCatalog {
String url = repository.getCatalogUrl().trim();
List<OptionalPluginInfo> plugins;
if (url.startsWith("http://";) || url.startsWith("https://";)) {
- plugins = loadHttp(url);
+ plugins = loadHttp(url, repository);
} else {
plugins = loadFile(Path.of(url));
}
@@ -79,19 +80,38 @@ public final class RemotePluginCatalog {
}
public static List<OptionalPluginInfo> loadHttp(String url) throws
HopException {
+ return loadHttp(url, null);
+ }
+
+ /**
+ * Download and parse a catalog, sending the repository's Basic credentials
when configured. A
+ * catalog hosted in a private repository (Forgejo, Nexus, …) is unreachable
without them.
+ */
+ public static List<OptionalPluginInfo> loadHttp(String url,
MarketplaceRepository repository)
+ throws HopException {
try {
- HttpClient client =
- HttpClient.newBuilder()
- .connectTimeout(Duration.ofSeconds(30))
- .followRedirects(HttpClient.Redirect.NORMAL)
- .build();
- HttpRequest request =
-
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(60)).GET().build();
+ HttpClient client = MarketplaceHttp.newClient();
+ HttpRequest.Builder builder =
+
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(60)).GET();
+ MarketplaceHttp.applyAuth(builder, repository);
+ HttpRequest request = builder.build();
HttpResponse<InputStream> response =
client.send(request, HttpResponse.BodyHandlers.ofInputStream());
if (response.statusCode() < 200 || response.statusCode() >= 300) {
- throw new HopException(
- "Unable to download catalog from " + url + " (HTTP " +
response.statusCode() + ")");
+ StringBuilder message = new StringBuilder();
+ message
+ .append("Unable to download catalog from ")
+ .append(url)
+ .append(" (HTTP ")
+ .append(response.statusCode())
+ .append(")");
+ if ((response.statusCode() == 401 || response.statusCode() == 403)
+ && (repository == null || !repository.hasCredentials())) {
+ message.append(
+ ". No credentials were sent; set HOP_MARKETPLACE_USERNAME /
HOP_MARKETPLACE_PASSWORD"
+ + " if the catalog is hosted in a private repository.");
+ }
+ throw new HopException(message.toString());
}
try (InputStream in = response.body()) {
return parse(in, url);
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
new file mode 100644
index 0000000000..7e0d7e7df1
--- /dev/null
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceHttp.java
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.net.URI;
+import java.net.URLEncoder;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.util.Base64;
+import org.apache.hop.core.exception.HopException;
+
+/**
+ * Single HTTP path for every marketplace request: catalog fetches, repository
browsing, definition
+ * imports and artifact downloads. Keeping them together means credentials are
applied — and
+ * recovered from — identically everywhere.
+ */
+public final class MarketplaceHttp {
+
+ private MarketplaceHttp() {}
+
+ public static HttpClient newClient() {
+ return HttpClient.newBuilder()
+ .connectTimeout(Duration.ofSeconds(30))
+ .followRedirects(HttpClient.Redirect.NORMAL)
+ .build();
+ }
+
+ /**
+ * GET with the repository's credentials, retrying once anonymously when the
server rejects
+ * credentials that came only from the environment.
+ *
+ * <p>{@code HOP_MARKETPLACE_USERNAME} / {@code HOP_MARKETPLACE_PASSWORD}
are global, so they are
+ * offered to every repository including public ones. Maven Central and the
ASF repository reject
+ * unknown credentials with 401 rather than ignoring them, which otherwise
makes a working
+ * anonymous repository look broken as soon as any private repository is
configured. Credentials
+ * set explicitly on the repository are never dropped this way — those are
deliberate.
+ */
+ public static <T> HttpResponse<T> send(
+ HttpClient client,
+ String url,
+ Duration timeout,
+ MarketplaceRepository repository,
+ HttpResponse.BodyHandler<T> handler)
+ throws IOException, InterruptedException {
+
+ HttpResponse<T> response = send(client, url, timeout, repository, handler,
true);
+ if (isAuthFailure(response.statusCode())
+ && repository != null
+ && repository.credentialsFromEnvironmentOnly()) {
+ close(response.body());
+ response = send(client, url, timeout, repository, handler, false);
+ }
+ return response;
+ }
+
+ private static <T> HttpResponse<T> send(
+ HttpClient client,
+ String url,
+ Duration timeout,
+ MarketplaceRepository repository,
+ HttpResponse.BodyHandler<T> handler,
+ boolean withCredentials)
+ throws IOException, InterruptedException {
+ HttpRequest.Builder builder =
HttpRequest.newBuilder(URI.create(url)).timeout(timeout).GET();
+ if (withCredentials) {
+ applyAuth(builder, repository);
+ }
+ return client.send(builder.build(), handler);
+ }
+
+ /** Authenticated GET returning the body as text. */
+ public static String getText(
+ HttpClient client, String url, MarketplaceRepository repository, String
label)
+ throws HopException {
+ try {
+ HttpResponse<String> response =
+ send(
+ client,
+ url,
+ Duration.ofSeconds(60),
+ repository,
+ HttpResponse.BodyHandlers.ofString());
+ if (response.statusCode() < 200 || response.statusCode() >= 300) {
+ throw new HopException(
+ label
+ + " failed HTTP "
+ + response.statusCode()
+ + " for "
+ + url
+ + authHint(response.statusCode(), repository));
+ }
+ return response.body();
+ } catch (HopException e) {
+ throw e;
+ } catch (Exception e) {
+ if (e instanceof InterruptedException) {
+ Thread.currentThread().interrupt();
+ }
+ throw new HopException(label + " failed for " + url, e);
+ }
+ }
+
+ /**
+ * HTTP Basic when credentials are configured. Forgejo accepts Basic on both
the Maven endpoint
+ * and the JSON API, with the access token supplied as the password.
+ */
+ public static void applyAuth(HttpRequest.Builder builder,
MarketplaceRepository repository) {
+ if (repository == null || !repository.hasCredentials()) {
+ return;
+ }
+ String credentials = repository.effectiveUsername() + ":" +
repository.effectivePassword();
+ builder.header(
+ "Authorization",
+ "Basic "
+ +
Base64.getEncoder().encodeToString(credentials.getBytes(StandardCharsets.UTF_8)));
+ }
+
+ public static boolean isAuthFailure(int status) {
+ return status == 401 || status == 403;
+ }
+
+ /** Explains an auth failure in terms of what was actually sent. */
+ public static String authHint(int status, MarketplaceRepository repository) {
+ if (!isAuthFailure(status)) {
+ return "";
+ }
+ if (repository == null || !repository.hasCredentials()) {
+ return ". No credentials were sent; set HOP_MARKETPLACE_USERNAME /
HOP_MARKETPLACE_PASSWORD"
+ + " if this repository is private.";
+ }
+ if (repository.credentialsFromEnvironmentOnly()) {
+ return ". Environment credentials for user '"
+ + repository.effectiveUsername()
+ + "' were rejected, and an anonymous retry also failed. Set
repository-specific"
+ + " credentials with HOP_MARKETPLACE_"
+ + repository.environmentIdPrefix()
+ + "_USERNAME / _PASSWORD if this repository needs different ones.";
+ }
+ return ". Basic auth was sent as user '"
+ + repository.effectiveUsername()
+ + "' from the repository configuration; check that password.";
+ }
+
+ private static void close(Object body) {
+ if (body instanceof Closeable closeable) {
+ try {
+ closeable.close();
+ } catch (IOException ignored) {
+ // best effort: the response is being discarded anyway
+ }
+ }
+ }
+
+ public static String enc(String value) {
+ return URLEncoder.encode(value, StandardCharsets.UTF_8);
+ }
+
+ public static String str(Object value) {
+ if (value == null) {
+ return null;
+ }
+ String s = String.valueOf(value).trim();
+ return s.isEmpty() ? null : s;
+ }
+}
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
index 648b92f873..6d73d488be 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepository.java
@@ -19,6 +19,8 @@ package org.apache.hop.marketplace.config;
import java.util.ArrayList;
import java.util.List;
+import java.util.Locale;
+import java.util.function.UnaryOperator;
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.lang3.StringUtils;
@@ -27,6 +29,19 @@ import org.apache.hop.marketplace.catalog.OptionalPluginInfo;
@Getter
@Setter
public class MarketplaceRepository {
+
+ /** Detect the browse API from the repository URL. */
+ public static final String BROWSER_AUTO = "auto";
+
+ /** Sonatype Nexus 3 REST search ({@code /service/rest/v1/search}). */
+ public static final String BROWSER_NEXUS = "nexus";
+
+ /** Forgejo / Gitea package registry ({@code /api/v1/packages/{owner}}). */
+ public static final String BROWSER_FORGEJO = "forgejo";
+
+ /** Marker in a Forgejo / Gitea package registry URL. */
+ private static final String FORGEJO_PACKAGES_MARKER = "/api/packages/";
+
private String id = MarketplaceConfig.DEFAULT_ASF_ID;
private String name;
private String url = MarketplaceConfig.DEFAULT_ASF_URL;
@@ -60,6 +75,31 @@ public class MarketplaceRepository {
*/
private String catalogUrl;
+ /**
+ * Which repository-manager API live browsing speaks: {@code auto}
(default), {@code nexus} or
+ * {@code forgejo}. Only used when {@link #browse} is true and no {@link
#catalogUrl} is set;
+ * downloads are plain Maven layout unless {@link #urlTemplate} says
otherwise.
+ */
+ private String browserType = BROWSER_AUTO;
+
+ /**
+ * Optional download URL template for repositories that do not serve Maven
layout — release
+ * assets, static file hosts, CDNs. When set, {@link #url} is ignored for
plugin zip downloads and
+ * this template is expanded instead.
+ *
+ * <p>Placeholders: <code>${groupId}</code>, <code>${groupPath}</code>
(groupId with dots replaced
+ * by slashes), <code>${artifactId}</code> and <code>${version}</code>.
Example for a Forgejo
+ * release asset:
+ *
+ * <pre>
+ *
https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip
+ * </pre>
+ *
+ * <p>There is no Maven metadata behind such a host, so versions must be
exact — pair this with a
+ * {@link #catalogUrl} that pins them. SNAPSHOT resolution does not apply.
+ */
+ private String urlTemplate;
+
/**
* Optional plugin metadata from a shareable repository definition
(import/export). Used to enrich
* live discovery results (names, categories, descriptions) or as a fallback
list if live browse
@@ -129,25 +169,83 @@ public class MarketplaceRepository {
}
/**
- * Effective credentials: repository config fields, then {@code
HOP_MARKETPLACE_USERNAME} / {@code
- * HOP_MARKETPLACE_PASSWORD}. No credentials means anonymous HTTP.
+ * Environment lookup, replaceable in tests. Package-private on purpose:
credential resolution is
+ * otherwise untestable, and it is the part most likely to go subtly wrong.
+ */
+ private static UnaryOperator<String> environment = System::getenv;
+
+ static void setEnvironmentForTesting(UnaryOperator<String> lookup) {
+ environment = lookup == null ? System::getenv : lookup;
+ }
+
+ private static String env(String name) {
+ return name == null ? null :
StringUtils.trimToNull(environment.apply(name));
+ }
+
+ /**
+ * This repository's environment variable prefix, e.g. {@code ACME} for id
{@code acme} or {@code
+ * LOCAL_NEXUS} for {@code local-nexus}.
+ */
+ public String environmentIdPrefix() {
+ if (StringUtils.isBlank(id)) {
+ return "";
+ }
+ return id.trim().toUpperCase(Locale.ROOT).replaceAll("[^A-Z0-9]", "_");
+ }
+
+ private String scopedEnv(String suffix) {
+ String prefix = environmentIdPrefix();
+ return StringUtils.isBlank(prefix) ? null : env("HOP_MARKETPLACE_" +
prefix + "_" + suffix);
+ }
+
+ /**
+ * Effective credentials, most specific first: the repository's own fields,
then repository-scoped
+ * environment variables ({@code HOP_MARKETPLACE_<ID>_USERNAME}), then the
global {@code
+ * HOP_MARKETPLACE_USERNAME}. No credentials means anonymous HTTP.
+ *
+ * <p>Scoped variables exist so several private repositories can each have
their own token; the
+ * global pair applies to every repository, which is why {@link
#credentialsFromEnvironmentOnly()}
+ * exists to let a rejected global credential fall back to anonymous.
*/
public String effectiveUsername() {
if (StringUtils.isNotBlank(username)) {
return username;
}
- String fromEnv =
- firstNonBlank(
- System.getenv("HOP_MARKETPLACE_USERNAME"),
System.getenv("HOP_MARKETPLACE_USER"));
- return StringUtils.isNotBlank(fromEnv) ? fromEnv : null;
+ return firstNonBlank(
+ scopedEnv("USERNAME"),
+ scopedEnv("USER"),
+ env("HOP_MARKETPLACE_USERNAME"),
+ env("HOP_MARKETPLACE_USER"));
}
public String effectivePassword() {
if (StringUtils.isNotBlank(password)) {
return password;
}
- String fromEnv = System.getenv("HOP_MARKETPLACE_PASSWORD");
- return StringUtils.isNotBlank(fromEnv) ? fromEnv : null;
+ return firstNonBlank(scopedEnv("PASSWORD"),
env("HOP_MARKETPLACE_PASSWORD"));
+ }
+
+ /**
+ * True when credentials were supplied by the environment rather than by
this repository entry.
+ *
+ * <p>Such credentials are not necessarily meant for this repository — the
global variables apply
+ * to all of them — so a server that rejects them is retried anonymously.
Credentials configured
+ * on the entry itself are deliberate and are never dropped.
+ */
+ public boolean credentialsFromEnvironmentOnly() {
+ return StringUtils.isAllBlank(username, password) && hasCredentials();
+ }
+
+ /**
+ * Resolved browse backend. When {@link #browserType} is blank or {@code
auto}, a Forgejo / Gitea
+ * package registry URL ({@code .../api/packages/{owner}/maven}) selects
{@code forgejo}; anything
+ * else falls back to {@code nexus}, which is the historical behaviour.
+ */
+ public String effectiveBrowserType() {
+ if (StringUtils.isNotBlank(browserType) &&
!BROWSER_AUTO.equalsIgnoreCase(browserType.trim())) {
+ return browserType.trim().toLowerCase(Locale.ROOT);
+ }
+ return url != null && url.contains(FORGEJO_PACKAGES_MARKER) ?
BROWSER_FORGEJO : BROWSER_NEXUS;
}
public boolean hasCredentials() {
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
index 88f4175679..fca58a11c7 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/config/MarketplaceRepositoryDefinition.java
@@ -20,9 +20,6 @@ package org.apache.hop.marketplace.config;
import java.io.IOException;
import java.io.InputStream;
import java.io.Writer;
-import java.net.URI;
-import java.net.http.HttpClient;
-import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
@@ -75,24 +72,31 @@ public final class MarketplaceRepositoryDefinition {
return load(Path.of(s));
}
+ /**
+ * Download and parse a shared repository definition.
+ *
+ * <p>No repository entry exists yet at import time, so credentials can only
come from the
+ * environment; a bare repository resolves exactly those. Without this, a
definition published in
+ * a private repository could not be imported by URL at all.
+ */
public static MarketplaceRepository loadFromHttp(String url) throws
HopException {
try {
- HttpClient client =
- HttpClient.newBuilder()
- .connectTimeout(Duration.ofSeconds(30))
- .followRedirects(HttpClient.Redirect.NORMAL)
- .build();
- HttpRequest request =
-
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(60)).GET().build();
+ MarketplaceRepository credentials = new MarketplaceRepository("import",
url);
HttpResponse<InputStream> response =
- client.send(request, HttpResponse.BodyHandlers.ofInputStream());
+ MarketplaceHttp.send(
+ MarketplaceHttp.newClient(),
+ url,
+ Duration.ofSeconds(60),
+ credentials,
+ HttpResponse.BodyHandlers.ofInputStream());
if (response.statusCode() < 200 || response.statusCode() >= 300) {
throw new HopException(
"Unable to download repository definition from "
+ url
+ " (HTTP "
+ response.statusCode()
- + ")");
+ + ")"
+ + MarketplaceHttp.authHint(response.statusCode(),
credentials));
}
try (InputStream in = response.body()) {
return parse(in, url);
@@ -100,6 +104,9 @@ public final class MarketplaceRepositoryDefinition {
} catch (HopException e) {
throw e;
} catch (Exception e) {
+ if (e instanceof InterruptedException) {
+ Thread.currentThread().interrupt();
+ }
throw new HopException("Unable to download repository definition from "
+ url, e);
}
}
@@ -157,6 +164,11 @@ public final class MarketplaceRepositoryDefinition {
}
repo.setBrowse(boolVal(map.get("browse"), false));
repo.setCatalogUrl(stringVal(map.get("catalogUrl")));
+ repo.setUrlTemplate(stringVal(map.get("urlTemplate")));
+ String browserType = stringVal(map.get("browserType"));
+ if (StringUtils.isNotBlank(browserType)) {
+ repo.setBrowserType(browserType);
+ }
repo.setSearchQuery(stringVal(map.get("searchQuery")));
repo.setIncludeSnapshots(boolVal(map.get("includeSnapshots"), true));
repo.setGroupIdFilter(stringVal(map.get("groupIdFilter")));
@@ -257,6 +269,14 @@ public final class MarketplaceRepositoryDefinition {
if (StringUtils.isNotBlank(repo.getCatalogUrl())) {
root.put("catalogUrl", repo.getCatalogUrl());
}
+ if (StringUtils.isNotBlank(repo.getUrlTemplate())) {
+ root.put("urlTemplate", repo.getUrlTemplate());
+ }
+ // Only export an explicit choice; auto-detection is the default and stays
implicit.
+ if (StringUtils.isNotBlank(repo.getBrowserType())
+ &&
!MarketplaceRepository.BROWSER_AUTO.equalsIgnoreCase(repo.getBrowserType())) {
+ root.put("browserType", repo.getBrowserType());
+ }
if (StringUtils.isNotBlank(repo.getSearchQuery())) {
root.put("searchQuery", repo.getSearchQuery());
}
@@ -342,6 +362,8 @@ public final class MarketplaceRepositoryDefinition {
}
existing.setBrowse(imported.isBrowse());
existing.setCatalogUrl(imported.getCatalogUrl());
+ existing.setBrowserType(imported.getBrowserType());
+ existing.setUrlTemplate(imported.getUrlTemplate());
existing.setSearchQuery(imported.getSearchQuery());
existing.setIncludeSnapshots(imported.isIncludeSnapshots());
existing.setGroupIdFilter(imported.getGroupIdFilter());
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
index 824db2adee..61e00d4483 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/gui/MarketplaceRepositoriesPanel.java
@@ -43,6 +43,7 @@ import org.eclipse.swt.layout.FormAttachment;
import org.eclipse.swt.layout.FormData;
import org.eclipse.swt.layout.FormLayout;
import org.eclipse.swt.widgets.Button;
+import org.eclipse.swt.widgets.Combo;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Label;
@@ -498,12 +499,55 @@ public class MarketplaceRepositoriesPanel {
fdCatalog.right = new FormAttachment(100, 0);
wCatalog.setLayoutData(fdCatalog);
+ Label wlUrlTemplate = new Label(general, SWT.RIGHT);
+ PropsUi.setLook(wlUrlTemplate);
+ wlUrlTemplate.setText(BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.UrlTemplate"));
+ FormData fdlUrlTemplate = new FormData();
+ fdlUrlTemplate.left = new FormAttachment(0, 0);
+ fdlUrlTemplate.top = new FormAttachment(wCatalog, margin);
+ fdlUrlTemplate.right = new FormAttachment(middle, -margin);
+ wlUrlTemplate.setLayoutData(fdlUrlTemplate);
+ Text wUrlTemplate = new Text(general, SWT.SINGLE | SWT.LEFT | SWT.BORDER);
+ PropsUi.setLook(wUrlTemplate);
+ wUrlTemplate.setToolTipText(
+ BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.UrlTemplate.Tooltip"));
+ wUrlTemplate.setText(Const.NVL(repo.getUrlTemplate(), ""));
+ FormData fdUrlTemplate = new FormData();
+ fdUrlTemplate.left = new FormAttachment(middle, margin);
+ fdUrlTemplate.top = new FormAttachment(wlUrlTemplate, 0, SWT.CENTER);
+ fdUrlTemplate.right = new FormAttachment(100, 0);
+ wUrlTemplate.setLayoutData(fdUrlTemplate);
+
+ Label wlBrowserType = new Label(general, SWT.RIGHT);
+ PropsUi.setLook(wlBrowserType);
+ wlBrowserType.setText(BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.BrowserType"));
+ FormData fdlBrowserType = new FormData();
+ fdlBrowserType.left = new FormAttachment(0, 0);
+ fdlBrowserType.top = new FormAttachment(wUrlTemplate, margin);
+ fdlBrowserType.right = new FormAttachment(middle, -margin);
+ wlBrowserType.setLayoutData(fdlBrowserType);
+ Combo wBrowserType = new Combo(general, SWT.SINGLE | SWT.LEFT | SWT.BORDER
| SWT.READ_ONLY);
+ PropsUi.setLook(wBrowserType);
+ wBrowserType.setToolTipText(
+ BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.BrowserType.Tooltip"));
+ wBrowserType.setItems(
+ MarketplaceRepository.BROWSER_AUTO,
+ MarketplaceRepository.BROWSER_NEXUS,
+ MarketplaceRepository.BROWSER_FORGEJO);
+ wBrowserType.setText(
+ Const.NVL(repo.getBrowserType(),
MarketplaceRepository.BROWSER_AUTO).toLowerCase());
+ FormData fdBrowserType = new FormData();
+ fdBrowserType.left = new FormAttachment(middle, margin);
+ fdBrowserType.top = new FormAttachment(wlBrowserType, 0, SWT.CENTER);
+ fdBrowserType.right = new FormAttachment(100, 0);
+ wBrowserType.setLayoutData(fdBrowserType);
+
Label wlSearch = new Label(general, SWT.RIGHT);
PropsUi.setLook(wlSearch);
wlSearch.setText(BaseMessages.getString(PKG,
"ManageRepositoriesDialog.Edit.SearchQuery"));
FormData fdlSearch = new FormData();
fdlSearch.left = new FormAttachment(0, 0);
- fdlSearch.top = new FormAttachment(wCatalog, margin);
+ fdlSearch.top = new FormAttachment(wBrowserType, margin);
fdlSearch.right = new FormAttachment(middle, -margin);
wlSearch.setLayoutData(fdlSearch);
Text wSearch = new Text(general, SWT.SINGLE | SWT.LEFT | SWT.BORDER);
@@ -688,6 +732,10 @@ public class MarketplaceRepositoriesPanel {
repo.setPrimary(wPrimary.getSelection());
repo.setBrowse(wBrowse.getSelection());
repo.setCatalogUrl(StringUtils.trimToNull(wCatalog.getText()));
+ repo.setUrlTemplate(StringUtils.trimToNull(wUrlTemplate.getText()));
+ repo.setBrowserType(
+ StringUtils.defaultIfBlank(
+ wBrowserType.getText().trim(),
MarketplaceRepository.BROWSER_AUTO));
repo.setSearchQuery(StringUtils.trimToNull(wSearch.getText()));
repo.setGroupIdFilter(StringUtils.trimToNull(wGroup.getText()));
repo.setIncludeSnapshots(wSnapshots.getSelection());
diff --git
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/resolve/MavenRepositoryClient.java
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/resolve/MavenRepositoryClient.java
index 5d52bd977a..c87f144b78 100644
---
a/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/resolve/MavenRepositoryClient.java
+++
b/plugins/misc/marketplace/src/main/java/org/apache/hop/marketplace/resolve/MavenRepositoryClient.java
@@ -20,22 +20,20 @@ package org.apache.hop.marketplace.resolve;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
-import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
-import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.ArrayList;
-import java.util.Base64;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.logging.ILogChannel;
+import org.apache.hop.marketplace.config.MarketplaceHttp;
import org.apache.hop.marketplace.config.MarketplaceRepository;
/** Downloads Maven layout artifacts over HTTP(S), with optional Basic
authentication. */
@@ -85,12 +83,40 @@ public class MavenRepositoryClient {
Path targetFile,
ITransferListener listener)
throws HopException {
- String base = repository.normalizedUrl();
- String relative = resolveZipRelativePath(repository, coordinates);
- String url = base + relative;
+ String url;
+ if (StringUtils.isNotBlank(repository.getUrlTemplate())) {
+ // Non-Maven host (release assets, static files): no layout, no metadata
lookup.
+ url = expandUrlTemplate(repository.getUrlTemplate(), coordinates);
+ } else {
+ url = repository.normalizedUrl() + resolveZipRelativePath(repository,
coordinates);
+ }
return download(url, repository, coordinates.gav(), targetFile, listener);
}
+ /**
+ * Expand a repository {@code urlTemplate} for one artifact. Supported
placeholders: {@code
+ * ${groupId}}, {@code ${groupPath}}, {@code ${artifactId}} and {@code
${version}}.
+ */
+ static String expandUrlTemplate(String template, MavenCoordinates
coordinates)
+ throws HopException {
+ if (StringUtils.isBlank(template)) {
+ throw new HopException("Repository urlTemplate is empty");
+ }
+ String expanded =
+ template
+ .replace("${groupId}", coordinates.groupId())
+ .replace("${groupPath}", coordinates.groupId().replace('.', '/'))
+ .replace("${artifactId}", coordinates.artifactId())
+ .replace("${version}", coordinates.version());
+ if (expanded.contains("${")) {
+ throw new HopException(
+ "Unresolved placeholder in urlTemplate '"
+ + template
+ + "'. Supported: ${groupId}, ${groupPath}, ${artifactId},
${version}");
+ }
+ return expanded;
+ }
+
/**
* Try each repository in order until a zip is downloaded. Aggregates
per-repo errors if all fail
* (fallback chain).
@@ -242,11 +268,13 @@ public class MavenRepositoryClient {
private String getText(String url, MarketplaceRepository repository) throws
HopException {
try {
- HttpRequest.Builder builder =
-
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofMinutes(2)).GET();
- applyBasicAuth(builder, repository);
HttpResponse<String> response =
- httpClient.send(builder.build(),
HttpResponse.BodyHandlers.ofString());
+ MarketplaceHttp.send(
+ httpClient,
+ url,
+ Duration.ofMinutes(2),
+ repository,
+ HttpResponse.BodyHandlers.ofString());
if (response.statusCode() == 401 || response.statusCode() == 403) {
throw new HopException(authFailureMessage(response.statusCode(), url,
repository));
}
@@ -274,11 +302,13 @@ public class MavenRepositoryClient {
ITransferListener progress = listener == null ? ITransferListener.NONE :
listener;
try {
Files.createDirectories(targetFile.getParent());
- HttpRequest.Builder builder =
-
HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofMinutes(30)).GET();
- applyBasicAuth(builder, repository);
HttpResponse<InputStream> response =
- httpClient.send(builder.build(),
HttpResponse.BodyHandlers.ofInputStream());
+ MarketplaceHttp.send(
+ httpClient,
+ url,
+ Duration.ofMinutes(30),
+ repository,
+ HttpResponse.BodyHandlers.ofInputStream());
if (response.statusCode() == 401 || response.statusCode() == 403) {
throw new HopException(authFailureMessage(response.statusCode(), url,
repository));
}
@@ -358,17 +388,7 @@ public class MavenRepositoryClient {
}
static void applyBasicAuth(HttpRequest.Builder builder,
MarketplaceRepository repository) {
- if (repository == null || !repository.hasCredentials()) {
- return;
- }
- String user = repository.effectiveUsername();
- String pass = repository.effectivePassword();
- if (StringUtils.isAnyBlank(user, pass)) {
- return;
- }
- String token =
- Base64.getEncoder().encodeToString((user + ":" +
pass).getBytes(StandardCharsets.UTF_8));
- builder.header("Authorization", "Basic " + token);
+ MarketplaceHttp.applyAuth(builder, repository);
}
private static String authFailureMessage(
diff --git
a/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
b/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
index 2e893c857e..3bd79da568 100644
---
a/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
+++
b/plugins/misc/marketplace/src/main/resources/org/apache/hop/marketplace/gui/messages/messages_en_US.properties
@@ -166,6 +166,10 @@ ManageRepositoriesDialog.Edit.Primary=Primary (tried first)
ManageRepositoriesDialog.Edit.Browse=Browse (list zips in query / Plugins tab)
ManageRepositoriesDialog.Edit.Browse.Tooltip=When enabled, marketplace query
and the Plugins tab always include a live list of *.zip artifacts from this
repository (Nexus search). Optional catalog URL overrides live search.
ManageRepositoriesDialog.Edit.CatalogUrl=Catalog URL
+ManageRepositoriesDialog.Edit.UrlTemplate=Download URL template
+ManageRepositoriesDialog.Edit.UrlTemplate.Tooltip=Optional. For hosts that do
not serve Maven layout (release assets, static files). Placeholders:
$'{'groupId'}', $'{'groupPath'}', $'{'artifactId'}', $'{'version'}'. When set,
the repository URL is not used for plugin downloads.
+ManageRepositoriesDialog.Edit.BrowserType=Browse API
+ManageRepositoriesDialog.Edit.BrowserType.Tooltip=Which repository manager API
is used to list plugins when Browse is on and no catalog URL is set. auto
detects Forgejo/Gitea package registries and falls back to Nexus. Downloads
never use this.
ManageRepositoriesDialog.Edit.SearchQuery=Search query
ManageRepositoriesDialog.Edit.GroupIdFilter=Group ID filter
ManageRepositoriesDialog.Edit.IncludeSnapshots=Include SNAPSHOT versions in
discovery
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowserTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowserTest.java
new file mode 100644
index 0000000000..01ef7ae01b
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/catalog/ForgejoRepositoryBrowserTest.java
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.catalog;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.List;
+import org.apache.hop.marketplace.catalog.ForgejoRepositoryBrowser.Candidate;
+import org.junit.jupiter.api.Test;
+
+class ForgejoRepositoryBrowserTest {
+
+ private static final String PACKAGES_JSON =
+ """
+ [
+ {
+ "id": 1,
+ "type": "maven",
+ "name": "com.acme:acme-parser",
+ "version": "2026.09",
+ "created_at": "2026-07-20T10:00:00Z"
+ },
+ {
+ "id": 2,
+ "type": "maven",
+ "name": "com.acme:acme-parser",
+ "version": "2026.06",
+ "created_at": "2026-04-01T10:00:00Z"
+ },
+ {
+ "id": 3,
+ "type": "container",
+ "name": "some/image",
+ "version": "1.0"
+ },
+ {
+ "id": 4,
+ "type": "maven",
+ "name": "no-colon-here",
+ "version": "1.0"
+ }
+ ]
+ """;
+
+ @Test
+ void parsesMavenPackagesOnly() throws Exception {
+ List<Candidate> candidates =
ForgejoRepositoryBrowser.parsePackagesPage(PACKAGES_JSON);
+ // container package and the colon-less name are both dropped
+ assertEquals(2, candidates.size());
+ assertEquals("com.acme", candidates.get(0).groupId());
+ assertEquals("acme-parser", candidates.get(0).artifactId());
+ assertEquals("2026.09", candidates.get(0).version());
+ assertEquals("2026-07-20T10:00:00Z", candidates.get(0).createdAt());
+ }
+
+ @Test
+ void emptyPageIsEmptyList() throws Exception {
+ assertTrue(ForgejoRepositoryBrowser.parsePackagesPage("[]").isEmpty());
+ assertTrue(ForgejoRepositoryBrowser.parsePackagesPage("").isEmpty());
+ assertTrue(ForgejoRepositoryBrowser.parsePackagesPage(null).isEmpty());
+ }
+
+ @Test
+ void splitsGroupAndArtifact() {
+ assertArrayEquals(
+ new String[] {"com.acme", "acme-parser"},
+ ForgejoRepositoryBrowser.splitPackageName("com.acme:acme-parser"));
+ assertArrayEquals(
+ new String[] {"org.apache.hop", "hop-tech-parquet"},
+ ForgejoRepositoryBrowser.splitPackageName("
org.apache.hop:hop-tech-parquet "));
+ assertNull(ForgejoRepositoryBrowser.splitPackageName("no-colon"));
+ assertNull(ForgejoRepositoryBrowser.splitPackageName(":leading"));
+ assertNull(ForgejoRepositoryBrowser.splitPackageName("trailing:"));
+ assertNull(ForgejoRepositoryBrowser.splitPackageName(""));
+ assertNull(ForgejoRepositoryBrowser.splitPackageName(null));
+ }
+
+ @Test
+ void findsPluginZipAmongFiles() throws Exception {
+ String files =
+ """
+ [
+ {"name": "acme-parser-2026.09.pom", "size": 100},
+ {"name": "acme-parser-2026.09.jar", "size": 200},
+ {"name": "acme-parser-2026.09.zip", "size": 300}
+ ]
+ """;
+ assertEquals("acme-parser-2026.09.zip",
ForgejoRepositoryBrowser.extractZipFileName(files));
+ }
+
+ @Test
+ void jarOnlyArtifactHasNoZip() throws Exception {
+ String files =
+ """
+ [
+ {"name": "acme-shared-2026.09.pom"},
+ {"name": "acme-shared-2026.09.jar"}
+ ]
+ """;
+ // Shared libraries must not be offered as installable plugins.
+ assertNull(ForgejoRepositoryBrowser.extractZipFileName(files));
+ assertNull(ForgejoRepositoryBrowser.extractZipFileName("[]"));
+ assertNull(ForgejoRepositoryBrowser.extractZipFileName(null));
+ }
+
+ @Test
+ void derivesHostAndOwnerFromRegistryUrl() {
+ String url = "https://forge.example.org/api/packages/acme/maven";;
+ assertEquals("https://forge.example.org";,
ForgejoRepositoryBrowser.extractApiBase(url));
+ assertEquals("acme", ForgejoRepositoryBrowser.extractOwner(url));
+
+ // trailing slash and a bare owner endpoint both work
+ assertEquals(
+ "acme",
+ ForgejoRepositoryBrowser.extractOwner(
+ "https://forge.example.org/api/packages/acme/maven/";));
+ assertEquals(
+ "acme",
ForgejoRepositoryBrowser.extractOwner("https://git.example.org/api/packages/acme";));
+ assertEquals(
+ "https://git.example.org";,
+
ForgejoRepositoryBrowser.extractApiBase("https://git.example.org/api/packages/acme";));
+ }
+
+ @Test
+ void nonForgejoUrlYieldsNothing() {
+ String nexus = "https://repository.apache.org/content/groups/public/";;
+ assertNull(ForgejoRepositoryBrowser.extractApiBase(nexus));
+ assertNull(ForgejoRepositoryBrowser.extractOwner(nexus));
+ assertNull(ForgejoRepositoryBrowser.extractApiBase(null));
+ assertNull(ForgejoRepositoryBrowser.extractOwner(null));
+ }
+}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/MarketplaceRepositoryBrowserTypeTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/MarketplaceRepositoryBrowserTypeTest.java
new file mode 100644
index 0000000000..35429ff499
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/MarketplaceRepositoryBrowserTypeTest.java
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+class MarketplaceRepositoryBrowserTypeTest {
+
+ @Test
+ void forgejoRegistryUrlIsDetected() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme",
"https://forge.example.org/api/packages/acme/maven";);
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
repo.effectiveBrowserType());
+ }
+
+ @Test
+ void nexusRemainsTheDefault() {
+ assertEquals(
+ MarketplaceRepository.BROWSER_NEXUS,
+ new MarketplaceRepository("asf",
"https://repository.apache.org/content/groups/public/";)
+ .effectiveBrowserType());
+ assertEquals(
+ MarketplaceRepository.BROWSER_NEXUS,
+ new MarketplaceRepository("local",
"http://127.0.0.1:8081/repository/hop-plugins/";)
+ .effectiveBrowserType());
+ }
+
+ @Test
+ void explicitTypeWinsOverDetection() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("proxy",
"https://mirror.example.com/api/packages/acme/maven";);
+ repo.setBrowserType("NEXUS");
+ assertEquals(MarketplaceRepository.BROWSER_NEXUS,
repo.effectiveBrowserType());
+
+ repo.setBrowserType(" Forgejo ");
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
repo.effectiveBrowserType());
+
+ repo.setBrowserType(MarketplaceRepository.BROWSER_AUTO);
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
repo.effectiveBrowserType());
+ }
+
+ @Test
+ void browserTypeSurvivesExportImport(@TempDir Path dir) throws Exception {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme",
"https://forge.example.org/api/packages/acme/maven";);
+ repo.setBrowse(true);
+ repo.setBrowserType(MarketplaceRepository.BROWSER_FORGEJO);
+
+ Path file = dir.resolve("repo.yaml");
+ MarketplaceRepositoryDefinition.save(file, repo);
+ assertTrue(Files.readString(file).contains("browserType"));
+
+ MarketplaceRepository loaded = MarketplaceRepositoryDefinition.load(file);
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
loaded.getBrowserType());
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
loaded.effectiveBrowserType());
+ }
+
+ @Test
+ void autoIsNotWrittenToYaml() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("asf",
"https://repository.apache.org/content/groups/public/";);
+ Map<String, Object> yaml = MarketplaceRepositoryDefinition.toYamlMap(repo,
false);
+ assertFalse(yaml.containsKey("browserType"));
+ }
+
+ @Test
+ void missingBrowserTypeKeepsDefault() throws Exception {
+ // A definition written before browserType existed must still load.
+ MarketplaceRepository repo =
+ MarketplaceRepositoryDefinition.load(
+ Files.writeString(
+ Files.createTempFile("repo", ".yaml"),
+ """
+ schemaVersion: "1.0"
+ kind: hop-marketplace-repository
+ id: legacy
+ url: https://repository.example.com/repository/hop/
+ browse: true
+ """));
+ assertEquals(MarketplaceRepository.BROWSER_AUTO, repo.getBrowserType());
+ assertEquals(MarketplaceRepository.BROWSER_NEXUS,
repo.effectiveBrowserType());
+ }
+
+ @Test
+ void urlTemplateSurvivesExportImport(@TempDir Path dir) throws Exception {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("acme",
"https://forge.example.org/api/packages/acme/maven";);
+ repo.setUrlTemplate(
+
"https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip";);
+
+ Path file = dir.resolve("template.yaml");
+ MarketplaceRepositoryDefinition.save(file, repo);
+ MarketplaceRepository loaded = MarketplaceRepositoryDefinition.load(file);
+ assertEquals(repo.getUrlTemplate(), loaded.getUrlTemplate());
+ }
+
+ @Test
+ void urlTemplateAbsentByDefault() {
+ MarketplaceRepository repo =
+ new MarketplaceRepository("asf",
"https://repository.apache.org/content/groups/public/";);
+ assertNull(repo.getUrlTemplate());
+ assertFalse(MarketplaceRepositoryDefinition.toYamlMap(repo,
false).containsKey("urlTemplate"));
+ }
+
+ /**
+ * Re-importing over an existing entry must carry the new fields across. The
upsert path copies
+ * field by field, so a field missing there is dropped silently on every
re-import.
+ */
+ @Test
+ void reimportOverExistingEntryKeepsBrowserTypeAndTemplate() throws Exception
{
+ MarketplaceConfig config = new MarketplaceConfig();
+ config.setRepositories(
+ new java.util.ArrayList<>(
+ List.of(new MarketplaceRepository("acme",
"https://old.example.org/repository/hop/";))));
+
+ MarketplaceRepository imported =
+ new MarketplaceRepository("acme",
"https://forge.example.org/api/packages/acme/maven";);
+ imported.setBrowserType(MarketplaceRepository.BROWSER_FORGEJO);
+ imported.setUrlTemplate(
+
"https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip";);
+
+ MarketplaceRepositoryDefinition.applyToConfig(config, imported, false);
+
+ MarketplaceRepository stored = config.findRepository("acme");
+ assertEquals(MarketplaceRepository.BROWSER_FORGEJO,
stored.getBrowserType());
+ assertEquals(imported.getUrlTemplate(), stored.getUrlTemplate());
+ }
+
+ @Test
+ void firstImportKeepsBrowserTypeAndTemplate() throws Exception {
+ MarketplaceConfig config = new MarketplaceConfig();
+ config.setRepositories(new java.util.ArrayList<>());
+
+ MarketplaceRepository imported =
+ new MarketplaceRepository("acme",
"https://forge.example.org/api/packages/acme/maven";);
+
imported.setUrlTemplate("https://example.org/${artifactId}-${version}.zip";);
+ MarketplaceRepositoryDefinition.applyToConfig(config, imported, false);
+
+ assertEquals(
+ "https://example.org/${artifactId}-${version}.zip";,
+ config.findRepository("acme").getUrlTemplate());
+ }
+}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryCredentialsTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryCredentialsTest.java
new file mode 100644
index 0000000000..808a1bf5f2
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/config/RepositoryCredentialsTest.java
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.config;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.Map;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+class RepositoryCredentialsTest {
+
+ private static void env(Map<String, String> values) {
+ MarketplaceRepository.setEnvironmentForTesting(values::get);
+ }
+
+ @AfterEach
+ void restoreEnvironment() {
+ MarketplaceRepository.setEnvironmentForTesting(null);
+ }
+
+ @Test
+ void noCredentialsMeansAnonymous() {
+ env(Map.of());
+ MarketplaceRepository repo = new MarketplaceRepository("asf",
"https://example.org/";);
+ assertNull(repo.effectiveUsername());
+ assertNull(repo.effectivePassword());
+ assertFalse(repo.hasCredentials());
+ assertFalse(repo.credentialsFromEnvironmentOnly());
+ }
+
+ @Test
+ void globalEnvironmentAppliesToAnyRepository() {
+ env(Map.of("HOP_MARKETPLACE_USERNAME", "reader",
"HOP_MARKETPLACE_PASSWORD", "secret"));
+ MarketplaceRepository repo = new MarketplaceRepository("asf",
"https://example.org/";);
+ assertEquals("reader", repo.effectiveUsername());
+ assertEquals("secret", repo.effectivePassword());
+ // Global credentials are not specific to this repository, so a rejection
may be retried.
+ assertTrue(repo.credentialsFromEnvironmentOnly());
+ }
+
+ @Test
+ void scopedEnvironmentBeatsGlobal() {
+ env(
+ Map.of(
+ "HOP_MARKETPLACE_USERNAME", "global-user",
+ "HOP_MARKETPLACE_PASSWORD", "global-pass",
+ "HOP_MARKETPLACE_ACME_USERNAME", "acme-user",
+ "HOP_MARKETPLACE_ACME_PASSWORD", "acme-pass"));
+ MarketplaceRepository acme = new MarketplaceRepository("acme",
"https://example.org/";);
+ assertEquals("acme-user", acme.effectiveUsername());
+ assertEquals("acme-pass", acme.effectivePassword());
+
+ // A different repository still gets the global pair.
+ MarketplaceRepository other = new MarketplaceRepository("asf",
"https://example.org/";);
+ assertEquals("global-user", other.effectiveUsername());
+ }
+
+ @Test
+ void repositoryIdIsNormalisedForEnvironmentNames() {
+ assertEquals(
+ "LOCAL_NEXUS", new MarketplaceRepository("local-nexus",
"u").environmentIdPrefix());
+ assertEquals("ACME", new MarketplaceRepository("acme",
"u").environmentIdPrefix());
+ assertEquals("A_B_C", new MarketplaceRepository("a.b-c",
"u").environmentIdPrefix());
+
+ env(Map.of("HOP_MARKETPLACE_LOCAL_NEXUS_USERNAME", "n"));
+ assertEquals("n", new MarketplaceRepository("local-nexus",
"u").effectiveUsername());
+ }
+
+ @Test
+ void configuredCredentialsAreNeverTreatedAsEnvironmental() {
+ env(Map.of("HOP_MARKETPLACE_USERNAME", "global",
"HOP_MARKETPLACE_PASSWORD", "global"));
+ MarketplaceRepository repo =
+ new MarketplaceRepository("private", "https://example.org/";, "user",
"pass");
+ assertEquals("user", repo.effectiveUsername());
+ assertEquals("pass", repo.effectivePassword());
+ // Deliberate credentials: a 401 must surface, not silently retry
anonymously.
+ assertFalse(repo.credentialsFromEnvironmentOnly());
+ }
+
+ @Test
+ void partiallyConfiguredCredentialsAreNotEnvironmental() {
+ // username on the entry, password from the environment: still a
deliberate pairing
+ env(Map.of("HOP_MARKETPLACE_PASSWORD", "from-env"));
+ MarketplaceRepository repo = new MarketplaceRepository("mixed",
"https://example.org/";);
+ repo.setUsername("configured");
+ assertTrue(repo.hasCredentials());
+ assertFalse(repo.credentialsFromEnvironmentOnly());
+ }
+
+ @Test
+ void authHintExplainsWhatWasSent() {
+ env(Map.of());
+ MarketplaceRepository anonymous = new MarketplaceRepository("asf",
"https://example.org/";);
+ assertTrue(MarketplaceHttp.authHint(401, anonymous).contains("No
credentials were sent"));
+
+ env(Map.of("HOP_MARKETPLACE_USERNAME", "u", "HOP_MARKETPLACE_PASSWORD",
"p"));
+ MarketplaceRepository fromEnv = new MarketplaceRepository("acme",
"https://example.org/";);
+ String hint = MarketplaceHttp.authHint(401, fromEnv);
+ assertTrue(hint.contains("anonymous retry also failed"));
+ assertTrue(hint.contains("HOP_MARKETPLACE_ACME_USERNAME"));
+
+ // Not an auth failure: no hint at all.
+ assertEquals("", MarketplaceHttp.authHint(404, fromEnv));
+ }
+}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/gui/MessagesFormatTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/gui/MessagesFormatTest.java
new file mode 100644
index 0000000000..eeeba88dc3
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/gui/MessagesFormatTest.java
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.gui;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.fail;
+
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.charset.StandardCharsets;
+import java.text.MessageFormat;
+import java.util.Properties;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Hop renders every message through {@link MessageFormat}, so a literal brace
in a message value is
+ * read as an argument placeholder and throws at display time rather than at
build time. Braces have
+ * to be quoted as {@code '{'} / {@code '}'}.
+ */
+class MessagesFormatTest {
+
+ private static final String BUNDLE =
+ "/org/apache/hop/marketplace/gui/messages/messages_en_US.properties";
+
+ private static Properties messages() throws Exception {
+ Properties properties = new Properties();
+ try (InputStream in =
MessagesFormatTest.class.getResourceAsStream(BUNDLE)) {
+ assertNotNull(in, "missing bundle " + BUNDLE);
+ properties.load(new InputStreamReader(in, StandardCharsets.UTF_8));
+ }
+ return properties;
+ }
+
+ @Test
+ void everyMessageIsAValidMessageFormatPattern() throws Exception {
+ Properties messages = messages();
+ for (String key : messages.stringPropertyNames()) {
+ String value = messages.getProperty(key);
+ try {
+ new MessageFormat(value);
+ } catch (IllegalArgumentException e) {
+ fail(
+ "Message '"
+ + key
+ + "' is not a valid MessageFormat pattern ("
+ + e.getMessage()
+ + "). Quote literal braces as '{' and '}':\n "
+ + value);
+ }
+ }
+ }
+
+ @Test
+ void placeholderDocumentationSurvivesFormatting() throws Exception {
+ // The urlTemplate tooltip documents ${...} placeholders, which collide
with MessageFormat.
+ String tooltip =
messages().getProperty("ManageRepositoriesDialog.Edit.UrlTemplate.Tooltip");
+ assertNotNull(tooltip);
+ String rendered = new MessageFormat(tooltip).format(new Object[0]);
+ assertEquals(
+ true,
+ rendered.contains("${artifactId}") && rendered.contains("${version}"),
+ "placeholders must reach the user with their braces intact, got:\n "
+ rendered);
+ }
+}
diff --git
a/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/resolve/UrlTemplateTest.java
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/resolve/UrlTemplateTest.java
new file mode 100644
index 0000000000..3e9f4f8103
--- /dev/null
+++
b/plugins/misc/marketplace/src/test/java/org/apache/hop/marketplace/resolve/UrlTemplateTest.java
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.marketplace.resolve;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import org.apache.hop.core.exception.HopException;
+import org.junit.jupiter.api.Test;
+
+class UrlTemplateTest {
+
+ private static final MavenCoordinates COORDS =
+ new MavenCoordinates("com.acme.hop", "acme-parser", "2026.09");
+
+ @Test
+ void expandsReleaseAssetTemplate() throws Exception {
+ String template =
+
"https://forge.example.org/acme/dist/releases/download/v${version}/${artifactId}-${version}.zip";;
+ assertEquals(
+
"https://forge.example.org/acme/dist/releases/download/v2026.09/acme-parser-2026.09.zip";,
+ MavenRepositoryClient.expandUrlTemplate(template, COORDS));
+ }
+
+ @Test
+ void expandsGroupPlaceholders() throws Exception {
+ assertEquals(
+
"https://cdn.example.org/com/acme/hop/acme-parser/2026.09/acme-parser-2026.09.zip";,
+ MavenRepositoryClient.expandUrlTemplate(
+
"https://cdn.example.org/${groupPath}/${artifactId}/${version}/${artifactId}-${version}.zip";,
+ COORDS));
+ assertEquals(
+ "https://cdn.example.org/com.acme.hop/acme-parser.zip";,
+ MavenRepositoryClient.expandUrlTemplate(
+ "https://cdn.example.org/${groupId}/${artifactId}.zip";, COORDS));
+ }
+
+ @Test
+ void templateWithoutPlaceholdersIsUsedAsIs() throws Exception {
+ assertEquals(
+ "https://example.org/fixed.zip";,
+
MavenRepositoryClient.expandUrlTemplate("https://example.org/fixed.zip";,
COORDS));
+ }
+
+ @Test
+ void unknownPlaceholderIsRejected() {
+ HopException e =
+ assertThrows(
+ HopException.class,
+ () ->
+ MavenRepositoryClient.expandUrlTemplate(
+ "https://example.org/${classifier}/${artifactId}.zip";,
COORDS));
+ // A silently unexpanded placeholder would produce a 404 that is hard to
diagnose.
+ assertTrue(e.getMessage().contains("Unresolved placeholder"));
+ }
+
+ @Test
+ void blankTemplateIsRejected() {
+ assertThrows(HopException.class, () ->
MavenRepositoryClient.expandUrlTemplate("", COORDS));
+ assertThrows(HopException.class, () ->
MavenRepositoryClient.expandUrlTemplate(null, COORDS));
+ }
+}
