[
https://issues.apache.org/jira/browse/TIKA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110724#comment-18110724
]
ASF GitHub Bot commented on TIKA-4856:
--------------------------------------
Copilot commented on code in PR #3096:
URL: https://github.com/apache/tika/pull/3096#discussion_r3915905130
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/ThumbnailDefaults.java:
##########
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.server.core.resource;
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+import org.apache.tika.config.loader.TikaJsonConfig;
+import org.apache.tika.parser.ParseContext;
+
+/**
+ * The parser configuration that makes a parse yield the document thumbnail
+ * as a raster image: the first PDF page rendered, the EMF/WMF thumbnail of
+ * an Office document rendered (that one only, not the pictures of embedded
+ * objects), in colour: the renderer's default is the grayscale OCR wants.
+ * The stored thumbnails of the other formats need no configuration.
+ * <p>
+ * Applied by {@code renderThumbnails=true} on {@code /rmeta}, {@code /unpack}
+ * and {@code /unpack/all}, and by {@code /unpack/thumbnail}. Three layers,
+ * each overriding the one before: the built-in defaults below, a
+ * {@code thumbnail-defaults} block in the server config with the same shape
+ * as a request config (parser configurations keyed by component name), and
+ * the request's own config part.
+ * <pre>
+ * "thumbnail-defaults": {
+ * "pdf-parser": {"imageStrategy": "RENDER_PAGES_AT_PAGE_END",
"maxRenderedPages": 1,
+ * "ocr": {"dpi": 150}}
+ * }
+ * </pre>
+ * A configuration for a parser that is not installed is never read, so the
+ * defaults are harmless on a server without that parser.
+ */
+public final class ThumbnailDefaults {
+
+ public static final String CONFIG_KEY = "thumbnail-defaults";
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+
+ private static final String BUILT_IN = """
+ {
+ "pdf-parser": {
+ "imageStrategy": "RENDER_PAGES_AT_PAGE_END",
+ "maxRenderedPages": 1,
+ "ocr": {"dpi": 96, "imageType": "RGB"}
+ },
+ "emf-parser": {"renderImage": true,
"renderOnlyEmbeddedResourceTypes": ["THUMBNAIL"]},
+ "wmf-parser": {"renderImage": true,
"renderOnlyEmbeddedResourceTypes": ["THUMBNAIL"]}
+ }
+ """;
+
+ /**
+ * Parser configurations keyed by component name, in application order.
+ */
+ private final Map<String, ObjectNode> components;
+
+ private ThumbnailDefaults(Map<String, ObjectNode> components) {
+ this.components = components;
+ }
+
+ /**
+ * No defaults at all, a base to {@link #with(String)} settings on.
+ */
+ public static ThumbnailDefaults none() {
+ return new ThumbnailDefaults(new LinkedHashMap<>());
+ }
+
+ /**
+ * These defaults with another set merged in, field by field.
+ */
+ public ThumbnailDefaults with(ThumbnailDefaults other) {
+ ThumbnailDefaults merged = this;
+ for (Map.Entry<String, ObjectNode> component :
other.components.entrySet()) {
+ merged = merged.with("{\"" + component.getKey() + "\": " +
component.getValue() + "}");
+ }
+ return merged;
Review Comment:
This merges `ThumbnailDefaults` by serializing each component to a JSON
string and reparsing it, which is avoidable overhead (and can be on a hot path
via `configureThumbnailParse`). Merge `ObjectNode`s directly (reusing the
existing `deepMerge`) to avoid repeated JSON parse/serialization.
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/UnpackerResource.java:
##########
@@ -215,15 +273,184 @@ public Response unpackAll(InputStream is, @Context
HttpHeaders httpHeaders, @Con
@POST
@Consumes("multipart/form-data")
@Produces("application/zip")
- public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info) throws Exception {
+ public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
ParseContext pc = tikaResource.createRequestContext();
Metadata metadata = tikaResource.newRequestMetadata();
try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
TikaResource.logRequest(LOG, "/unpack/all", metadata);
+ if (renderThumbnails) {
+ //under the request's config, which setupMultipartConfig has
already merged
+ tikaResource.getThumbnailDefaults().applyTo(pc);
+ }
return doUnpack(tis, metadata, pc, true);
}
}
+ /**
+ * Returns the document thumbnail with its metadata (simple PUT).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @PUT
+ @Produces("application/json")
+ public Response unpackThumbnail(InputStream is, @Context HttpHeaders
httpHeaders,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis = TikaInputStream.get(is)) {
+ fillMetadata(null, metadata, httpHeaders.getRequestHeaders());
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ /**
+ * Returns the document thumbnail with its metadata (multipart POST,
"file" part).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @POST
+ @Consumes("multipart/form-data")
+ @Produces("application/json")
+ public Response unpackThumbnailMultipart(List<Attachment> attachments,
@Context HttpHeaders httpHeaders,
+ @QueryParam("renderThumbnails")
boolean renderThumbnails)
+ throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String METADATA_SUFFIX = ".metadata.json";
+ /**
+ * A thumbnail travels base64-encoded inside a JSON object, so it is
+ * bounded here regardless of the unpack limits; camera previews and
+ * page renderings are a few MB at most.
+ */
+ static final long MAX_THUMBNAIL_BYTES = 32L * 1024 * 1024;
+ /**
+ * What {@code /unpack/thumbnail} adds regardless of rendering: the text
+ * of the images is not wanted.
+ */
+ private static final ThumbnailDefaults NO_OCR = ThumbnailDefaults.none()
+ .with("{\"pdf-parser\": {\"ocr\": {\"strategy\": \"NO_OCR\"}}, "
+ + "\"tesseract-ocr-parser\": {\"skipOcr\": true}}");
+
+ /**
+ * Parses in unpack mode with the thumbnail configuration, then selects
+ * the thumbnail among the extracted embedded documents.
+ */
+ private Response doUnpackThumbnail(TikaInputStream tis, Metadata metadata,
ParseContext pc,
+ boolean renderThumbnails) throws
Exception {
+ PipesParsingHelper helper = tikaResource.getPipesParsingHelper();
+ if (helper == null) {
+ throw new WebApplicationException("Pipes-based parsing is not
enabled", Response.Status.SERVICE_UNAVAILABLE);
+ }
+ configureThumbnailParse(pc, renderThumbnails);
+
+ PipesParsingHelper.UnpackResult result = helper.parseUnpack(tis,
metadata, pc, false);
+ if (result.zipFile() == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ try (ZipFile zip = new ZipFile(result.zipFile().toFile())) {
+ Map<String, Metadata> extracted = readExtractedMetadata(zip);
+ Metadata thumbnail = ThumbnailSelector.select(new
ArrayList<>(extracted.values()));
+ if (thumbnail == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ String entryName = null;
+ for (Map.Entry<String, Metadata> e : extracted.entrySet()) {
+ if (e.getValue() == thumbnail) {
+ entryName = e.getKey();
+ }
+ }
+ ZipEntry imageEntry = entryName == null ? null :
zip.getEntry(entryName);
+ if (imageEntry == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ if (imageEntry.getSize() > MAX_THUMBNAIL_BYTES) {
+ throw new WebApplicationException("thumbnail larger than " +
MAX_THUMBNAIL_BYTES + " bytes",
+ Response.Status.REQUEST_ENTITY_TOO_LARGE);
+ }
+ byte[] image;
+ try (InputStream is = zip.getInputStream(imageEntry)) {
+ //the entry size is a claim; read one byte past the limit to
know
+ image = is.readNBytes((int) MAX_THUMBNAIL_BYTES + 1);
+ }
+ if (image.length > MAX_THUMBNAIL_BYTES) {
+ throw new WebApplicationException("thumbnail larger than " +
MAX_THUMBNAIL_BYTES + " bytes",
+ Response.Status.REQUEST_ENTITY_TOO_LARGE);
+ }
+ StringWriter metadataJson = new StringWriter();
+ JsonMetadata.toJson(thumbnail, metadataJson);
+ ObjectNode root = MAPPER.createObjectNode();
+ root.set("metadata", MAPPER.readTree(metadataJson.toString()));
+ root.put("image", Base64.getEncoder().encodeToString(image));
+ return
Response.ok(MAPPER.writeValueAsString(root)).type("application/json").build();
Review Comment:
This performs multiple conversions (Metadata -> JSON string -> JSON parse ->
JSON string). Consider eliminating the intermediate `StringWriter` and re-parse
step, and return the `ObjectNode` directly (letting the JAX-RS Jackson provider
serialize it), which reduces allocations and avoids double JSON processing on
every request.
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/UnpackerResource.java:
##########
@@ -215,15 +273,184 @@ public Response unpackAll(InputStream is, @Context
HttpHeaders httpHeaders, @Con
@POST
@Consumes("multipart/form-data")
@Produces("application/zip")
- public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info) throws Exception {
+ public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
ParseContext pc = tikaResource.createRequestContext();
Metadata metadata = tikaResource.newRequestMetadata();
try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
TikaResource.logRequest(LOG, "/unpack/all", metadata);
+ if (renderThumbnails) {
+ //under the request's config, which setupMultipartConfig has
already merged
+ tikaResource.getThumbnailDefaults().applyTo(pc);
+ }
return doUnpack(tis, metadata, pc, true);
}
}
+ /**
+ * Returns the document thumbnail with its metadata (simple PUT).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @PUT
+ @Produces("application/json")
+ public Response unpackThumbnail(InputStream is, @Context HttpHeaders
httpHeaders,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis = TikaInputStream.get(is)) {
+ fillMetadata(null, metadata, httpHeaders.getRequestHeaders());
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ /**
+ * Returns the document thumbnail with its metadata (multipart POST,
"file" part).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @POST
+ @Consumes("multipart/form-data")
+ @Produces("application/json")
+ public Response unpackThumbnailMultipart(List<Attachment> attachments,
@Context HttpHeaders httpHeaders,
+ @QueryParam("renderThumbnails")
boolean renderThumbnails)
+ throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String METADATA_SUFFIX = ".metadata.json";
+ /**
+ * A thumbnail travels base64-encoded inside a JSON object, so it is
+ * bounded here regardless of the unpack limits; camera previews and
+ * page renderings are a few MB at most.
+ */
+ static final long MAX_THUMBNAIL_BYTES = 32L * 1024 * 1024;
+ /**
+ * What {@code /unpack/thumbnail} adds regardless of rendering: the text
+ * of the images is not wanted.
+ */
+ private static final ThumbnailDefaults NO_OCR = ThumbnailDefaults.none()
+ .with("{\"pdf-parser\": {\"ocr\": {\"strategy\": \"NO_OCR\"}}, "
+ + "\"tesseract-ocr-parser\": {\"skipOcr\": true}}");
+
+ /**
+ * Parses in unpack mode with the thumbnail configuration, then selects
+ * the thumbnail among the extracted embedded documents.
+ */
+ private Response doUnpackThumbnail(TikaInputStream tis, Metadata metadata,
ParseContext pc,
+ boolean renderThumbnails) throws
Exception {
+ PipesParsingHelper helper = tikaResource.getPipesParsingHelper();
+ if (helper == null) {
+ throw new WebApplicationException("Pipes-based parsing is not
enabled", Response.Status.SERVICE_UNAVAILABLE);
+ }
+ configureThumbnailParse(pc, renderThumbnails);
+
+ PipesParsingHelper.UnpackResult result = helper.parseUnpack(tis,
metadata, pc, false);
+ if (result.zipFile() == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ try (ZipFile zip = new ZipFile(result.zipFile().toFile())) {
+ Map<String, Metadata> extracted = readExtractedMetadata(zip);
+ Metadata thumbnail = ThumbnailSelector.select(new
ArrayList<>(extracted.values()));
+ if (thumbnail == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ String entryName = null;
+ for (Map.Entry<String, Metadata> e : extracted.entrySet()) {
+ if (e.getValue() == thumbnail) {
+ entryName = e.getKey();
Review Comment:
`entryName` lookup relies on reference identity (`==`) and scans the entire
map even after finding the match. Prefer returning the zip entry name directly
from the selector (e.g., select the `Map.Entry<String, Metadata>`), or at least
`break` after assigning `entryName` to avoid unnecessary iteration and make the
intent clearer.
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/UnpackerResource.java:
##########
@@ -215,15 +273,184 @@ public Response unpackAll(InputStream is, @Context
HttpHeaders httpHeaders, @Con
@POST
@Consumes("multipart/form-data")
@Produces("application/zip")
- public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info) throws Exception {
+ public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
ParseContext pc = tikaResource.createRequestContext();
Metadata metadata = tikaResource.newRequestMetadata();
try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
TikaResource.logRequest(LOG, "/unpack/all", metadata);
+ if (renderThumbnails) {
+ //under the request's config, which setupMultipartConfig has
already merged
+ tikaResource.getThumbnailDefaults().applyTo(pc);
+ }
return doUnpack(tis, metadata, pc, true);
}
}
+ /**
+ * Returns the document thumbnail with its metadata (simple PUT).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @PUT
+ @Produces("application/json")
+ public Response unpackThumbnail(InputStream is, @Context HttpHeaders
httpHeaders,
+ @QueryParam("renderThumbnails") boolean
renderThumbnails) throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis = TikaInputStream.get(is)) {
+ fillMetadata(null, metadata, httpHeaders.getRequestHeaders());
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ /**
+ * Returns the document thumbnail with its metadata (multipart POST,
"file" part).
+ */
+ @jakarta.ws.rs.Path("/thumbnail")
+ @POST
+ @Consumes("multipart/form-data")
+ @Produces("application/json")
+ public Response unpackThumbnailMultipart(List<Attachment> attachments,
@Context HttpHeaders httpHeaders,
+ @QueryParam("renderThumbnails")
boolean renderThumbnails)
+ throws Exception {
+ ParseContext pc = tikaResource.createRequestContext();
+ Metadata metadata = tikaResource.newRequestMetadata();
+ try (TikaInputStream tis =
tikaResource.setupMultipartConfig(attachments, metadata, pc)) {
+ TikaResource.logRequest(LOG, "/unpack/thumbnail", metadata);
+ return doUnpackThumbnail(tis, metadata, pc, renderThumbnails);
+ }
+ }
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String METADATA_SUFFIX = ".metadata.json";
+ /**
+ * A thumbnail travels base64-encoded inside a JSON object, so it is
+ * bounded here regardless of the unpack limits; camera previews and
+ * page renderings are a few MB at most.
+ */
+ static final long MAX_THUMBNAIL_BYTES = 32L * 1024 * 1024;
+ /**
+ * What {@code /unpack/thumbnail} adds regardless of rendering: the text
+ * of the images is not wanted.
+ */
+ private static final ThumbnailDefaults NO_OCR = ThumbnailDefaults.none()
+ .with("{\"pdf-parser\": {\"ocr\": {\"strategy\": \"NO_OCR\"}}, "
+ + "\"tesseract-ocr-parser\": {\"skipOcr\": true}}");
+
+ /**
+ * Parses in unpack mode with the thumbnail configuration, then selects
+ * the thumbnail among the extracted embedded documents.
+ */
+ private Response doUnpackThumbnail(TikaInputStream tis, Metadata metadata,
ParseContext pc,
+ boolean renderThumbnails) throws
Exception {
+ PipesParsingHelper helper = tikaResource.getPipesParsingHelper();
+ if (helper == null) {
+ throw new WebApplicationException("Pipes-based parsing is not
enabled", Response.Status.SERVICE_UNAVAILABLE);
+ }
+ configureThumbnailParse(pc, renderThumbnails);
+
+ PipesParsingHelper.UnpackResult result = helper.parseUnpack(tis,
metadata, pc, false);
+ if (result.zipFile() == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ try (ZipFile zip = new ZipFile(result.zipFile().toFile())) {
+ Map<String, Metadata> extracted = readExtractedMetadata(zip);
+ Metadata thumbnail = ThumbnailSelector.select(new
ArrayList<>(extracted.values()));
+ if (thumbnail == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ String entryName = null;
+ for (Map.Entry<String, Metadata> e : extracted.entrySet()) {
+ if (e.getValue() == thumbnail) {
+ entryName = e.getKey();
+ }
+ }
+ ZipEntry imageEntry = entryName == null ? null :
zip.getEntry(entryName);
+ if (imageEntry == null) {
+ throw new WebApplicationException(Response.Status.NO_CONTENT);
+ }
+ if (imageEntry.getSize() > MAX_THUMBNAIL_BYTES) {
+ throw new WebApplicationException("thumbnail larger than " +
MAX_THUMBNAIL_BYTES + " bytes",
+ Response.Status.REQUEST_ENTITY_TOO_LARGE);
+ }
+ byte[] image;
+ try (InputStream is = zip.getInputStream(imageEntry)) {
+ //the entry size is a claim; read one byte past the limit to
know
+ image = is.readNBytes((int) MAX_THUMBNAIL_BYTES + 1);
+ }
+ if (image.length > MAX_THUMBNAIL_BYTES) {
+ throw new WebApplicationException("thumbnail larger than " +
MAX_THUMBNAIL_BYTES + " bytes",
+ Response.Status.REQUEST_ENTITY_TOO_LARGE);
+ }
+ StringWriter metadataJson = new StringWriter();
+ JsonMetadata.toJson(thumbnail, metadataJson);
+ ObjectNode root = MAPPER.createObjectNode();
+ root.set("metadata", MAPPER.readTree(metadataJson.toString()));
+ root.put("image", Base64.getEncoder().encodeToString(image));
+ return
Response.ok(MAPPER.writeValueAsString(root)).type("application/json").build();
+ } finally {
+ result.cleanup();
+ }
+ }
+
+ /**
+ * What only makes sense when the thumbnail is all the caller wants: no
+ * text, no OCR, only THUMBNAIL and RENDERING embedded documents extracted,
+ * together with their metadata, down to the rendering of a thumbnail
+ * (depth 2). With {@code renderThumbnails} the {@link ThumbnailDefaults}
+ * are laid under that, the same switch as on the other endpoints; without
+ * it only stored thumbnails are found. The request's own parser
+ * configuration wins where present.
+ */
+ private void configureThumbnailParse(ParseContext pc, boolean
renderThumbnails) {
+ //the text is not part of the answer: do not extract it
+ tikaResource.setupContentHandlerFactory(pc, "ignore");
+ (renderThumbnails ? tikaResource.getThumbnailDefaults().with(NO_OCR) :
NO_OCR).applyTo(pc);
+ StandardUnpackSelector selector = new StandardUnpackSelector();
+ selector.setIncludeEmbeddedResourceTypes(new HashSet<>(Arrays.asList(
+ TikaCoreProperties.EmbeddedResourceType.THUMBNAIL.name(),
+ TikaCoreProperties.EmbeddedResourceType.RENDERING.name())));
+ pc.set(UnpackSelector.class, selector);
Review Comment:
For multipart `/unpack/thumbnail`, a client-provided request config can
still override rendering behavior (e.g., render many pages/images) even though
the endpoint only returns one thumbnail; this can amplify CPU/disk usage and
create very large intermediate unzip outputs before the final 32MiB thumbnail
cap is enforced. Consider enforcing a hard upper bound for thumbnail-mode
rendering (e.g., clamp `maxRenderedPages` to 1, and/or set explicit
embedded/rendering limits) regardless of request config to protect server
availability.
##########
tika-server/tika-server-standard/src/test/java/org/apache/tika/server/standard/UnpackerThumbnailTest.java:
##########
@@ -0,0 +1,208 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.server.standard;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.awt.image.BufferedImage;
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.imageio.ImageIO;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import jakarta.ws.rs.core.Response;
+import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
+import org.apache.cxf.jaxrs.client.WebClient;
+import org.apache.cxf.jaxrs.lifecycle.SingletonResourceProvider;
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.serialization.config.JsonConfigHelper;
+import org.apache.tika.server.core.CXFTestBase;
+import org.apache.tika.server.core.TikaServerParseExceptionMapper;
+import org.apache.tika.server.core.resource.RecursiveMetadataResource;
+import org.apache.tika.server.core.resource.UnpackerResource;
+import org.apache.tika.server.core.writer.MetadataListMessageBodyWriter;
+
+/**
+ * {@code /unpack/thumbnail} end to end: the document thumbnail comes back as
+ * JSON with its metadata and the image as base64.
+ */
+public class UnpackerThumbnailTest extends CXFTestBase {
+
+ private static final String THUMBNAIL_PATH = "/unpack/thumbnail";
+ private static final String UNPACK_CONFIG_TEMPLATE =
"/configs/cxf-unpack-test-template.json";
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+
+ private Path unpackTempDir;
+
+ @Override
+ protected void setUpResources(JAXRSServerFactoryBean sf) {
+ sf.setResourceClasses(UnpackerResource.class,
RecursiveMetadataResource.class);
+ sf.setResourceProvider(UnpackerResource.class,
+ new SingletonResourceProvider(new
UnpackerResource(tikaResource)));
+ sf.setResourceProvider(RecursiveMetadataResource.class,
+ new SingletonResourceProvider(new
RecursiveMetadataResource(tikaResource)));
+ }
+
+ @Override
+ protected void setUpProviders(JAXRSServerFactoryBean sf) {
+ List<Object> providers = new ArrayList<>();
+ providers.add(new TikaServerParseExceptionMapper());
+ providers.add(new MetadataListMessageBodyWriter());
+ sf.setProviders(providers);
+ }
+
+ @Override
+ protected InputStream getPipesConfigInputStream() throws IOException {
+ unpackTempDir =
Files.createTempDirectory("tika-unpack-thumbnail-test-");
+ Path pluginsDir = Paths.get("target/plugins").toAbsolutePath();
+ Map<String, Object> replacements = new HashMap<>();
+ replacements.put("UNPACK_EMITTER_BASE_PATH",
unpackTempDir.toAbsolutePath().toString());
Review Comment:
This test creates a temp directory but never cleans it up, which can
accumulate files across local runs/CI workers. Add a teardown hook (e.g.,
`@AfterEach`/`@AfterAll` depending on lifecycle) that recursively deletes
`unpackTempDir` once the test server is done with it.
> /unpack/thumbnail: return the document thumbnail with its metadata
> ------------------------------------------------------------------
>
> Key: TIKA-4856
> URL: https://issues.apache.org/jira/browse/TIKA-4856
> Project: Tika
> Issue Type: New Feature
> Reporter: Dominik Schmidt
> Priority: Major
>
> With TIKA-4850 through TIKA-4855 every container format that carries a
> thumbnail emits it as a THUMBNAIL embedded document, the PDF parser renders
> pages as RENDERING documents, and the EMF/WMF renderer turns the vector
> thumbnails of Office documents into raster ones. Getting "the thumbnail of
> this file" out of that still takes format knowledge on the client: the
> THUMBNAIL of a Word or Excel file is an EMF/WMF whose usable form is the
> RENDERING underneath it, a PDF has no THUMBNAIL but a page RENDERING, the
> THUMBNAIL of a DOCX inside a ZIP is not the ZIP's, and with rendering enabled
> the picture of an embedded OLE object is a RENDERING too. Plus the request
> config that switches the renderers on.
> Proposal: POST /unpack/thumbnail next to /unpack and /unpack/all, multipart
> like them. It runs the usual forked parse in unpack mode with a fixed parse
> context (PDF page 1 rendered, EMF/WMF rendered) and picks, in this order: the
> raster THUMBNAIL at depth 1; the rendering of that thumbnail; the depth-1
> RENDERING of PDF page 1. The endpoint extracts what the document carries; it
> does not resize, convert or generate previews.
> The response is JSON: the /rmeta metadata object of the selected embedded
> document, and the image as base64. Thumbnails are small, so the encoding
> overhead does not matter, and the caller gets type, dimensions, origin
> (stored thumbnail or rendering, tk:rendering:rendered-by) and path in one
> round trip without unpacking a zip. 204 when the document has no thumbnail.
> {
> "metadata": {
> "Content-Type": "image/png",
> "Content-Length": "8459",
> "tiff:ImageWidth": "800",
> "tiff:ImageLength": "1131",
> "tk:embedded-resource-type": "RENDERING",
> "tk:embedded-resource-path": "/thumbnail.emf/thumbnail.png",
> "tk:embedded-depth": "2",
> "tk:rendering:rendered-by": "poi-metafile-renderer",
> "tk:resource-name": "thumbnail.png"
> },
> "image": "iVBORw0KGgoAAAANSUhEUgAA..."
> }
> To keep the selection rule short, the metafile renderer could give the
> rendering of a THUMBNAIL the THUMBNAIL type as well (its
> tk:rendering:rendered-by tells it apart), so a raster thumbnail is a
> THUMBNAIL regardless of whether the document stored it as PNG or as EMF.
> What do you think?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
