Copilot commented on code in PR #3178:
URL: https://github.com/apache/tika/pull/3178#discussion_r4018339895
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/UnpackerResource.java:
##########
@@ -224,15 +233,28 @@ public Response unpackAll(InputStream is, @Context
HttpHeaders httpHeaders, @Con
* @param info URI info
* @return streaming zip response
*/
- @jakarta.ws.rs.Path("/all{id:(/.*)?}")
+ @jakarta.ws.rs.Path("/all")
+ @POST
+ @Consumes("multipart/form-data")
+ @Produces("application/zip")
+ public Response unpackAllWithConfig(List<Attachment> attachments,
+ @Context HttpHeaders httpHeaders)
throws Exception {
+ return unpackAllWithConfig(attachments, httpHeaders, null);
+ }
+
+ /** As {@code POST /unpack/all}, with the sidecar handler named in the
path. */
+ @jakarta.ws.rs.Path("/all/{" + HANDLER_TYPE_PARAM + "}")
@POST
@Consumes("multipart/form-data")
@Produces("application/zip")
- public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders, @Context UriInfo info) throws Exception {
- rejectPresetInWildcard(info);
+ public Response unpackAllWithConfig(List<Attachment> attachments, @Context
HttpHeaders httpHeaders,
+
@jakarta.ws.rs.PathParam(HANDLER_TYPE_PARAM) String handlerTypeName)
Review Comment:
This route change also removes the existing `POST /unpack/all/config`
endpoint: the old `/all{id:(/.*)?}` method accepted that path as the multipart
config variant, while the new `/all/{handler}` treats `config` as a handler
name and `applyHandler` returns 400. `release-tools/uat/run-uat.sh` and the
integration-test documentation still exercise this URL, so either preserve an
explicit compatibility route or update all shipped UAT/docs and call out the
breaking API change.
##########
tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/UnpackerResource.java:
##########
@@ -100,7 +99,7 @@
* "unpack-config": {
* "outputFormat": "FRICTIONLESS",
* "outputMode": "ZIPPED",
- * "includeFullMetadata": true
+ * "includeMetadata": true
* }
Review Comment:
The example now uses the new `includeMetadata` setting, but the output tree
below still says `metadata.json` is conditional on the deprecated
`includeFullMetadata=true`. With this change Frictionless writes it by default
unless `includeMetadata=false`; update that description so the class-level API
documentation matches the new behavior.
##########
docs/modules/ROOT/pages/pipes/unpack-config.adoc:
##########
@@ -247,13 +252,120 @@ When using Frictionless output format, the ZIP archive
contains:
----
output.zip
├── datapackage.json # Manifest with file list, SHA256 hashes, mimetypes
-├── metadata.json # Full RMETA metadata (if includeFullMetadata=true)
+├── metadata.json # Full RMETA metadata (unless includeMetadata=false)
└── unpacked/
├── 00000001.pdf
├── 00000002.png
└── ...
----
+The `tk:content` field inside `metadata.json` (and inside the per-file
`.metadata.json`
+sidecars `includeMetadata` adds to REGULAR zip output) is Markdown by default.
On tika-server
+change it with `/unpack/all/{handlerType}` or a `content-handler-factory` in
the `config`
+part; in a config, the factory alone. Each entry records the handler it was
written with in
+`tk:content-handler-type`:
+
+[source,json]
+----
+{
+ "unpack-config": { "outputFormat": "FRICTIONLESS" },
+ "basic-content-handler-factory": { "type": "XML" }
+}
+----
+
+To get metadata with no extracted text at all -- one sidecar per file, but no
`tk:content` --
+select the `IGNORE` handler. On the test corpus below this drops the sidecars
from 20,257 to
+11,132 bytes and is the way to avoid shipping every embedded document's text
alongside its
+bytes:
+
+[source,json]
+----
+{
+ "unpack-config": { "includeMetadata": true },
+ "basic-content-handler-factory": { "type": "IGNORE" }
+}
+----
+
+On tika-server the handler is also a path segment on
`/unpack/all/{handlerType}`, which
+needs no `config` part and so no `allowPerRequestConfig`. Plain `/unpack`
carries no
+metadata for a handler to render, so it takes no segment.
+
+[source,bash]
+----
+curl -T container.docx http://localhost:9998/unpack/all/ignore # files +
metadata, no text
+----
+
+Naming the handler in both the path and a `config` part is a `400`, as on
`/tika` and
+`/rmeta`.
+
+== What each surface produces
+
+The five knobs interact differently per surface. The defaults are unchanged
from 4.0.0.
Review Comment:
This statement is no longer accurate: the PR changes the Frictionless
default so `metadata.json` is emitted when `includeMetadata` is unset, whereas
the previous default was opt-in via `includeFullMetadata`. Please describe the
changed default instead of saying all defaults are unchanged.
##########
docs/modules/ROOT/pages/using-tika/server/index.adoc:
##########
@@ -158,6 +159,14 @@ Output format on the preset routes: an explicit format
segment (`/tika/preset/{n
always wins; without one, a `ContentHandlerFactory` the preset binds decides,
then the
config's, then the endpoint default (Markdown).
+Output format: name the handler once. A format segment sets it --
`/tika/config/xml`,
+`/rmeta/config/xml`, and `/unpack/all/{handlerType}`, which selects how
`tk:content` is
+rendered in the metadata `/all` adds (`/unpack/all/ignore` for none; plain
`/unpack` carries
+no metadata, so it takes no segment). With no segment, a
`content-handler-factory` in the
Review Comment:
This new description says plain `/unpack` carries no metadata, but the same
PR makes a server-level Frictionless `/unpack` emit `metadata.json` by default
(as covered by `UnpackFrictionlessTest`). The restriction is that plain
`/unpack` has no handler path segment, not that it can never carry metadata;
please clarify which handler is used there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
