[
https://issues.apache.org/jira/browse/TIKA-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137167#comment-18137167
]
ASF GitHub Bot commented on TIKA-5025:
--------------------------------------
dschmidt opened a new pull request, #3519:
URL: https://github.com/apache/tika/pull/3519
Since TIKA-4988 the Docker image runs a thumbnail server with nothing but
`-e TIKA_PRESETS=thumbnail`; the first file over 1 GiB brought the config file
back, because `maxRequestSizeBytes` and the other limits of TIKA-4991 live in
the `server` block and had no other way in. tika-server now takes
`--serverSetting <field>=<value>`, repeatable, and the environment variable
`TIKA_SERVER_<FIELD>`, the field's name in upper snake case
(`TIKA_SERVER_MAX_REQUEST_SIZE_BYTES=4294967296`). Every field of the block is
covered, not a hand-picked few, so the next limit needs no ticket.
The values go through the same Jackson binding as the file:
`TikaServerConfig.load` collects them, logs each with its source (and what it
overrides in the file), and applies them with `readerForUpdating` after the
file's `server` block is deserialized. That way the field's own setter checks
the value (`requestLogLevel must be one of ...`), a wrong type is reported with
the variable's name and value, and an unknown field fails startup naming the
fields there are, as it does in the file. The option wins over the variable,
which wins over the file; `-p`, `-h`, `-i` and `--metricsPort` stay on top as
before. `configPath` is refused with a pointer to `-c`; `tlsConfig` and
`endpoints` take an object and a list and stay in the file. Only the front end
reads the `server` block, so nothing is folded into the forks' config, unlike
the presets.
The 413 body now says `Request body exceeds maxRequestSizeBytes (1073741824
bytes); raise it in the config's server block or with
TIKA_SERVER_MAX_REQUEST_SIZE_BYTES`, for the declared Content-Length and the
chunked case alike, so an operator reads the answer instead of the server log.
Tests: `TikaServerConfigTest` (option and variable parsing, precedence,
override of a file value, the error messages, the name mapping) and
`ServerSettingsIntegrationTest`, which starts the real server with
`TIKA_SERVER_MAX_REQUEST_SIZE_BYTES=10` and with `--serverSetting`, checks the
413 and its hint end to end, plus the unknown-field startup failure.
Docs: server page (options table, configuration section), Docker page,
CHANGES.
Open for discussion: the option is `--serverSetting <field>=<value>` rather
than one option per field, because commons-cli needs the options registered up
front; a generated `--<field>` per setter is possible if that reads better.
Values are plain byte counts, no `4g` suffixes.
https://issues.apache.org/jira/browse/TIKA-5025
> tika-server: set the request limits from the environment and the command line
> -----------------------------------------------------------------------------
>
> Key: TIKA-5025
> URL: https://issues.apache.org/jira/browse/TIKA-5025
> Project: Tika
> Issue Type: Improvement
> Reporter: Dominik Schmidt
> Priority: Major
>
> Since TIKA-4988 the Docker image runs a thumbnail server with nothing but -e
> TIKA_PRESETS=thumbnail. The first file over 1 GiB brings the config file
> back: maxRequestSizeBytes (TIKA-4991, default 1073741824) lives in the server
> block of the JSON config and can only be changed by mounting a file and
> passing -c. The same goes for its siblings maxInFlightRequestBytes,
> maxRequestReadMillis, maxResponseWriteMillis and maxQueuePauseMillis.
> Measured on 4.2.1-SNAPSHOT with a 1.46 GB MPEG-4 audiobook: PUT
> /unpack/preset/thumbnail answers 413 "Request body exceeds
> maxRequestSizeBytes" from the declared Content-Length alone; with {"server":
> {"maxRequestSizeBytes": 4294967296}, "presets": {"thumbnail": true}} the
> cover comes back in 4 s.
> Proposal: every setting of the server block can be given as
> TIKA_SERVER_<UPPER_SNAKE_CASE> in the environment, e.g.
> TIKA_SERVER_MAX_REQUEST_SIZE_BYTES=4294967296, and as --server-<kebab-case>
> <value> on the command line, following TIKA-4988: both are merged into the
> startup config before the front end and the forked workers load it, so the
> forks keep receiving the resolved config over stdin and never read the
> environment; command line and environment win over the file; an unknown key
> fails startup; the startup log says where a value came from. Sizes accept the
> plain byte count, suffixes (4g) would be nice but are a separate decision.
> Also: the 413 body should name the limit and where to raise it, as the preset
> 404 does, so an operator reads the answer instead of the server log.
> Docs: the server page's limits section and the Docker page get the env form
> next to the config form.
> Related: TIKA-4988 (TIKA_PRESETS), TIKA-4991 (the limits), TIKA-4909 (env
> interpolation inside a config file, which still needs the file).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
