[
https://issues.apache.org/jira/browse/TIKA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison updated TIKA-4794:
------------------------------
Description:
*Background*
Tika's metadata keys are inconsistent, and 4.0.0 is the last free break until
5.0 (whatever ships is frozen for the 4.x line):
* Tika's own keys are scattered across ~8 prefixes (X-TIKA:, tika:, tika_pg:,
rendering:, signature:, imagereader:, bare hasSignature/embeddedResourceType),
with 3+ casings inside X-TIKA: alone (snake, camel, Title-Hyphen, UPPER
sub-namespaces).
* resourceName shipped half-migrated (bare → X-TIKA:resourceName) — the worst
state.
* The metadata object is also a control plane (media type drives routing,
resourceName drives detection, X-TIKA:* carries provenance/digests), but the
boundary is convention only: a crafted file can inject or overwrite Tika's own
keys.
* Format namespaces mix casings; QuattroPro reuses WordPerfect's prefix
(collision); there's no rule for "is this key Tika's or the file's".
Options
# Leave as-is. Ship the inconsistency + half-migrated resourceName +
convention-only boundary; frozen until 5.x
# tk:. Consolidate every Tika-asserted key under one kebab-cased tk: prefix;
finish resourceName → tk:resource-name; add a reserved-namespace trust
boundary. Format namespaces untouched.
# tk: + kebab. Everything in (2), plus apply the casing rule inside the
format namespaces.
*Guiding principles (2 and 3)*
1. Who asserted the key? Tika-computed → owned, normalized, trusted.
File/standard-authored → verbatim (the spelling/capitalization is the interop
contract).
2. Match enforcement to stakes. Control-plane / Tika-native keys → reserved,
writable only via a Property; cosmetic drift → convention.
3. Casing follows key-name authorship, not the namespace (the prefix is just
the domain). Tika coined the name → kebab; a standard/file authored it →
verbatim.
4. Clean 3→4 break, documented with a migration table; no auto-remap
machinery.
*2 — tk: (rationale + consistency)*
One prefix, one casing for everything Tika asserts, replacing the 8 scattered
prefixes and their mixed casings; resourceName finished. tk: (and legacy
X-TIKA: ) become reserved: a String write to them is dropped unless trusted, so
a scraped file can't forge Tika's routing/provenance/digest/content keys or
inject fake tk:chunks embeddings — enforced once in tika-core, not per-parser
discipline. Consistency: any Tika key is now predictable (tk: + kebab), and the
trust boundary is mechanism rather than convention.
*3 — tk: + kebab (rationale + consistency)*
Extends "Tika-coined → kebab" into the format namespaces (e.g.
pdf:hasMarkedContent → pdf:has-marked-content, zip:compressedSize →
zip:compressed-size, access_permission: → access-permission: ). Names from a
standard stay verbatim (dc:, xmp:, EXIF/tiff:, MAPI, email headers, ECMA-376
OOXML props, PDF/A/X sub-namespaces) — re-casing them would break
round-trip/interop. Also fixes the QuattroPro/WordPerfect prefix collision.
Consistency: a Tika-authored key is kebab wherever it lives (under tk: or a
format prefix); a standard/file key keeps its source spelling — one rule
answers casing everywhere.
If we did steps 2 or 3, we'd probably want to release a 4.0.0-beta-2.
Steps 2 or 3 would be fairly major changes for downstream users, but it would
allow us to clean up years of metadata key name bloat.
was:
*Background*
Tika's metadata keys are inconsistent, and 4.0.0 is the last free break until
5.0 (whatever ships is frozen for the 4.x line):
* Tika's own keys are scattered across ~8 prefixes (X-TIKA:, tika:, tika_pg:,
rendering:, signature:, imagereader:, bare hasSignature/embeddedResourceType),
with 3+ casings inside X-TIKA: alone (snake, camel, Title-Hyphen, UPPER
sub-namespaces).
* resourceName shipped half-migrated (bare → X-TIKA:resourceName) — the worst
state.
* The metadata object is also a control plane (media type drives routing,
resourceName drives detection, X-TIKA:* carries provenance/digests), but the
boundary is convention only: a crafted file can inject or overwrite Tika's own
keys.
* Format namespaces mix casings; QuattroPro reuses WordPerfect's prefix
(collision); there's no rule for "is this key Tika's or the file's".
Options
# Leave as-is. Ship the inconsistency + half-migrated resourceName +
convention-only boundary; frozen until 5.x
# tk:. Consolidate every Tika-asserted key under one kebab-cased tk: prefix;
finish resourceName → tk:resource-name; add a reserved-namespace trust
boundary. Format namespaces untouched.
# tk: + kebab. Everything in (2), plus apply the casing rule inside the
format namespaces.
*Guiding principles (2 and 3)*
1. Who asserted the key? Tika-computed → owned, normalized, trusted.
File/standard-authored → verbatim (the spelling/capitalization is the interop
contract).
2. Match enforcement to stakes. Control-plane / Tika-native keys → reserved,
writable only via a Property; cosmetic drift → convention.
3. Casing follows key-name authorship, not the namespace (the prefix is just
the domain). Tika coined the name → kebab; a standard/file authored it →
verbatim.
4. Clean 3→4 break, documented with a migration table; no auto-remap
machinery.
*2 — tk: (rationale + consistency)*
One prefix, one casing for everything Tika asserts, replacing the 8 scattered
prefixes and their mixed casings; resourceName finished. tk: (and legacy
X-TIKA: ) become reserved: a String write to them is dropped unless trusted, so
a scraped file can't forge Tika's routing/provenance/digest/content keys or
inject fake tk:chunks embeddings — enforced once in tika-core, not per-parser
discipline. Consistency: any Tika key is now predictable (tk: + kebab), and the
trust boundary is mechanism rather than convention.
*3 — tk: + kebab (rationale + consistency)*
Extends "Tika-coined → kebab" into the format namespaces (e.g.
pdf:hasMarkedContent → pdf:has-marked-content, zip:compressedSize →
zip:compressed-size, access_permission: → access-permission: ). Names from a
standard stay verbatim (dc:, xmp:, EXIF/tiff:, MAPI, email headers, ECMA-376
OOXML props, PDF/A/X sub-namespaces) — re-casing them would break
round-trip/interop. Also fixes the QuattroPro/WordPerfect prefix collision.
Consistency: a Tika-authored key is kebab wherever it lives (under tk: or a
format prefix); a standard/file key keeps its source spelling — one rule
answers casing everywhere.
If we did steps 2 or 3, we'd probably want to release a 4.0.0-beta-2
> Normalize/simplify metadata keys in 4.x
> ---------------------------------------
>
> Key: TIKA-4794
> URL: https://issues.apache.org/jira/browse/TIKA-4794
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Major
>
> *Background*
> Tika's metadata keys are inconsistent, and 4.0.0 is the last free break until
> 5.0 (whatever ships is frozen for the 4.x line):
> * Tika's own keys are scattered across ~8 prefixes (X-TIKA:, tika:,
> tika_pg:, rendering:, signature:, imagereader:, bare
> hasSignature/embeddedResourceType), with 3+ casings inside X-TIKA: alone
> (snake, camel, Title-Hyphen, UPPER sub-namespaces).
> * resourceName shipped half-migrated (bare → X-TIKA:resourceName) — the
> worst state.
> * The metadata object is also a control plane (media type drives routing,
> resourceName drives detection, X-TIKA:* carries provenance/digests), but the
> boundary is convention only: a crafted file can inject or overwrite Tika's
> own keys.
> * Format namespaces mix casings; QuattroPro reuses WordPerfect's prefix
> (collision); there's no rule for "is this key Tika's or the file's".
> Options
> # Leave as-is. Ship the inconsistency + half-migrated resourceName +
> convention-only boundary; frozen until 5.x
> # tk:. Consolidate every Tika-asserted key under one kebab-cased tk:
> prefix; finish resourceName → tk:resource-name; add a reserved-namespace
> trust boundary. Format namespaces untouched.
> # tk: + kebab. Everything in (2), plus apply the casing rule inside the
> format namespaces.
> *Guiding principles (2 and 3)*
> 1. Who asserted the key? Tika-computed → owned, normalized, trusted.
> File/standard-authored → verbatim (the spelling/capitalization is the interop
> contract).
> 2. Match enforcement to stakes. Control-plane / Tika-native keys →
> reserved, writable only via a Property; cosmetic drift → convention.
> 3. Casing follows key-name authorship, not the namespace (the prefix is
> just the domain). Tika coined the name → kebab; a standard/file authored it →
> verbatim.
> 4. Clean 3→4 break, documented with a migration table; no auto-remap
> machinery.
> *2 — tk: (rationale + consistency)*
> One prefix, one casing for everything Tika asserts, replacing the 8
> scattered prefixes and their mixed casings; resourceName finished. tk: (and
> legacy X-TIKA: ) become reserved: a String write to them is dropped unless
> trusted, so a scraped file can't forge Tika's
> routing/provenance/digest/content keys or inject fake tk:chunks embeddings —
> enforced once in tika-core, not per-parser discipline. Consistency: any Tika
> key is now predictable (tk: + kebab), and the trust boundary is mechanism
> rather than convention.
> *3 — tk: + kebab (rationale + consistency)*
> Extends "Tika-coined → kebab" into the format namespaces (e.g.
> pdf:hasMarkedContent → pdf:has-marked-content, zip:compressedSize →
> zip:compressed-size, access_permission: → access-permission: ). Names from a
> standard stay verbatim (dc:, xmp:, EXIF/tiff:, MAPI, email headers, ECMA-376
> OOXML props, PDF/A/X sub-namespaces) — re-casing them would break
> round-trip/interop. Also fixes the QuattroPro/WordPerfect prefix collision.
> Consistency: a Tika-authored key is kebab wherever it lives (under tk: or a
> format prefix); a standard/file key keeps its source spelling — one rule
> answers casing everywhere.
>
> If we did steps 2 or 3, we'd probably want to release a 4.0.0-beta-2.
>
> Steps 2 or 3 would be fairly major changes for downstream users, but it would
> allow us to clean up years of metadata key name bloat.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
