Lynnbb88 opened a new pull request, #3473: URL: https://github.com/apache/tika/pull/3473
## Issue [TIKA-4408](https://issues.apache.org/jira/browse/TIKA-4408) Python scripts starting with `#!/usr/bin/env python` or `#!/usr/bin/env python3` are detected as `application/x-sh`, including when a `.py` filename hint is supplied. The existing magic rule looks for `/bin/env python` at offset 1, where these inputs contain `!`. ## Changes Replace that rule with two literal prefixes anchored at offset 0: ```xml <match value="#!/usr/bin/env python" type="string" offset="0"/> <match value="#! /usr/bin/env python" type="string" offset="0"/> ``` This follows the paired no-space/single-space convention of the neighboring direct-path Python rules and preserves prefix matching for `python`, `python3`, and version suffixes such as `python3.11`. Add 18 regression checks covering content-only detection, `.py` filename hints, direct Python paths, Shell controls, filename-only detection, and file-start anchoring. Add a changelog entry for the unreleased version. No new dependencies or public API changes. General whitespace handling, `env` options before the interpreter (such as `-S`), and stricter interpreter-token boundaries are outside this change. ## Validation Fresh local checks on macOS with JDK 21, based on Apache `main` at `9f8b48425fea3630020924f6ae96e98eabb9ce12`: - With the new tests and the original production rule: 18 tests, 7 failures, 0 errors. - With the fix: all 18 Python regression checks passed. - Full `tika-core` unit suite: 1,418 tests reported, 0 failures, 0 errors, 2 skipped. - Alternate Xerces provider: 12 tests reported, 0 failures, 0 errors, 2 skipped. - `TikaIT`: 1 test passed; annotation processor: all 8 tests passed. - Checkstyle, Spotless, forbidden-API checks, and Apache RAT passed; `BUILD SUCCESS`. ```sh ./mvnw -B -ntp clean install -pl tika-core -am -Ppedantic \ -Dtika.test.locale=random -Dmaven.repo.local="$PWD/.local_m2_repo" ``` The negative control used the same command with `-Dtest=PythonShebangDetectionTest -Dsurefire.failIfNoSpecifiedTests=false`. These checks cover core and its reactor prerequisites; the full repository build has not been run. ## AI assistance OpenAI Codex assisted with investigation, implementation, testing, and PR preparation. The commit includes a `Generated-by` trailer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
