allthingssecurity opened a new pull request, #28595: URL: https://github.com/apache/camel/pull/28595
# Description [CAMEL-26087](https://issues.apache.org/jira/browse/CAMEL-26087) Two cases where `xtokenize` (`XMLTokenExpressionIterator`) fails on valid XML: 1. A document whose prolog has a comment or a doctype and no XML declaration fails in every mode with `XMLStreamException: XMLStreamReader does not supporting getting location`, for example `<!-- generated by the order export -->\n<orders><order>1</order>...`. The second compliance check (CAMEL-8164) takes the text up to the location of the first start element and treats it as proof of a reader without usable locations when it starts with `<` but not with `<?`. With a compliant reader (Woodstox, which camel-stax depends on) that text is the prolog, so a comment or doctype before the root trips it. The message points at the StAX implementation, not at the input, and nothing is split. Files from exporters that start with a licence or "generated by" comment are common. 2. Mode `w` with a descendant path (`//order`) whose matches are at different depths: `<orders><order>1</order><box><order>2</order></box></orders>` routes the first token and then fails with `IndexOutOfBoundsException: Index -1 out of bounds for length 0`, so the rest of the document is lost. With a container inside a container of the same name the wrapper of a later token is unbalanced: `<orders><box><box><order>1</order></box></box><order>2</order></orders>` gives `<orders><box><order>2</order></orders>` as the second token. The wrap mode keeps one text segment per start tag it saw, logged with the name of the element. When an element ends after a match, it popped the log down to the first entry with that name. An element that starts right after a match gets no entry of its own (it shares the entry of its previous sibling, so matched siblings do not pile up in the wrapper), so its end popped the whole log; and an inner element of the same name stopped the pop too early. Changes: - The compliance check now looks at what the reader read after that text: a reader that reports the location of the start of the start tag has not consumed the tag into the token yet, so the text read after it starts with `<name` followed by `>`, `/` or white space (`RecordableReader.startsWithStartTag`). A reader that reports the location after the start tag fails the check as before. I checked both with the readers at hand: Woodstox 7.3.0 reports the start of the tag (offset 0, 21, 11 and 16 for no prolog, a declaration, a comment and a doctype), the JDK reader reports the end of it (6, 17 and 28 without a declaration; with one it already fails the first check), so the JDK reader is still refused. A comment that contains the root's start tag (`<!-- starts with <orders> -->`) is no longer a problem either. A reader that reports the end of the tag is still taken for a compliant one when the root's first child has the same name and follows it directly (`<a><a>`). - Mode `w` records the depth of each segment instead of the name, and the end of an element removes the segments deeper than it. The segment at the element's own depth stays for its next sibling, as before, and the existing wrap mode tests pass unchanged. The preceding siblings in the wrapper (CAMEL-17466) are unchanged. - No upgrade guide entry: input that failed is now split, and tokens that were not well-formed now are; nothing for users to change. - Not in this PR: the text mode and the `>` in an attribute value are in CAMEL-26087 (separate PR, same class; the two branches merge cleanly). - Backport: camel-4.23.x, 4.22.x, 4.18.x and 4.14.x have the same code (read, not built). Tests: - New `XMLTokenizePrologAndWrapDepthTest`, 5 tests, all fail on main: - `documentStartingWithACommentIsSplit`, `documentStartingWithADoctypeIsSplit`, `commentWithTheStartTagOfTheRootIsSplit`: `XMLStreamException: XMLStreamReader does not supporting getting location`; - `wrapModeWithMatchesAtDifferentDepths` (three matches, one in a `box`): `IndexOutOfBoundsException: Index -1 out of bounds for length 0`; - `wrapModeWithAContainerInAContainerOfTheSameName`: `Expected: <<orders><order>2</order></orders>> but was: <<orders><box><order>2</order></orders>>`. - Unit tests of camel-stax with the fix: 116 run, 0 failures, 2 skipped (both skipped on main too), including the existing wrap mode tests. - A differential run against a namespace-aware DOM parser on 20,000 generated well-formed documents: on main 3,065 of them (all that start with a comment and have no declaration) fail in every mode, and mode `w` with `//*:order` fails on 633 more and gives 72 tokens that are not well-formed; with this change none fail, and mode `w` gives the matched element in every token for `//*:order`. With `/*:root/*:order` the only differences left are the preceding siblings of CAMEL-17466. - The formatter and import-sort plugins ran. I built offline with `-Djackson3-version=3.2.3` (3.2.4 is not in my local repository) and `-Dasm-version=9.11`. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26087](https://issues.apache.org/jira/browse/CAMEL-26087). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built camel-stax and ran its tests, including the formatter and import-sort plugins. No generated file changes. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
