dashorst opened a new pull request, #1876:
URL: https://github.com/apache/wicket/pull/1876

   `XmlPullParser` now reads attributes the way the HTML tokenizer does, so `<a 
href=foo/bar.html>` keeps `href="foo/bar.html"` instead of turning into 
`href="foo" bar.html=""` (#1864). Auditing the parser against the HTML 
specification, with fuzzing and mutation testing, turned up twelve more defects 
in the same code: attribute names and quoted values, the quote scanning that 
finds the end of a tag, raw-text detection for `<script>` and `<style>`, CDATA, 
and two crashes. This PR fixes all of them and documents the parser's contract 
in its class Javadoc, which until now was one line.
   
   Fixes #1864
   
   ## Commits
   
   Each commit can be read on its own. The first adds every test, red where the 
parser was wrong; each later commit fixes one aspect and turns its own tests 
green. With this order, each test can be seen to fail before its fix.
   
   1. **Document XmlPullParser and test it against the HTML tokenizer.** Class 
Javadoc, plus `XmlPullParserAttributeTest`, `XmlPullParserRawTextTest`, and new 
tests in `XmlPullParserTest` and `FullyBufferedReaderTest`. 
   2. **Find the end of a tag without backslash escapes or recursion.** 
`FullyBufferedReader.findOutOfQuotes`: `title="Next >>"` was rejected, about 
ten thousand `>` in one quoted value overflowed the stack, and `\"` counted as 
an escaped quote although HTML, XML and the attribute parser have no escapes.
   3. **Read attribute names and unquoted values the way HTML does** (#1864). 
Unquoted values run to whitespace. Names such as `(click)`, `[value]`, `#ref` 
and `on:click|once` are kept as written. Text that doesn't match is no longer 
skipped silently, and quoted values are no longer trimmed.
   4. **Decide on raw text for script and style by element name and type.** 
Fixes `<style>` without attributes (off by one), custom elements like 
`<script-loader>` swallowing their content, `type` sniffed by substring 
(`data-type=`, unquoted, empty, `application/javascript`, JSON), raw text 
ending at `</scripts`, and a `StringIndexOutOfBoundsException` on `<script 
type=>`.
   5. **Accept quotes in CDATA sections and reject `</>` with a 
ParseException.**
   
   ## Where to look
   
   Core: `wicket-core/.../markup/parser/XmlPullParser.java`. Read 
`parseAttributes` and `isScriptOrDataBlock` alongside the class Javadoc, which 
states the contract they implement. Then 
`wicket-util/.../io/FullyBufferedReader.java#findOutOfQuotes`.
   
   Tests (can be skimmed): the four test classes. The expected values come from 
the HTML specification, not from what the code returned.
   
   `MetaPattern.STRING` and `VariableAssignmentParser` are deliberately 
unchanged. `ValueMap` and `CommaSeparatedVariableParser` use them too, and they 
are public API. The parser now has its own attribute tokenizer instead.
   
   ## Behaviour changes
   
   No public API changes. Templates that parsed before may now parse 
differently:
   
   - Unquoted values containing `/`, `:`, `?` and the like, and framework 
attribute names (Angular, Vue, Alpine, Svelte, htmx), are now rendered as 
written in component tags and in tags changed by filters.
   - Whitespace around a quoted value is kept: `class=" a "` used to become 
`class="a"`.
   - Text in a tag that isn't an attribute follows HTML recovery: `<tag a="1" 
"junk">` gains an attribute named `"junk"` instead of `junk`. A duplicate 
attribute is still a `ParseException`, as before.
   - `\"` inside a quoted value now ends the value. Before, the bracket finder 
treated it as an escape (WICKET-3773), but the attribute parser didn't.
   - A `<script>` with an empty type, `application/javascript` or a JSON type 
is now raw text. It used to be parsed as a template, so a Wicket component 
inside one is no longer found. Other non-JavaScript types are still templates 
(WICKET-5288).
   - A custom element whose name starts with `script` or `style` is now parsed 
as markup instead of raw text.
   
   ## Testing
   
   - `mvn clean verify -Pjs-test` is green.
   - Crash fuzzing: 1.5 million random markup strings, with only a 
`ParseException` allowed. It found `</>` and `<script type=>`. A seeded 
20,000-input version is in 
`XmlPullParserTest.anyMarkupEitherParsesOrIsRejectedWithParseException`.
   - Differential fuzzing against jsoup's HTML5 tokenizer, run outside the 
repository because jsoup is not a dependency: 400,000 generated attribute 
lists, zero differences after the fix. The only remaining difference was 
numeric character references without a closing `;` (`&#9`), which 
`Strings.unescapeMarkup` does not decode. That is entity decoding rather than 
tokenizing, so it is left for a separate issue.
   - PIT on `XmlPullParser` and `FullyBufferedReader` with these tests kills 
192 of 210 mutants. The survivors are equivalent mutants, `getString()` of a 
conditional comment (which nothing reads), a `>` check in `skipUntil` that 
duplicates what `next()` does, and helpers this PR doesn't touch.
   
   Not changed: for a plain text token, `getString()` returns text from the 
position marker rather than the token alone. Nothing in Wicket calls it.
   
   Backporting to `wicket-11.x` / `wicket-10.x` is a separate decision. The 
behaviour changes above argue for master only.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to