nick-boss-tech opened a new pull request, #5103: URL: https://github.com/apache/solr/pull/5103
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-3704 ## What happens today **Highlighting a stored DatePointField fails with "Invalid Date String".** The highlighter reads each stored value as text through the field type. For a DatePointField, that text is the stored epoch milliseconds. For 2012-08-01T00:00:01.999Z, the text is `1343779201999` ([FieldType.java L405-L413](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/schema/FieldType.java#L405-L413)). The highlighter's analyzer passes that text to the date parser, which rejects it ([DatePointField.java L197-L198](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/schema/DatePointField.java#L197-L198)). ## What this change does **Date values now reach the highlighter as ISO instants, such as `2012-08-01T00:00:01.999Z`.** - `DatePointField.toExternal` returns the ISO form ([DatePointField.java L161-L163](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/schema/DatePointField.java#L161-L163)). - A value that is already a `java.util.Date` gets the same form ([DefaultSolrHighlighter.java L828-L829](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java#L828-L829)). Date docValues fields that are not stored reach the highlighter this way ([SolrDocumentFetcher.java L735-L736](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/search/SolrDocumentFetcher.java#L735-L736)). - `DatePointField.toExternal` is also used outside the highlighter. Luke's per-field value ([LukeRequestHandler.java L788](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/handler/admin/LukeRequestHandler.java#L788)) and the printed value of a date unique key ([IndexSchema.java L360](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/schema/IndexSchema.java#L360), [L369](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/java/org/apache/solr/schema/IndexSchema.java#L369)) now use the ISO form. ## Proof **The new test fails on the base code and passes with this change.** - `HighlighterTest.testHighlightDatePointFieldDoesNotFail` ([HighlighterTest.java L91-L125](https://github.com/nick-boss-tech/solr/blob/de63d4e5d5d1ddde0da6a100a631e254c078bd55/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java#L91-L125)) indexes `2012-08-01T00:00:01.999Z` in a stored date field. It runs a range query with `hl.fl` on that field, once per highlighter method. A second query highlights a text field next to it. - On the base code, with this test file, HighlighterTest runs 36 tests with 1 failure, which is this test, failing with `Invalid Date String:'1343779201999'`. - With this change, HighlighterTest passes 36 of 36 at this head. - Extra runs at this head: LukeRequestHandlerTest 8 of 8, TestPointFields 104 of 104. - Tidy, the Error Prone compile, and the module check pass at this head. The changelog YAML parses. ## Limits **Tested: stored date fields. Not tested: docValues-only date fields and date unique keys.** - No test in this change covers a docValues-only date field. That path takes the `java.util.Date` branch. - The extra runs above are LukeRequestHandlerTest and TestPointFields. No test in this change covers a date unique key. - The UnifiedHighlighter code is not changed. The new test runs `hl.method=unified` with a match-count check only. Changelog: `changelog/unreleased/SOLR-3704-highlight-date-point.yml` ### AI assistance AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
