serhiy-bzhezytskyy opened a new issue, #73: URL: https://github.com/apache/solr-orbit/issues/73
### Description `convert-workload` mistranslates two kinds of OpenSearch date bound. Both are independent of #68 (bracket inclusivity) and both survive the fix in #69, because they happen earlier, in `_convert_date_to_solr_format`. **1. A datetime with a space separator is passed through verbatim, and Solr rejects it** `yyyy-MM-dd HH:mm:ss` is not in the format map, so the value is logged and used as-is. A space inside a range term ends the term, and Solr answers HTTP 400 — the converted operation cannot run at all. ``` $ convert-workload # nyc_taxis date_histogram_calendar_interval Unknown OpenSearch date format: 'yyyy-MM-dd HH:mm:ss' Could not parse date '2015-01-01 00:00:00', using as-is -> dropoff_datetime:[2015-01-01 00:00:00 TO 2016-01-01 00:00:00] ``` Against Solr 10.0.0: ``` HTTP 400 org.apache.solr.search.SyntaxError: Cannot parse 'dropoff_datetime:[2015-01-01 00:00:00 TO 2016-01-01 00:00:00]': Encountered " <RANGE_GOOP> "00:00:00 "" at line 1, column 28. ``` Two nyc_taxis operations (`date_histogram_calendar_interval`, `date_histogram_fixed_interval`) are affected. **2. A whole-day `lte` is not rounded, and loses a day** A bound with no time names a *day*, and OpenSearch rounds it to that day's edge: `lte` and `gt` to its LAST millisecond, `gte` and `lt` to its first. Solr rounds nothing. Only the two that move to the end of the day need translating, and the converter translates neither. For nyc_taxis `date_histogram_facet` (`lte: 21/01/2015`, format `dd/MM/yyyy`), upstream covers the whole of 21 January; the converter cuts the filter at the first instant of the 21st. ### Measured effect 300,649 nyc_taxis documents indexed into Solr 10.0.0 and OpenSearch 3.8.0 from one file, with the same ids, ids and contents verified on both sides: | | OpenSearch | converted | corrected | |---|---:|---:|---:| | `date_histogram_facet` matches | 17,651 | 16,811 | 17,651 | | daily buckets returned | 21 | 20 | 21 | The id sets are equal after the fix; before it, 840 documents are missing and 0 are extra, and the bucket for `2015-01-21` is absent entirely rather than short. ### Expected Both bounds should convert so the Solr query selects the same documents as the OpenSearch query it came from: ``` dropoff_datetime:[2015-01-01T00:00:00Z TO 2016-01-01T00:00:00Z} dropoff_datetime:[2015-01-01T00:00:00Z TO 2015-01-22T00:00:00Z} ``` Naming the following day's first instant with an exclusive bracket says "the whole of the 21st" without depending on how fine Solr's date precision happens to be. ### Version `main` at 6c6c48a, and reproduced with #69 applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
