cgivre commented on pull request #2122: URL: https://github.com/apache/drill/pull/2122#issuecomment-740768976
> So I tried to fix the test which was really hard because the ordering of the columns in case of a wildcard was based on the hash (i.e. visually random). > > * I made the ordering of the columns in case of the wildcard a stable textual ordering (by means of a TreeMap instead of a HashMap). > * I found that the Casts were the same for all columns (which is incorrect, so I fixed that). Can you clarify this? The unit tests do verify that Drill is producing the correct data types and they seemed to be working. > * As a consequence the tests become much easier to read because everything is sorted. :-) > > At this point I think the type remapping doesn't work. > I haven't seen any testing code or documentation so I'm not sure. Can you clarify which type remapping you're referring to? When we did the refactoring, I added unit tests that check for data types. Most fields are strings, so they get mapped to `VARCHAR` types. However, the timestamps and dates get mapped to the appropriate data types. Also, the pieces of the times also get mapped to `DOUBLE` data types. (I think). You can confirm that in each unit test where it constructs a schema. https://github.com/apache/drill/blob/7a614d8ce7a0018c6ffaa12955d56366ee62a6d6/contrib/format-httpd/src/test/java/org/apache/drill/exec/store/httpd/TestHTTPDLogReader.java#L224-L228 > Because of the lack of testing I'm pretty sure one of my changes broke it. > @cgivre If you can please give me a hint on how you intended it to work I can pick that up for a test to include. I actually liked your original implementation with the config option. What I was thinking however was that if the option was enabled, dissector would only be added if the user executed a star query OR they included one of the raw `user-agent` fields in the query. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
