[
https://issues.apache.org/jira/browse/DRILL-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733047#comment-15733047
]
ASF GitHub Bot commented on DRILL-5070:
---------------------------------------
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/684
Golden copies are needed only for tests that, today, do their "testing" by
dumping generated code to stdout. Such tests do absolutely nothing (other than
detect crashes) when run under Maven. The golden copy assures that such
low-level tests are verified.
Other tests verify results by executing code. No need to save a golden copy
of code for those.
Yes, when code gen changes, the golden copies have to be recaptured.
However, how else can we verify that the new changes do what we expect if we
don't actually examine them and save the desired final state? How else do we
detect unexpected changes to generated code? (For example, if we generated a
bunch of unneeded boilerplate, we'd not detect that in an execution test.)
This test is also going to be a way to verify DRILL-5052: capture code
using the current code gen technique, then verify against the code path
described in DRILL-5052. The two paths should be identical except for an extra
"extends" in the code from DRILL-5052.
Is it worth testing such cases or is it OK to simply trust that the
generated code is probably OK?
> Code gen: create methods in fixed order to allow test verification
> ------------------------------------------------------------------
>
> Key: DRILL-5070
> URL: https://issues.apache.org/jira/browse/DRILL-5070
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
>
> A handy technique in testing is to compare generated code against a "golden"
> copy that defines the expected results. However, at present, Drill generates
> code using the method order returned by {{Class.getDeclaredMethods}}, but
> this method makes no guarantee about the order of the methods. The order
> varies from one run to the next. There is some evidence [this
> link|http://stackoverflow.com/questions/28585843/java-reflection-getdeclaredmethods-in-declared-order-strange-behaviour]
> that order can vary even within a single run, though a quick test was unable
> to reproduce this case.
> If method order does indeed vary within a single run, then the order can
> impact the Drill code cache since it compares the sources from two different
> generation events to detect duplicate code.
> This issue appeared when attempting to modify tests to capture generated code
> for comparison to future results. Even a simple generated case from
> {{ExpressionTest.testBasicExpression()}} that generates {{if(true) then 1
> else 0 end}} (all constants) produced methods in different orders on each
> test run.
> The fix is simple, in the {{SignatureHolder}} constructor, sort methods by
> name after retrieving them from the class. The sort ensures that method order
> is deterministic. Fortunately, the number of methods is small, so the sort
> step adds little cost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
