We should use the structured response (and do structural comparisons) for plan verification. The text plan should be cleaned up so that it is more useful and user friendly. We definitely shouldn't depend on what is effectively a random toString method for functional verification (I know we do now). We should figure out a way to implement the abstract functionality verification using a builder pattern or similar. Think Calcite's RelBuilder with a more generic interfaces. Maybe we create a JsonRelMatcher tree and then we can use that to verify a plan.
As I started to write up an example expression proposal, I start to think: this is arbitrary json tree matching, any way we can use something that already exists. For example, there are probably 100 json libraries that do basically this... thoughts? -- Jacques Nadeau CTO and Co-Founder, Dremio On Mon, Oct 26, 2015 at 6:32 PM, rahul challapalli < [email protected]> wrote: > Drillers, > > We have a handful of tests in the drill test framework (Refer to [1] for > some examples) which compare the text version of the plan to an expected > result. The goal is to compare the structure of the text plan and also > selected attributes of some operators ( Eg : numFiles, list of scanned > files). The current implementation uses a regex based comparison where the > QA person constructs an equivalent regex based expected file. This reduces > the QA productivity and is not flexible w.r.t to moving parts in the plan. > Any thoughts around this? > > Few suggestions : > 1. Run explain plan with a flag so that the text plan returned is > deterministic and does not contain any moving parts. This version of the > plan returned can contain the list of files scanned in an alphabetically > sorted order or in some way which is deterministic. > 2. Use the JSON plan and do an operator DAG based comparison. This can be > very flexible. Even with this we still need to have some tests against the > Text version of the plan. > > [1] > > https://github.com/mapr/drill-test-framework/tree/master/framework/resources/Functional/partition_pruning/dfs/csv/plan > > - Rahul >
