With the json tree matching, we still want to control what determines an equality between 2 json tree's at the level of each operator. We should be able to ignore the attributes/child-nodes which represent any costing information or moving parts. With this requirement don't we still need to come up with CustomMatcher?
Even if we end up using the JSON plan for the majority of the functional tests, we should still have tests against the text version of the plan. Can we somehow standardize on the contents of the text plan instead of relying on the toString method of the respective operators? - Rahul On Tue, Oct 27, 2015 at 9:25 AM, Jacques Nadeau <[email protected]> wrote: > We should use the structured response (and do structural comparisons) for > plan verification. The text plan should be cleaned up so that it is more > useful and user friendly. We definitely shouldn't depend on what is > effectively a random toString method for functional verification (I know we > do now). We should figure out a way to implement the abstract functionality > verification using a builder pattern or similar. Think Calcite's RelBuilder > with a more generic interfaces. Maybe we create a JsonRelMatcher tree and > then we can use that to verify a plan. > > As I started to write up an example expression proposal, I start to think: > this is arbitrary json tree matching, any way we can use something that > already exists. For example, there are probably 100 json libraries that do > basically this... > > thoughts? > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Mon, Oct 26, 2015 at 6:32 PM, rahul challapalli < > [email protected]> wrote: > > > Drillers, > > > > We have a handful of tests in the drill test framework (Refer to [1] for > > some examples) which compare the text version of the plan to an expected > > result. The goal is to compare the structure of the text plan and also > > selected attributes of some operators ( Eg : numFiles, list of scanned > > files). The current implementation uses a regex based comparison where > the > > QA person constructs an equivalent regex based expected file. This > reduces > > the QA productivity and is not flexible w.r.t to moving parts in the > plan. > > Any thoughts around this? > > > > Few suggestions : > > 1. Run explain plan with a flag so that the text plan returned is > > deterministic and does not contain any moving parts. This version of the > > plan returned can contain the list of files scanned in an alphabetically > > sorted order or in some way which is deterministic. > > 2. Use the JSON plan and do an operator DAG based comparison. This can be > > very flexible. Even with this we still need to have some tests against > the > > Text version of the plan. > > > > [1] > > > > > https://github.com/mapr/drill-test-framework/tree/master/framework/resources/Functional/partition_pruning/dfs/csv/plan > > > > - Rahul > > >
