[
https://issues.apache.org/jira/browse/DRILL-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170143#comment-15170143
]
ASF GitHub Bot commented on DRILL-4437:
---------------------------------------
GitHub user jaltekruse opened a pull request:
https://github.com/apache/drill/pull/394
DRILL-4437: Operator unit tests
Introduces a framework for testing physical operators without the need to
start a complete Drill server.
Several small interface changes were made to allow for the test
declarations to be cleaner or to share code with some existing test
infrastructure. I went ahead and opened JIRAs for them, but they are included
on this branch as individual commits. If any of these changes are too large I
can break them out into separate reviews.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jaltekruse/incubator-drill 4437-op-unit-tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/394.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #394
----
commit 571c90cfda04310b3ff659a42c13f52b714848e5
Author: Jason Altekruse <[email protected]>
Date: 2016-02-11T01:36:47Z
DRILL-4445: Standardize the Physical and Logical plan nodes to use Lists
instead of arrays for their inputs
Remove some extra translation logic used to move between the
two representations.
TODO - look back the the Join logical node, has two JsonCreator annotations,
but only one will be used. Not sure if the behavior of which is chosen
is considered documented behavior, should just fix it on our end.
commit e034dc5276af51241afba839cdf5da2c4930caf7
Author: Jason Altekruse <[email protected]>
Date: 2016-02-25T03:37:21Z
DRILL-4448: Clean up deserialization of oderings in sorts
commit 9c2b061610be052b1b05d2e86228b6914a395cc8
Author: Jason Altekruse <[email protected]>
Date: 2016-02-26T22:11:47Z
DRILL-4442: Move getSV2 and getSV4 methods to VectorAccessible
Up one level from previous location RecordBatch, most implementations
already implement the method as they implement RecordBatch rather than
VectorAccessible itself. Add unsupported operation exception to others.
commit 28280456f4f0766d6b2e1c78dff5334cde56afe3
Author: Jason Altekruse <[email protected]>
Date: 2016-02-26T22:55:30Z
DRILL-4437: Operator unit test framework
----
> Implement framework for testing operators in isolation
> ------------------------------------------------------
>
> Key: DRILL-4437
> URL: https://issues.apache.org/jira/browse/DRILL-4437
> Project: Apache Drill
> Issue Type: Test
> Components: Tools, Build & Test
> Reporter: Jason Altekruse
> Assignee: Jason Altekruse
> Fix For: 1.6.0
>
>
> Most of the tests written for Drill are end-to-end. We spin up a full
> instance of the server, submit one or more SQL queries and check the results.
> While integration tests like this are useful for ensuring that all features
> are guaranteed to not break end-user functionality overuse of this approach
> has caused a number of pain points.
> Overall the tests end up running a lot of the exact same code, parsing and
> planning many similar queries.
> Creating consistent reproductions of issues, especially edge cases found in
> clustered environments can be extremely difficult. Even the simpler case of
> testing cases where operators are able to handle a particular series of
> incoming batches of records has required hacks like generating large enough
> files so that the scanners happen to break them up into separate batches.
> These tests are brittle as they make assumptions about how the scanners will
> work in the future. An example of when this could break, we might do perf
> evaluation to find out we should be producing larger batches in some cases.
> Existing tests that are trying to test multiple batches by producing a few
> more records than the current threshold for batch size would not be testing
> the same code paths.
> We need to make more parts of the system testable without initializing the
> entire Drill server, as well as making the different internal settings and
> state of the server configurable for tests.
> This is a first effort to enable testing the physical operators in Drill by
> mocking the components of the system necessary to enable operators to
> initialize and execute.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
