[
https://issues.apache.org/jira/browse/DRILL-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094201#comment-15094201
]
Jason Altekruse edited comment on DRILL-4259 at 1/14/16 9:41 PM:
-----------------------------------------------------------------
That's what I was thinking, although I'm not very familiar with the setup of
the regression tests. I know that some tests set options themselves by just
including the alter sessions statements in the test files. I also know that we
added an enhancement for the framework to set options globally, so that we
could leave the decimal tests turned on [1] (even though we turned of the
feature by default due to some lingering pref and correctness issues).
I think what I would be looking to do would be configure a few different
combinations of configurations that could be applied over an entire suite, so
that we can run a number of tests with a few different configurations. Doing
this by manually changing the test files to include near copies with the new
options set doesn't seem like the right way to go. Let me know if this seems
like a reasonable approach. I can work on the enhancement to the framework to
enable it if necessary.
[1] -
https://github.com/mapr/drill-test-framework/blob/master/framework/before-run.sql
was (Author: jaltekruse):
That's what I was thinking, although I'm not very familiar with the setup of
the regression tests. I know that some tests set options themselves by just
including the alter sessions statements in the test files. I also know that we
added an enhancement for the framework to set options globally, so that we
could leave the decimal tests turned on [1] (even though we turned of the
feature by default due to some lingering pref and correctness issues).
I think what I would be looking to do would be configure a few different
combinations of configurations that could be applied over an entire suite, so
that we can run a number of tests with a few different configurations. Doing
this by manually changing the test files to include near copes with the new
options set doesn't seem like the right way to go. Let me know if this seems
like a reasonable approach. I can work on the enhancement to the framework to
enable it if necessary.
[1] -
https://github.com/mapr/drill-test-framework/blob/master/framework/before-run.sql
> Add new functional tests to ensure that failures can be detected independent
> of the testing environment
> -------------------------------------------------------------------------------------------------------
>
> Key: DRILL-4259
> URL: https://issues.apache.org/jira/browse/DRILL-4259
> Project: Apache Drill
> Issue Type: Test
> Reporter: Jason Altekruse
>
> In DRILL-4243 an out of memory issue was fixed after a change to the memory
> allocator made memory limits more strict. While the regression tests had been
> run by the team at Dremio prior to merging the patch, running the tests on a
> cluster with more cores changed the memory limits on the queries and caused
> several tests to fail.
> While changes of this magnitude are not going to be common, we should have a
> test suite that reliably fails independent of the environment it is run
> (assuming that there are sufficient resources for the tests to run).
> It would be good to at least try to reproduce this failure on a few different
> setups (cores, nodes in cluster) by adjusting available configuration options
> and adding tests with those different configurations so that the tests will
> fail in different environments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
