progers-sumo commented on issue #1891: DRILL-7409: Moving test with big test data to the drill-test-framework. URL: https://github.com/apache/drill/pull/1891#issuecomment-550422224 @denysord88, thanks for the explanation. Developers often run the Drill unit tests to verify that a change works. We count on the unit tests to be complete. If some bit of functionality is not covered in unit tests, then it is possible to break that functionality in a PR, and to not learn of that error by running unit tests. When a test exists only in the test framework, we must rely on a MapR employee to run the pre-commit tests for us. If the test fails, we are not set up to run the test to track down the problem. The test framework works best when it acts as an end-to-end test at scale: across many data types, many query types and so on. If this particular test is only for one specific bit of functionality, and uses a small (80MB) file, perhaps it should remain in unit tests? The alternative is to package the test framework in a way that others can run it. Impala did a nice job of providing scripts to set up a "mini cluster" to allow running the E2E tests on a developer box (typically an Ubuntu system in AWS.) Is there a way to retain the test in the unit tests, perhaps with a smaller file? One would think that to test Int dictionary encoding, one could do so with a file of, say, a dozen or a hundred rows, which should result in a small Parquet file.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
