[
https://issues.apache.org/jira/browse/DRILL-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968547#comment-16968547
]
ASF GitHub Bot commented on DRILL-7409:
---------------------------------------
progers-sumo commented on issue #1891: DRILL-7409: Moving test with big test
data to the drill-test-framework.
URL: https://github.com/apache/drill/pull/1891#issuecomment-550422224
@denysord88, thanks for the explanation. Developers often run the Drill unit
tests to verify that a change works. We count on the unit tests to be complete.
If some bit of functionality is not covered in unit tests, then it is possible
to break that functionality in a PR, and to not learn of that error by running
unit tests.
When a test exists only in the test framework, we must rely on a MapR
employee to run the pre-commit tests for us. If the test fails, we are not set
up to run the test to track down the problem.
The test framework works best when it acts as an end-to-end test at scale:
across many data types, many query types and so on.
If this particular test is only for one specific bit of functionality, and
uses a small (80MB) file, perhaps it should remain in unit tests?
The alternative is to package the test framework in a way that others can
run it. Impala did a nice job of providing scripts to set up a "mini cluster"
to allow running the E2E tests on a developer box (typically an Ubuntu system
in AWS.)
Is there a way to retain the test in the unit tests, perhaps with a smaller
file? One would think that to test Int dictionary encoding, one could do so
with a file of, say, a dozen or a hundred rows, which should result in a small
Parquet file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Remove bigIntDictionary.parquet from project sources
> ----------------------------------------------------
>
> Key: DRILL-7409
> URL: https://issues.apache.org/jira/browse/DRILL-7409
> Project: Apache Drill
> Issue Type: Task
> Components: Tools, Build & Test
> Affects Versions: 1.16.0
> Reporter: Vova Vysotskyi
> Assignee: Denys Ordynskiy
> Priority: Minor
> Fix For: 1.17.0
>
>
> {{bigIntDictionary.parquet}} file has size of 1.8M, but it is used in single
> unit test {{TestColumnReaderFactory.testBigIntWithDictionary}}. We should
> either move this test to a test-framework or recreate a smaller file that
> will still allow us to verify this case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
