[
https://issues.apache.org/jira/browse/DRILL-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954272#comment-16954272
]
ASF GitHub Bot commented on DRILL-7403:
---------------------------------------
paul-rogers commented on issue #1871: DRILL-7403: Validate batch checks, vector
integretity in unit tests
URL: https://github.com/apache/drill/pull/1871#issuecomment-543515384
@arina-ielchiieva, addressed the comments. Since they were minor, went ahead
and squashed commits. Local tests passed up to the one that usually fails for
me:
```
[ERROR] Errors:
[ERROR]
TestDynamicUDFSupport.testReRegisterTheSameJarWithDifferentContent:600->BaseTestQuery.testRunAndReturn:340
ยป Rpc
```
Tried enabling the check for only the "new" scan. But, somehow, this still
checked the Parquet reader:
```
[INFO] Running org.apache.drill.exec.store.parquet2.TestDrillParquetReader
columns-offsets - UInt4Vector: Invalid offset at index 2049 = 4098 exceeds
maximum of 4096
columns-offsets - UInt4Vector: Invalid offset at index 2050 = 4100 exceeds
maximum of 4096
```
Since this PR is just about introducing the test code, I went ahead and
disabled calls to the code. Later PRs will try to enable the checks
operator-by-operator so we can find issues gradually.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Validate batch checks, vector integretity in unit tests
> -------------------------------------------------------
>
> Key: DRILL-7403
> URL: https://issues.apache.org/jira/browse/DRILL-7403
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.16.0, 1.17.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
> Fix For: 1.17.0
>
>
> Drill provides a {{BatchValidator}} that checks vectors. It is disabled by
> default. This enhancement adds more checks, including checks for row counts
> (of which there are surprisingly many.)
> Since most operators will fail if the check is enabled, this enhancement also
> adds a table to keep track of which operators pass the checks (and for which
> checks should be enabled) and those that still need work. This allows the
> checks to exist in the code, and to be enabled incrementally as we fix the
> various problems.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
