[jira] [Commented] (DRILL-5318) Create a sub-operator test framework
[
https://issues.apache.org/jira/browse/DRILL-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977742#comment-15977742
]
ASF GitHub Bot commented on DRILL-5318:
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/788
Revised the commit header. Note that the commit messages also identifies
that this PR is dependent on DRILL-5323, and so this PR must be committed after
DRILL-5323.
Sorry for the confusion. The work was broken into multiple commits to ease
the burden on code reviewers. Unfortunately, there is no free lunch, and the
multiple commits puts a large burden on the submitter to keep all branches in
sync, and on the committer to pull the PRs in the correct order.
> Create a sub-operator test framework
>
>
> Key: DRILL-5318
> URL: https://issues.apache.org/jira/browse/DRILL-5318
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.11.0
>
> Attachments: Sub-OperatorTestFramework.pdf
>
>
> Drill provides two unit test frameworks for whole-server, SQL-based testing:
> the original {{BaseTestQuery}} and the newer {{ClusterFixture}}. Both use the
> {{TestBuilder}} mechanism to build system-level functional tests that run
> queries and check results.
> Jason provided an operator-level test framework based, in part on mocks:
> As Drill operators become more complex, we have a crying need for true
> unit-level tests at a level below the whole system and below operators. That
> is, we need to test the individual pieces that, together, form the operator.
> This umbrella ticket includes a number of tasks needed to create the
> sub-operator framework. Our intention is that, over time, as we find the need
> to revisit existing operators, or create new ones, we can employ the
> sub-operator test framework to exercise code at a finer granularity than is
> possible prior to this framework.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5318) Create a sub-operator test framework
[
https://issues.apache.org/jira/browse/DRILL-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977737#comment-15977737
]
ASF GitHub Bot commented on DRILL-5318:
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/788
Please see the initial comment which identifies that this PR depends on
several others. Two have been done, only DRILL-5323 remains. Please commit
DRILL-5323 first, then this one.
Note that, per your request, this PR is rebased onto DRILL-5323. The
history here includes all changes from DRILL-5323 plus additional changes
specific to DRILL-5318. As I understand it, if you first commit DRILL-5323 the
common changes will reside in master, and will be ignored when you then (as a
second step) commit this PR.
> Create a sub-operator test framework
>
>
> Key: DRILL-5318
> URL: https://issues.apache.org/jira/browse/DRILL-5318
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.11.0
>
> Attachments: Sub-OperatorTestFramework.pdf
>
>
> Drill provides two unit test frameworks for whole-server, SQL-based testing:
> the original {{BaseTestQuery}} and the newer {{ClusterFixture}}. Both use the
> {{TestBuilder}} mechanism to build system-level functional tests that run
> queries and check results.
> Jason provided an operator-level test framework based, in part on mocks:
> As Drill operators become more complex, we have a crying need for true
> unit-level tests at a level below the whole system and below operators. That
> is, we need to test the individual pieces that, together, form the operator.
> This umbrella ticket includes a number of tasks needed to create the
> sub-operator framework. Our intention is that, over time, as we find the need
> to revisit existing operators, or create new ones, we can employ the
> sub-operator test framework to exercise code at a finer granularity than is
> possible prior to this framework.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (DRILL-5440) Sqlline is not connecting to Hive database
[
https://issues.apache.org/jira/browse/DRILL-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parag Darji updated DRILL-5440:
---
Summary: Sqlline is not connecting to Hive database (was: Sqlline is not
showing Hive database)
> Sqlline is not connecting to Hive database
> --
>
> Key: DRILL-5440
> URL: https://issues.apache.org/jira/browse/DRILL-5440
> Project: Apache Drill
> Issue Type: Task
> Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled,
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
> Labels: newbie
> Fix For: 1.10.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Sqlline is not showing hive database.
> {code}
> sqlline -u "jdbc:drill:schema:hiveremote;drillbit=host1.fqdn;auth=kerberos"
> apache drill 1.10.0
> "what ever the mind of man can conceive and believe, drill can query"
> 0: jdbc:drill:schema:hiveremote> show schemas;
> +-+
> | SCHEMA_NAME |
> +-+
> | INFORMATION_SCHEMA |
> | sys |
> +-+
> 2 rows selected (2.074 seconds)
> 0: jdbc:drill:schema:hiveremote> show databases;
> +-+
> | SCHEMA_NAME |
> +-+
> | INFORMATION_SCHEMA |
> | sys |
> +-+
> 2 rows selected (0.226 seconds)
> 0: jdbc:drill:schema:hiveremote> !connect jdbc:mysql://hive.fqdn:10010/hive
> Enter username for jdbc:mysql://hive.fqdn:10010/hive:
> {code}
> I've kerberos enabled so I tried below command, but it's giving auth error.
> {code}
> sqlline --maxWidth=1 -u
> "jdbc:drill:schema=hive:drillbit=host1.fqdn;auth=kerberos;principal=drill/host1.f...@lab.com;keytab:/home/drill/.keytab/drill.keytab"
> error: Error: Failure in connecting to Drill:
> org.apache.drill.exec.rpc.NonTransientRpcException:
> javax.security.sasl.SaslException: Authentication failed: Authentication
> failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
> Authentication failed. Incorrect credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill:
> org.apache.drill.exec.rpc.NonTransientRpcException:
> javax.security.sasl.SaslException: Authentication failed: Authentication
> failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
> Authentication failed. Incorrect credentials?]
> {code}
> When I try to use username/password, it doesn't return anything, just sits
> there for hours
> {code}
> drill@:/home/drill> sqlline -u "jdbc:drill:drillbit=host1.fqdn;auth=kerberos"
> apache drill 1.10.0
> "drill baby drill"
> 0: jdbc:drill:drillbit=> !connect
> Usage: connect[driver]
> 0: jdbc:drill:drillbit=> !connect jdbc:mysql://hivenode.fqdn:10010/hive test
> test
> {code}
> Below query doesn't return anything:
> {code}
> sqlline -u
> "jdbc:mysql://hive.fqdn:10010/default;zk=host1.fqdn:2181;service_name=drill;service_host=host2.fqdn;keytab=/home/drill/.keytab/drill.keytab";user="drill/host2.f...@lab.com"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (DRILL-5440) Sqlline is not showing Hive database
[
https://issues.apache.org/jira/browse/DRILL-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parag Darji updated DRILL-5440:
---
Description:
Sqlline is not showing hive database.
{code}
sqlline -u "jdbc:drill:schema:hiveremote;drillbit=host1.fqdn;auth=kerberos"
apache drill 1.10.0
"what ever the mind of man can conceive and believe, drill can query"
0: jdbc:drill:schema:hiveremote> show schemas;
+-+
| SCHEMA_NAME |
+-+
| INFORMATION_SCHEMA |
| sys |
+-+
2 rows selected (2.074 seconds)
0: jdbc:drill:schema:hiveremote> show databases;
+-+
| SCHEMA_NAME |
+-+
| INFORMATION_SCHEMA |
| sys |
+-+
2 rows selected (0.226 seconds)
0: jdbc:drill:schema:hiveremote> !connect jdbc:mysql://hive.fqdn:10010/hive
Enter username for jdbc:mysql://hive.fqdn:10010/hive:
{code}
I've kerberos enabled so I tried below command, but it's giving auth error.
{code}
sqlline --maxWidth=1 -u
"jdbc:drill:schema=hive:drillbit=host1.fqdn;auth=kerberos;principal=drill/host1.f...@lab.com;keytab:/home/drill/.keytab/drill.keytab"
error: Error: Failure in connecting to Drill:
org.apache.drill.exec.rpc.NonTransientRpcException:
javax.security.sasl.SaslException: Authentication failed: Authentication
failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
Authentication failed. Incorrect credentials?] (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill:
org.apache.drill.exec.rpc.NonTransientRpcException:
javax.security.sasl.SaslException: Authentication failed: Authentication
failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
Authentication failed. Incorrect credentials?]
{code}
When I try to use username/password, it doesn't return anything, just sits
there for hours
{code}
drill@:/home/drill> sqlline -u "jdbc:drill:drillbit=host1.fqdn;auth=kerberos"
apache drill 1.10.0
"drill baby drill"
0: jdbc:drill:drillbit=> !connect
Usage: connect[driver]
0: jdbc:drill:drillbit=> !connect jdbc:mysql://hivenode.fqdn:10010/hive test
test
{code}
Below query doesn't return anything:
{code}
sqlline -u
"jdbc:mysql://hive.fqdn:10010/default;zk=host1.fqdn:2181;service_name=drill;service_host=host2.fqdn;keytab=/home/drill/.keytab/drill.keytab";user="drill/host2.f...@lab.com"
{code}
was:
Sqlline is not showing hive database.
{code}
sqlline -u "jdbc:drill:schema:hiveremote;drillbit=host1.fqdn;auth=kerberos"
apache drill 1.10.0
"what ever the mind of man can conceive and believe, drill can query"
0: jdbc:drill:schema:hiveremote> show schemas;
+-+
| SCHEMA_NAME |
+-+
| INFORMATION_SCHEMA |
| sys |
+-+
2 rows selected (2.074 seconds)
0: jdbc:drill:schema:hiveremote> show databases;
+-+
| SCHEMA_NAME |
+-+
| INFORMATION_SCHEMA |
| sys |
+-+
2 rows selected (0.226 seconds)
0: jdbc:drill:schema:hiveremote> !connect jdbc:mysql://hive.fqdn:10010/hive
Enter username for jdbc:mysql://hive.fqdn:10010/hive:
{code}
I've kerberos enabled so I tried below command, but it's giving auth error.
{code}
sqlline --maxWidth=1 -u
"jdbc:drill:schema=hive:drillbit=host1.fqdn;auth=kerberos;principal=drill/host1.f...@lab.com;keytab:/home/drill/.keytab/drill.keytab"
error: Error: Failure in connecting to Drill:
org.apache.drill.exec.rpc.NonTransientRpcException:
javax.security.sasl.SaslException: Authentication failed: Authentication
failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
Authentication failed. Incorrect credentials?] (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill:
org.apache.drill.exec.rpc.NonTransientRpcException:
javax.security.sasl.SaslException: Authentication failed: Authentication
failed. Incorrect credentials? [Caused by javax.security.sasl.SaslException:
Authentication failed. Incorrect credentials?]
{code}
When I try to use username/password, it doesn't return anything, just sits
there for hours
{code}
drill@:/home/drill> sqlline -u "jdbc:drill:drillbit=host1.fqdn;auth=kerberos"
apache drill 1.10.0
"drill baby drill"
0: jdbc:drill:drillbit=> !connect
Usage: connect[driver]
0: jdbc:drill:drillbit=> !connect jdbc:mysql://hivenode.fqdn:10010/hive test
test
{code}
> Sqlline is not showing Hive database
>
>
> Key: DRILL-5440
> URL: https://issues.apache.org/jira/browse/DRILL-5440
> Project: Apache Drill
> Issue Type: Task
> Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled,
> Hardware: VmWare
>Reporter: Parag D
[jira] [Closed] (DRILL-5411) Getting 0 rows when there are more than 100000 in the mongoDB collection
[
https://issues.apache.org/jira/browse/DRILL-5411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucas Alvarez Argüero closed DRILL-5411.
Resolution: Not A Problem
> Getting 0 rows when there are more than 10 in the mongoDB collection
>
>
> Key: DRILL-5411
> URL: https://issues.apache.org/jira/browse/DRILL-5411
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - MongoDB
>Affects Versions: 1.10.0
> Environment: VM1("ubuntu/trusty64"): mongo1
> • mongoS (mongo server)
> • MongoD shard1 (Primary, secondary,secondary)
> • Mongo config server
> • Drillbit
> VM2("ubuntu/trusty64"): mongo2
> • MongoD shard2 (Primary, secondary,secondary)
> • Mongo config server
> • Drillbit
> VM3("ubuntu/trusty64"): mongo3
> • MongoD shard3 (Primary, secondary,secondary)
> • Mongo config server
> • Drillbit
> VM4("ubuntu/trusty64"): zk1
> • Zookeeper in quorum
> VM5("ubuntu/trusty64"): zk2
> • Zookeeper in quorum
> VM6("ubuntu/trusty64"): zk3
> • Zookeeper in quorum
>Reporter: Lucas Alvarez Argüero
>
> Getting 0 rows when there are more than 10 in the mongoDB collection
> Drills works perfectly when I am using mongo as storage when there are less
> than 10(aprox) documents in the collection (partitioned) but when there
> are more documents, drill return zero rows but still can count all documents
> (but it can’t count documents using where).
> Less than 10:
> select v.measInfo_id,v.endTime from mongo.mandarinaTime3.MeasValue vlimit
> 3;
> +--+-+
> | measInfo_id | endTime |
> +--+-+
> | [B@1a7d4b45 | 2016-09-19 |
> | [B@17d8ac99 | 2016-09-19 |
> | [B@122b7d0a | 2016-09-19 |
> +--+-+
> 3 rows selected (0.313 seconds)
> More than 10:
> 0: jdbc:drill:> select v.measInfo_id,v.endTime from
> mongo.mandarinaTime3.MeasValue vlimit 3;
>
> +--+--+
> | measInfo_id | endTime |
> +--+--+
> +--+--+
> No rows selected (0.341 seconds)
> 0: jdbc:drill:> select count() from mongo.mandarinaTime3.MeasValue v;
>
> +-+
> | EXPR$0 |
> +-+
> | 502068 |
> +-+
> 1 row selected (0.426 seconds)
> 0: jdbc:drill:> select count() from mongo.mandarinaTime3.MeasValue vWhere
> endtime='2016-09-19';
> +-+
> | EXPR$0 |
> +-+
> | 0 |
> +-+
> 1 row selected (0.98 seconds)
> If the collection isn’t partitioned, drill also works perfectly
> drill mongo plugin:
> {
> "type": "mongo",
> "connection": "mongodb://mongo1:27017/",
> "enabled": true
> }
> mongo sharded collection:
> { "_id" : "mandarinaTime3", "primary" : "b", "partitioned" : true }
> mandarinaTime3.MeasCollecFile
> shard key: { "_id" : 1 }
> unique: false
> balancing: true
> chunks:
> b 1
> { "_id" : { "$minKey" : 1 } } -->> { "_id" : {
> "$maxKey" : 1 } } on : b Timestamp(1, 0)
> mandarinaTime3.MeasInfo
> shard key: { "_id" : 1 }
> unique: false
> balancing: true
> chunks:
> a 1
> b 1
> c 1
> { "_id" : { "$minKey" : 1 } } -->> { "_id" :
> ObjectId("58e364dddc7a033f5c08c7c6") } on : a Timestamp(2, 0)
> { "_id" : ObjectId("58e364dddc7a033f5c08c7c6") } -->>
> { "_id" : ObjectId("58e364e0dc7a033f5c08c8b0") } on : c Timestamp(3, 0)
> { "_id" : ObjectId("58e364e0dc7a033f5c08c8b0") } -->>
> { "_id" : { "$maxKey" : 1 } } on : b Timestamp(3, 1)
> mandarinaTime3.MeasValue
> shard key: { "_id" : 1 }
> unique: false
> balancing: true
> chunks:
> a 7
> b 7
> c 7
> too many chunks to print, use verbose if you want to
> force print
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5318) Create a sub-operator test framework
[
https://issues.apache.org/jira/browse/DRILL-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976961#comment-15976961
]
ASF GitHub Bot commented on DRILL-5318:
---
Github user sudheeshkatkam commented on the issue:
https://github.com/apache/drill/pull/788
I am confused by your comment. Which should be committed first, 5318 or
5323? Your comments says 5318, but the commit message for 5318 and the fact
that 5318 is based on 5323 suggests 5323 should be committed first?
Please format the commit message in 70fdc88 so it starts with "DRILL-:
..."
> Create a sub-operator test framework
>
>
> Key: DRILL-5318
> URL: https://issues.apache.org/jira/browse/DRILL-5318
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.11.0
>
> Attachments: Sub-OperatorTestFramework.pdf
>
>
> Drill provides two unit test frameworks for whole-server, SQL-based testing:
> the original {{BaseTestQuery}} and the newer {{ClusterFixture}}. Both use the
> {{TestBuilder}} mechanism to build system-level functional tests that run
> queries and check results.
> Jason provided an operator-level test framework based, in part on mocks:
> As Drill operators become more complex, we have a crying need for true
> unit-level tests at a level below the whole system and below operators. That
> is, we need to test the individual pieces that, together, form the operator.
> This umbrella ticket includes a number of tasks needed to create the
> sub-operator framework. Our intention is that, over time, as we find the need
> to revisit existing operators, or create new ones, we can employ the
> sub-operator test framework to exercise code at a finer granularity than is
> possible prior to this framework.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5079) PreparedStatement dynamic parameters to avoid SQL Injection test
[ https://issues.apache.org/jira/browse/DRILL-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976832#comment-15976832 ] Casey Morton commented on DRILL-5079: - +1 This is a HUGE security risk > PreparedStatement dynamic parameters to avoid SQL Injection test > > > Key: DRILL-5079 > URL: https://issues.apache.org/jira/browse/DRILL-5079 > Project: Apache Drill > Issue Type: Improvement > Components: Client - JDBC >Affects Versions: 1.8.0 >Reporter: Wahyu Sudrajat >Priority: Critical > Labels: security > > Capability to use PreparedStatement with dynamic parameters to prevent SQL > Injection. > For example: > select * from PEOPLE where FIRST_NAME = ? and LAST_NAME = ? limit 100 > As for now, Drill will return: > Error Message:PreparedStatementCallback; uncategorized SQLException for SQL > []; SQL state [null]; error code [0]; Failed to create prepared statement: > PLAN ERROR: Cannot convert RexNode to equivalent Drill expression. RexNode > Class: org.apache.calcite.rex.RexDynamicParam, RexNode Digest: ?0 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-4139) Exception while trying to prune partition. java.lang.UnsupportedOperationException: Unsupported type: BIT & Interval
[
https://issues.apache.org/jira/browse/DRILL-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976767#comment-15976767
]
Volodymyr Vysotskyi commented on DRILL-4139:
FIXED_LEN_BYTE_ARRAY values are serializied in metadata incorrectly. This issue
is similar to DRILL-3937, but there was a problem with BINARY type.
FIXED_LEN_BYTE_ARRAY parquet type corresponds to VARBINARY, DECIMAL and
INTERVAL drill types. BINARY parquet type corresponds to VARBINARY, VARCHAR and
DECIMAL drill types. As these parquet types have the common drill equivalents,
they should be serialized in the same way. [Current
solution|https://github.com/apache/drill/blob/4d4e0c2b23caead69dd4c6c02c07a9800b3c7611/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java#L1714]
would not work for FIXED_LEN_BYTE_ARRAY type because INTERVAL logical type has
little endian byte order.
> Exception while trying to prune partition.
> java.lang.UnsupportedOperationException: Unsupported type: BIT & Interval
>
>
> Key: DRILL-4139
> URL: https://issues.apache.org/jira/browse/DRILL-4139
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.3.0
> Environment: 4 node cluster on CentOS
>Reporter: Khurram Faraaz
>Assignee: Volodymyr Vysotskyi
>
> Exception while trying to prune partition.
> java.lang.UnsupportedOperationException: Unsupported type: BIT
> is seen in drillbit.log after Functional run on 4 node cluster.
> Drill 1.3.0 sys.version => d61bb83a8
> {code}
> 2015-11-27 03:12:19,809 [29a835ec-3c02-0fb6-d3c1-bae276ef7385:foreman] INFO
> o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
> class: org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2
> 2015-11-27 03:12:19,809 [29a835ec-3c02-0fb6-d3c1-bae276ef7385:foreman] INFO
> o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
> filter tree: 0 ms
> 2015-11-27 03:12:19,810 [29a835ec-3c02-0fb6-d3c1-bae276ef7385:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.UnsupportedOperationException: Unsupported type: BIT
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:479)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:235)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r8.jar:1.4.0-drill-r8]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
> [calcite-core-1.4.0-drill-r8.jar:1.4.0-drill-r8]
> at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
> [calcite-core-1.4.0-drill-r8.jar:1.4.0-drill-r8]
> at
> org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
> [calcite-core-1.4.0-drill-r8.jar:1.4.0-drill-r8]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:164)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:184)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
[jira] [Commented] (DRILL-5318) Create a sub-operator test framework
[
https://issues.apache.org/jira/browse/DRILL-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976249#comment-15976249
]
ASF GitHub Bot commented on DRILL-5318:
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/788
Rebased onto DRILL-5318 so that this branch includes the DRILL-5318 code
that this PR depends upon, as well as the additional code required for this PR.
Fixed the merge errors that caused the compilation issue.
Please commit DRILL-5318 first. Then, commit this PR. Given that this PR is
already rebased on 5318, this PR should merge cleanly into master.
> Create a sub-operator test framework
>
>
> Key: DRILL-5318
> URL: https://issues.apache.org/jira/browse/DRILL-5318
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.11.0
>
> Attachments: Sub-OperatorTestFramework.pdf
>
>
> Drill provides two unit test frameworks for whole-server, SQL-based testing:
> the original {{BaseTestQuery}} and the newer {{ClusterFixture}}. Both use the
> {{TestBuilder}} mechanism to build system-level functional tests that run
> queries and check results.
> Jason provided an operator-level test framework based, in part on mocks:
> As Drill operators become more complex, we have a crying need for true
> unit-level tests at a level below the whole system and below operators. That
> is, we need to test the individual pieces that, together, form the operator.
> This umbrella ticket includes a number of tasks needed to create the
> sub-operator framework. Our intention is that, over time, as we find the need
> to revisit existing operators, or create new ones, we can employ the
> sub-operator test framework to exercise code at a finer granularity than is
> possible prior to this framework.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5323) Provide test tools to create, populate and compare row sets
[
https://issues.apache.org/jira/browse/DRILL-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976245#comment-15976245
]
ASF GitHub Bot commented on DRILL-5323:
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/785
Rebased onto latest master and squashed.
> Provide test tools to create, populate and compare row sets
> ---
>
> Key: DRILL-5323
> URL: https://issues.apache.org/jira/browse/DRILL-5323
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> Operators work with individual row sets. A row set is a collection of records
> stored as column vectors. (Drill uses various terms for this concept. A
> record batch is a row set with an operator implementation wrapped around it.
> A vector container is a row set, but with much functionality left as an
> exercise for the developer. And so on.)
> To simplify tests, we need a {{TestRowSet}} concept that wraps a
> {{VectorContainer}} and provides easy ways to:
> * Define a schema for the row set.
> * Create a set of vectors that implement the schema.
> * Populate the row set with test data via code.
> * Add an SV2 to the row set.
> * Pass the row set to operator components (such as generated code blocks.)
> * Compare the results of the operation with an expected result set.
> * Dispose of the underling direct memory when work is done.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5323) Provide test tools to create, populate and compare row sets
[
https://issues.apache.org/jira/browse/DRILL-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976201#comment-15976201
]
ASF GitHub Bot commented on DRILL-5323:
---
Github user sohami commented on the issue:
https://github.com/apache/drill/pull/785
Thanks for the changes, LGTM.
+1
> Provide test tools to create, populate and compare row sets
> ---
>
> Key: DRILL-5323
> URL: https://issues.apache.org/jira/browse/DRILL-5323
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> Operators work with individual row sets. A row set is a collection of records
> stored as column vectors. (Drill uses various terms for this concept. A
> record batch is a row set with an operator implementation wrapped around it.
> A vector container is a row set, but with much functionality left as an
> exercise for the developer. And so on.)
> To simplify tests, we need a {{TestRowSet}} concept that wraps a
> {{VectorContainer}} and provides easy ways to:
> * Define a schema for the row set.
> * Create a set of vectors that implement the schema.
> * Populate the row set with test data via code.
> * Add an SV2 to the row set.
> * Pass the row set to operator components (such as generated code blocks.)
> * Compare the results of the operation with an expected result set.
> * Dispose of the underling direct memory when work is done.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
