[
https://issues.apache.org/jira/browse/DRILL-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781875#comment-15781875
]
ASF GitHub Bot commented on DRILL-5152:
---------------------------------------
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/708
DRILL-5152: Enhance the mock data source: better data, SQL access
Provides an enhanced version of the mock data source. See the JIRA entry
for motivation, package-info.java for details of operation.
Allows tests to write queries of the form:
```
select id_i, name_s50 from `mock`.`employee_1K` ...
```
Where id_i is a field of random, uniformly distributed integers and
name_s50 is a VARCHAR column of width 50 of randomly generated strings. The _1K
suffix says to generate 1000 rows. The names are just for convenience, the
suffixes tell the mock data source what to generate.
Examples of use will appear in a later commit that includes a revised test
framework. Existing tests that use the physical plan version of the mock data
source work as before.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paul-rogers/drill DRILL-5152
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/708.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #708
----
commit e9741ce59621209e18cf405c8fe4a614d955ed2a
Author: Paul Rogers <prog...@maprtech.com>
Date: 2016-12-22T05:47:20Z
DRILL-5152: Enhance the mock data source: better data, SQL access
Provides an enhanced version of the mock data source. See the JIRA
entry for motivation, package-info.java for details of operation.
----
> Enhance the mock data source: better data, SQL access
> -----------------------------------------------------
>
> Key: DRILL-5152
> URL: https://issues.apache.org/jira/browse/DRILL-5152
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build & Test
> Affects Versions: 1.9.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
>
> Drill provides a mock data storage engine that generates random data. The
> mock engine is used in some older unit tests that need a volume of data, but
> that are not too particular about the details of the data.
> The mock data source continues to have use even for modern tests. For
> example, the work in the external storage batch requires tests with varying
> amounts of data, but the exact form of the data is not important, just the
> quantity. For example, if we want to ensure that spilling happens at various
> trigger points, we need to read the right amount of data for that trigger.
> The existing mock data source has two limitations:
> 1. It generates only "black/white" (alternating) values, which is awkward for
> use in sorting.
> 2. The mock generator is accessible only from a physical plan, but not from
> SQL queries.
> This enhancement proposes to fix both limitations:
> 1. Generate a uniform, randomly distributed set of values.
> 2. Provide an encoding that lets a SQL query specify the data to be generated.
> Example SQL query:
> {code}
> SELECT id_i, name_s50 FROM `mock`.employee_10K;
> {code}
> The above says to generate two fields: INTEGER (the "_i" suffix) and
> VARCHAR(50) (the "_s50") suffix; and to generate 10,000 rows (the "_10K"
> suffix on the table.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
