reorg rn, 1.0 > 1.1 minor edit
add unix_timestamp enhancement from Jason Project: http://git-wip-us.apache.org/repos/asf/drill/repo Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/219fb0bc Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/219fb0bc Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/219fb0bc Branch: refs/heads/gh-pages Commit: 219fb0bc71c736bb0193bfbd32fb9a5aee080bf4 Parents: 3009b06 Author: Kristine Hahn <[email protected]> Authored: Mon Jul 6 11:08:24 2015 -0700 Committer: Kristine Hahn <[email protected]> Committed: Mon Jul 6 11:24:00 2015 -0700 ---------------------------------------------------------------------- .../060-text-files-csv-tsv-psv.md | 48 +- .../047-installing-drill-on-the-cluster.md | 6 +- .../050-starting-drill-in-distributed-mode.md | 2 +- ...20-installing-drill-on-linux-and-mac-os-x.md | 10 +- .../030-starting-drill-on-linux-and-mac-os-x.md | 2 +- .../040-installing-drill-on-windows.md | 6 +- .../050-starting-drill-on-windows.md | 4 +- _docs/log-and-debug/005-query-audit-logging.md | 6 +- _docs/query-data/010-query-data-introduction.md | 4 +- .../020-querying-parquet-files.md | 2 +- _docs/rn/010-0.5.0rn.md | 29 -- _docs/rn/010-1.1.0-rn.md | 389 +++++++++++++++ _docs/rn/020-0.4.0rn.md | 42 -- _docs/rn/020-1.0.0-rn.md | 493 +++++++++++++++++++ _docs/rn/030-0.9.0-rn.md | 29 ++ _docs/rn/030-alpha-rn.md | 39 -- _docs/rn/030-m1-alpha-rn.md | 39 -- _docs/rn/040-0.6.0-rn.md | 32 -- _docs/rn/050-0.7.0-rn.md | 56 --- _docs/rn/050-0.8.0-rn.md | 28 ++ _docs/rn/060-0.7.0-rn.md | 56 +++ _docs/rn/060-0.8.0-rn.md | 28 -- _docs/rn/070-0.6.0-rn.md | 32 ++ _docs/rn/070-0.9.0-rn copy.md | 29 -- _docs/rn/070-0.9.0-rn.md | 29 -- _docs/rn/080-1.0.0-rn.md | 493 ------------------- _docs/rn/080-m1-alpha-rn.md | 39 ++ _docs/rn/090-alpha-rn.md | 39 ++ _docs/rn/100-0.5.0rn.md | 29 ++ _docs/rn/110-0.4.0rn.md | 42 ++ .../030-date-time-functions-and-arithmetic.md | 11 +- .../010-sql-window-functions-introduction.md | 8 +- 32 files changed, 1233 insertions(+), 868 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md ---------------------------------------------------------------------- diff --git a/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md b/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md index 268d037..b3dce16 100644 --- a/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md +++ b/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md @@ -2,10 +2,34 @@ title: "Text Files: CSV, TSV, PSV" parent: "Data Sources and File Formats" --- -The section ["Plugin Configuration Basics"]({{site.baseurl}}/docs/plugin-configuration-basics) covers attributes that you configure for use with a CSV, TSV, PSV (comma-, tab-, pipe-separated) values text file. This section presents examples of how to use those attributes and [tips for performant querying]({{site.baseurl}}/docs/text-files-csv-tsv-psv/#tips-for-performant-querying) of these text files. -## Managing Headers in Text Files -In the storage plugin configuration, you set attributes on the text reader format configuration. This section presents examples of using the following attributes defined in ["List of Attributes and Definitions"]({{site.baseurl}}/docs/plugin-configuration-basics/#list-of-attributes-and-definitions): +Best practices for reading text files are: + +* Select data from particular columns +* CAST data +* Use a distributed file system + +### Select Data from Particular Columns + +Converting text files to another format, such as Parquet, using the CTAS command and a SELECT * statement is not recommended. Instead, select data from particular columns using the [COLUMN[n] syntax]({{site.baseurl}}/docs/querying-plain-text-files), and then assign meaningful column +names using aliases. For example: + + CREATE TABLE parquet_users AS SELECT CAST(COLUMNS[0] AS INT) AS user_id, + COLUMNS[1] AS username, CAST(COLUMNS[2] AS TIMESTAMP) AS registration_date + FROM `users.csv1`; + +You need to select particular columns instead of using SELECT * for performance reasons. Drill reads CSV, TSV, and PSV files into a list of +VARCHARS, rather than individual columns. While parquet supports and Drill reads lists, as of this release of Drill, the read path for complex data is not optimized. + +### Cast data + +You can also improve performance by casting the VARCHAR data to INT, FLOAT, DATETIME, and so on when you read the data from a text file. Drill performs better reading fixed-width than reading VARCHAR data. + +### Use a Distributed File System +Using a distributed file system, such as HDFS, instead of a local file system to query the files also improves performance because currently Drill does not split files on block splits. + +## Configuring Drill to Read Text Files +In the storage plugin configuration, you can set the following attributes that affect how Drill reads CSV, TSV, PSV (comma-, tab-, pipe-separated) files. ["List of Attributes and Definitions"]({{site.baseurl}}/docs/plugin-configuration-basics/#list-of-attributes-and-definitions): * String lineDelimiter = "\n"; One or more characters used to denote a new record. Allows reading files with windows line endings. @@ -18,7 +42,9 @@ In the storage plugin configuration, you set attributes on the text reader forma * char comment = '#'; A single character used to denote a comment line. * boolean skipFirstLine = false; - Set to true to avoid reading headers as data. + Set to true to avoid reading headers as data. + +For more information about storage plugin configuration, see ["List of Attributes and Definitions"]({{site.baseurl}}/docs/plugin-configuration-basics/#list-of-attributes-and-definitions). You can deal with a mix of text files with and without headers either by creating two separate format plugins or by creating two format plugins within the same storage plugin. The former approach is typically easier than the latter. @@ -73,17 +99,3 @@ For example: "delimiter": "," }, -## Tips for Performant Querying - -Converting these files to another format, such as Parquet, using the CTAS command and a SELECT * statement is not recommended. Drill reads CSV, TSV, and PSV files into a list of -VARCHARS, rather than individual columns. While parquet supports lists and -Drill reads them, the read path for complex data is not yet optimized. Select data from particular columns using the [COLUMN[n] syntax]({{site.baseurl}}/docs/querying-plain-text-files), and then assign meaningful column -names using aliases. For example: - -CREATE TABLE parquet_users AS SELECT CAST(COLUMNS[0] AS INT) AS user_id, -COLUMNS[1] AS username, CAST(COLUMNS[2] AS TIMESTAMP) AS registration_date -FROM `users.csv1`; - -Cast the VARCHAR data to INT, FLOAT, DATETIME, and so on. You get better performance reading fixed-width than reading VARCHAR data. - -Using a distributed file system, such as HDFS, instead of a local file system to query the files also improves performance because currently Drill does not split files on block splits. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/047-installing-drill-on-the-cluster.md ---------------------------------------------------------------------- diff --git a/_docs/install/047-installing-drill-on-the-cluster.md b/_docs/install/047-installing-drill-on-the-cluster.md index 1045878..b4f4abd 100644 --- a/_docs/install/047-installing-drill-on-the-cluster.md +++ b/_docs/install/047-installing-drill-on-the-cluster.md @@ -4,12 +4,12 @@ parent: "Installing Drill in Distributed Mode" --- Complete the following steps to install Drill on designated nodes: - 1. Download the Drill tarball. + 1. Download the Drill tarball. For example: - curl http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz + curl http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz 2. Explode the tarball to the directory of your choice, such as `/opt`: - tar -xzvf apache-drill-1.0.0.tar.gz + tar -xzvf apache-drill-1.1.0.tar.gz 3. In `drill-override.conf,` create a unique Drill `cluster ID`, and provide Zookeeper host names and port numbers to configure a connection to your Zookeeper quorum. 1. Edit `drill-override.conf` located in the `/conf` directory. 2. Provide a unique `cluster-id` and the Zookeeper host names and port numbers in `zk.connect`. If you install Drill on multiple nodes, assign the same `cluster ID` to each Drill node so that all Drill nodes share the same ID. The default Zookeeper port is 2181. http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/050-starting-drill-in-distributed-mode.md ---------------------------------------------------------------------- diff --git a/_docs/install/050-starting-drill-in-distributed-mode.md b/_docs/install/050-starting-drill-in-distributed-mode.md index 9ef5d0b..0d1f25c 100644 --- a/_docs/install/050-starting-drill-in-distributed-mode.md +++ b/_docs/install/050-starting-drill-in-distributed-mode.md @@ -21,7 +21,7 @@ For example, to restart a Drillbit, navigate to the Drill installation directory Using the Drill shell, you can connect to relational databases and execute SQL commands. To start the Drill shell, run one of the following scripts, which are located in the bin directory of the Drill installation: * `drill-conf` - Opens the shell using the connection string to ZooKeeper nodes specified in `drill-override.conf` in `/opt/mapr/drill/drill-1.0.0/conf`. + Opens the shell using the connection string to ZooKeeper nodes specified in `drill-override.conf` in `/opt/mapr/drill/drill-1.1.0/conf`. * `drill-localhost` Opens the Drill shell using a connection to the ZooKeeper running on the local host. http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md index 065bee2..bb3b349 100755 --- a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md +++ b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md @@ -8,16 +8,16 @@ Complete the following steps to install Drill: 1. In a terminal windows, change to the directory where you want to install Drill. -2. To download the latest version of Apache Drill, download Drill from the [Drill web site](http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz) or run one of the following commands, depending on which you have installed on your system: - * `wget http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz` - * `curl -o apache-drill-1.0.0.tar.gz http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz` +2. To download the latest version of Apache Drill, download Drill from the [Drill web site](http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz) or run one of the following commands, depending on which you have installed on your system: + * `wget http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz` + * `curl -o apache-drill-1.1.0.tar.gz http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz` 3. Copy the downloaded file to the directory where you want to install Drill. 4. Extract the contents of the Drill tar.gz file. Use sudo only if necessary: - tar -xvzf apache-drill-1.0.0.tar.gz + tar -xvzf apache-drill-1.1.0.tar.gz -The extraction process creates the installation directory named apache-drill-1.0.0 containing the Drill software. +The extraction process creates the installation directory named apache-drill-1.1.0 containing the Drill software. At this point, you can [start Drill]({{site.baseurl}}/docs/starting-drill-on-linux-and-mac-os-x). \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/installing-drill-in-embedded-mode/030-starting-drill-on-linux-and-mac-os-x.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/030-starting-drill-on-linux-and-mac-os-x.md b/_docs/install/installing-drill-in-embedded-mode/030-starting-drill-on-linux-and-mac-os-x.md index 961d95f..594773e 100644 --- a/_docs/install/installing-drill-in-embedded-mode/030-starting-drill-on-linux-and-mac-os-x.md +++ b/_docs/install/installing-drill-in-embedded-mode/030-starting-drill-on-linux-and-mac-os-x.md @@ -6,7 +6,7 @@ Start the Drill shell using the `drill-embedded` command. The command uses a jdb 1. Navigate to the Drill installation directory. For example: - cd apache-drill-1.0.0 + cd apache-drill-1.1.0 2. Issue the following command to start the Drill shell: http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md b/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md index 2440667..984594f 100755 --- a/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md +++ b/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md @@ -4,9 +4,9 @@ parent: "Installing Drill in Embedded Mode" --- First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mode-prerequisites), including setting the JAVA_HOME environment variable, and then install Drill. Complete the following steps to install Drill: -1. Click the following link to download the latest version of Apache Drill: [http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz](http://getdrill.org/drill/download/apache-drill-1.0.0.tar.gz) -2. Move the `apache-drill-1.0.0.tar.gz` file to a directory where you want to install Drill. -3. Unzip the `TAR.GZ` file using a third-party tool. If the tool you use does not unzip the TAR file as well as the `TAR.GZ` file, unzip the `apache-drill-1.0.0.tar` to extract the Drill software. The extraction process creates the installation directory named apache-drill-1.0.0 containing the Drill software. For example: +1. Download the latest version of Apache Drill: [http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz](http://getdrill.org/drill/download/apache-drill-1.1.0.tar.gz) +2. Move the `apache-drill-1.1.0.tar.gz` file to a directory where you want to install Drill. +3. Unzip the `TAR.GZ` file using a third-party tool. If the tool you use does not unzip the TAR file as well as the `TAR.GZ` file, unzip the `apache-drill-1.1.0.tar` to extract the Drill software. The extraction process creates the installation directory named apache-drill-1.1.0 containing the Drill software. For example:  At this point, you can [start Drill]({{site.baseurl}}/docs/starting-drill-on-windows). \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/install/installing-drill-in-embedded-mode/050-starting-drill-on-windows.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/050-starting-drill-on-windows.md b/_docs/install/installing-drill-in-embedded-mode/050-starting-drill-on-windows.md index 64579a9..c53c591 100644 --- a/_docs/install/installing-drill-in-embedded-mode/050-starting-drill-on-windows.md +++ b/_docs/install/installing-drill-in-embedded-mode/050-starting-drill-on-windows.md @@ -5,8 +5,8 @@ parent: "Installing Drill in Embedded Mode" Start the Drill shell using the **sqlline command**. The `zk=local` means the local node is the ZooKeeper node. Complete the following steps to launch the Drill shell: 1. Open Command Prompt. -2. Open the apache-drill-1.0.0 folder. For example: - ``cd apache-drill-1.0.0`` +2. Open the apache-drill-1.1.0 folder. For example: + ``cd apache-drill-1.1.0`` 3. Go to the bin directory. For example: ``cd bin`` 4. Type the following command on the command line: http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/log-and-debug/005-query-audit-logging.md ---------------------------------------------------------------------- diff --git a/_docs/log-and-debug/005-query-audit-logging.md b/_docs/log-and-debug/005-query-audit-logging.md index 7829352..884d52d 100644 --- a/_docs/log-and-debug/005-query-audit-logging.md +++ b/_docs/log-and-debug/005-query-audit-logging.md @@ -11,7 +11,7 @@ The query log provides audit log functionality for the queries executed by vario For example, to check the most recent queries, query the log using this command: - SELECT * FROM dfs.`default`.`/Users/drill-user/apache-drill-1.0.0/log/sqlline_queries.json` t ORDER BY `start` LIMIT 5; + SELECT * FROM dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json` t ORDER BY `start` LIMIT 5; +----------------+------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+----------------+------------+ | finish | outcome | queryId | queryText | schema | start | username | @@ -30,7 +30,7 @@ For example, to check the most recent queries, query the log using this command: To check the total number of queries executed since the session started on the Drillbit, use the following command: - SELECT COUNT(*) FROM dfs.`default`.`/Users/drill-user/apache-drill-1.0.0/log/sqlline_queries.json`; + SELECT COUNT(*) FROM dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json`; +---------+ | EXPR$0 | @@ -43,7 +43,7 @@ To check the total number of queries executed since the session started on the D To get the total number of successful and failed executions, run the following command: - SELECT outcome, COUNT(*) FROM dfs.`default`.`/Users/drill-user/apache-drill-1.0.0/log/sqlline_queries.json` GROUP BY outcome; + SELECT outcome, COUNT(*) FROM dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json` GROUP BY outcome; +------------+---------+ | outcome | EXPR$1 | http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/query-data/010-query-data-introduction.md ---------------------------------------------------------------------- diff --git a/_docs/query-data/010-query-data-introduction.md b/_docs/query-data/010-query-data-introduction.md index 1d5ae5d..310a7b4 100644 --- a/_docs/query-data/010-query-data-introduction.md +++ b/_docs/query-data/010-query-data-introduction.md @@ -9,7 +9,7 @@ The query specifies the data source location and include data casting. ## Specifying the Data Source Location The optional [USE statement]({{site.baseurl}}/docs/use) runs subsequent queries against a particular [storage plugin]({{site.baseurl}}/docs/connect-a-data-source-introduction/). The USE statement typically saves typing some of the storage plugin information in the FROM statement. If you omit the USE statement, specify a storage plugin, such as dfs, and optionally a workspace, such as default, and a path to the data source using dot notation and back ticks. For example: -``dfs.`default`.`/Users/drill-user/apache-drill-1.0.0/log/sqlline_queries.json```; +``dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json```; ## Casting Data In some cases, Drill converts schema-less data to correctly-typed data implicitly. In this case, you do not need to [cast the data]({{site.baseurl}}/docs/supported-data-types/#casting-and-converting-data-types) to another type. The file format of the data and the nature of your query determines the requirement for casting or converting. Differences in casting depend on the data source. @@ -38,5 +38,5 @@ Remember the following tips when querying data with Drill: * Include a semicolon at the end of SQL statements, except when you issue a [Drill shell command]({{site.baseurl}}/docs/configuring-the-drill-shell/). `Example: `!set maxwidth 10000` * Use backticks around [keywords]({{site.baseurl}}/docs/reserved-keywords), special characters, and [identifiers]({{site.baseurl}}/docs/lexical-structure/#identifier) that SQL cannot parse, such as the keyword default and a path that contains a forward slash character: - Example: ``SELECT * FROM dfs.`default`.`/Users/drilluser/apache-drill-1.1.0-SNAPSHOT/sample-data/nation.parquet`;`` + Example: ``SELECT * FROM dfs.`default`.`/Users/drilluser/apache-drill-1.1.0/sample-data/nation.parquet`;`` * When selecting all (SELECT *) schema-less data, the order of returned columns might differ from the stored order and might vary from query to query. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/query-data/query-a-file-system/020-querying-parquet-files.md ---------------------------------------------------------------------- diff --git a/_docs/query-data/query-a-file-system/020-querying-parquet-files.md b/_docs/query-data/query-a-file-system/020-querying-parquet-files.md index 5823f0e..c8ecb65 100644 --- a/_docs/query-data/query-a-file-system/020-querying-parquet-files.md +++ b/_docs/query-data/query-a-file-system/020-querying-parquet-files.md @@ -38,7 +38,7 @@ systems. To view the data in the `nation.parquet` file, issue the query appropriate for your operating system: - SELECT * FROM dfs.`<path-to-installation>/apache-drill-<version>/apache-drill-1.0.0/sample-data/nation.parquet`; + SELECT * FROM dfs.`<path-to-installation>/sample-data/nation.parquet`; The query returns the following results: http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/rn/010-0.5.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/010-0.5.0rn.md b/_docs/rn/010-0.5.0rn.md deleted file mode 100644 index 2dff978..0000000 --- a/_docs/rn/010-0.5.0rn.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -title: "Apache Drill 0.5.0 Release Notes" -parent: "Release Notes" ---- - -Apache Drill 0.5.0, the first beta release for Drill, is designed to help -enthusiasts start working and experimenting with Drill. It also continues the -Drill monthly release cycle as we drive towards general availability. - -The 0.5.0 release is primarily a bug fix release, with [more than 100 JIRAs](h -ttps://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&versi -on=12324880) closed, but there are some notable features. For information -about the features, see the [Apache Drill Blog for the 0.5.0 -release](https://blogs.apache.org/drill/entry/apache_drill_beta_release_see). - -This release is available as [binary](http://www.apache.org/dyn/closer.cgi/inc -ubator/drill/drill-0.5.0-incubating/apache-drill-0.5.0-incubating.tar.gz) and -[source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.5.0-incu -bating/apache-drill-0.5.0-incubating-src.tar.gz) tarballs that are compiled -against Apache Hadoop. Drill has been tested against MapR, Cloudera, and -Hortonworks Hadoop distributions. There are associated build profiles and -JIRAs that can help you run Drill against your preferred distribution. - -Apache Drill 0.5.0 Key Notes and Limitations - - * The current release supports in memory and beyond memory execution. However, you must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. - * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior, such as Sort. Others operations, such as streaming aggregate, may have partial support that leads to unexpected results. - * There are known issues with joining text files without using an intervening view. See [DRILL-1401](https://issues.apache.org/jira/browse/DRILL-1401) for more information. - http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/rn/010-1.1.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/010-1.1.0-rn.md b/_docs/rn/010-1.1.0-rn.md new file mode 100755 index 0000000..0e373c5 --- /dev/null +++ b/_docs/rn/010-1.1.0-rn.md @@ -0,0 +1,389 @@ +--- +title: "Apache Drill 1.1.0 Release Notes" +parent: "Release Notes" +--- + +It has been about 6 weeks since the release of Drill 1.0.0. Today we're happy to announce the availability of Drill 1.1.0, providing 119 additional enhancements and bug fixes. + +## Noteworthy New Features in Drill 1.1.0 + +Drill now supports window functions, automatic partitioning, and Hive impersonation. + +### [Ranking Window Functions]({{site.baseurl}}/docs/ranking-window-functions/) + +* ROW_NUMBER +* RANK +* DENSE_RANK +* PERCENT_RANK +* CUME _DIST + +### [Aggregate Window Functions]({{site.baseurl}}/docs/aggregate-window-functions/) + +* COUNT +* SUM +* MIN +* MAX +* AVG + +### [Automatic Partitioning]({{site.baseurl}}/docs/partition-pruning/#automatic-partitioning) in CTAS (DRILL-3333) + +When a table is created with a partition by clause, the parquet writer will create separate files for the different partition values. The data will first be sorted by the partition keys, and the parquet writer will create a new file when it encounters a new value for the partition columns. + +When queries are issued against data that was created this way, partition pruning will work if the filter contains a partition column. Unlike directory-based partitioning, no view is required, nor is it necessary to reference the dir* column names. + +### [Hive impersonation]({{site.baseurl}}/docs/configuring-user-impersonation-with-hive-authorization) support (DRILL-3203) + +When impersonation is enabled, Drill now supports impersonating the user who issued the query when accessing Hive metadata/data (instead of accessing Hive as the user that started the drillbit). + +## Enhancements and Bug Fixes + +<h2>Sub-task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3203'>DRILL-3203</a>] - Add support for impersonation in Hive storage plugin +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3277'>DRILL-3277</a>] - SUM(CAST(columns[0] AS INT)) OVER(...) gives wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3278'>DRILL-3278</a>] - SUM(CAST(columns[0] AS BIGINT)) OVER(...) gives wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3281'>DRILL-3281</a>] - window functions that involve TIME columns generate wrong results +</li> +</ul> +<h2>Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-669'>DRILL-669</a>] - Information Schema should be schema sensitive +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1315'>DRILL-1315</a>] - Allow specifying Zookeeper root znode and cluster-id as JDBC parameters +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1673'>DRILL-1673</a>] - Flatten function can not work well with nested arrays. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1820'>DRILL-1820</a>] - Fix broken SCM and project links caused by graduation of Drill to a Apache TLP +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2023'>DRILL-2023</a>] - Hive function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2137'>DRILL-2137</a>] - ResultsSetMetaData.getColumnName() returns "none" (rather than right class name) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2167'>DRILL-2167</a>] - Order by on a repeated index from the output of a flatten on large no of records results in incorrect results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2310'>DRILL-2310</a>] - Drill fails to start in embedded mode on windows +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2346'>DRILL-2346</a>] - Star is not expanded correctly in create view if view fields are specified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2403'>DRILL-2403</a>] - TimePrintMillis.toString() misses leading zeros in post-decimal-point part +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2416'>DRILL-2416</a>] - Zookeeper in sqlline connection string does not override the entry from drill-override.conf +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2447'>DRILL-2447</a>] - Calling getObject on a closed ResultSet object should throw a SQLException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2449'>DRILL-2449</a>] - JDBC : DatabaseMetaData.getProcedures should return an empty result set +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2450'>DRILL-2450</a>] - JDBC : DatabaseMetaData.getColumns is missing the 'COLUMN_SIZE' in the result set +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2462'>DRILL-2462</a>] - JDBC : ResultSetMetaData.isNullable returns true even when the column is a required one +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2480'>DRILL-2480</a>] - [umbrella] Identify, fix INFORMATION_SCHEMA and JDBC metadata bugs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2494'>DRILL-2494</a>] - Binding parameters to a PreparedStatement throws a SQLException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2531'>DRILL-2531</a>] - getColumns() not right/implemented for INTERVAL types +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2555'>DRILL-2555</a>] - JDBC driver throws RuntimeExceptions rather than SQLExceptions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2592'>DRILL-2592</a>] - Jdbc-all driver includes transitive dependencies +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2595'>DRILL-2595</a>] - Sqlline Usage needs to be corrected +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2622'>DRILL-2622</a>] - C++ Client valgrind errors in sync API +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2628'>DRILL-2628</a>] - sqlline hangs and then asserts when trying to execute anything on a dead JDBC connection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2763'>DRILL-2763</a>] - [umbrella] Implement INFORMATION_SCHEMA.COLUMNS enough for relevant tools +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2782'>DRILL-2782</a>] - Decide, implement behavior for transaction-related JDBC methods +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2866'>DRILL-2866</a>] - Incorrect error message reporting schema change when streaming aggregation and hash join are disabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2903'>DRILL-2903</a>] - Update TestDrillbitResilience tests so that they correctly manage canceled queries that get to complete too quickly. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2923'>DRILL-2923</a>] - Ensure all unit tests pass without assertions enabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2967'>DRILL-2967</a>] - Incompatible types error reported in a "not in" query with compatible data types +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2985'>DRILL-2985</a>] - REGRESSION : NPE seen for project distinct values from CSV +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2988'>DRILL-2988</a>] - Correlated exists subquery returns wrong result if join columns in subquery are not fully qualified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3004'>DRILL-3004</a>] - Failure in planning join when disabling hash join and exchanges +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3019'>DRILL-3019</a>] - Extra column in Schema of Recordbatch from scanning Values +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3028'>DRILL-3028</a>] - Exception in correlated subquery with exists when columns in subquery are not qualified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3032'>DRILL-3032</a>] - Join between complex (nested repeated lists) data results in "LATE type is not supported" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3034'>DRILL-3034</a>] - Apply UserException to port-binding error; handle UserException in embedded-Drill case +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3035'>DRILL-3035</a>] - Create ControlsInjector interface to enforce implementing methods +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3078'>DRILL-3078</a>] - Tracking bug for ODBC doc changes from Simba +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3094'>DRILL-3094</a>] - TPCH query 15 returns non-deterministic result +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3120'>DRILL-3120</a>] - Windows startup throws NPE +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3125'>DRILL-3125</a>] - Drill UI Profile page fails to load for a query in some scenarios +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3134'>DRILL-3134</a>] - Doc: "Supported ... Types" section doesn't include complex types +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3143'>DRILL-3143</a>] - MaterializedField#clone should deep copy itself without disregarding its children +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3144'>DRILL-3144</a>] - Doc.: JDBC Driver section is SQuirrel-specific, should be moved +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3147'>DRILL-3147</a>] - tpcds-sf1-parquet query 73 causes memory leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3155'>DRILL-3155</a>] - Composite vectors leak memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3159'>DRILL-3159</a>] - Make JDBC throttling threshold configurable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3172'>DRILL-3172</a>] - Can not plan exception when over clause is empty +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3173'>DRILL-3173</a>] - Invalid inputs are NOT handled properly by Window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3174'>DRILL-3174</a>] - Calcite blocks queries whose type-missmatch can be resolved by Drill's Implicit casting +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3177'>DRILL-3177</a>] - Upgrade Mongo java driver to 3.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3179'>DRILL-3179</a>] - Example output doesn't match example data (ticket_sales.json) in Complex JSON doc +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3182'>DRILL-3182</a>] - Window function with DISTINCT qualifier returns seemingly incorrect result +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3183'>DRILL-3183</a>] - Query that uses window functions returns NPE +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3188'>DRILL-3188</a>] - Restrict the types of window frames that can be specified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3190'>DRILL-3190</a>] - Invalid FragmentState transition from CANCELLATION_REQUESTED in QueryManager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3195'>DRILL-3195</a>] - Throw unsupported exception for Window functions that are not currently supported +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3196'>DRILL-3196</a>] - Disable multiple partition by clauses in the same sql query +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3197'>DRILL-3197</a>] - Query that uses window functions fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3198'>DRILL-3198</a>] - JDBC driver returns null from DatabaseMetaData.getTypeInfo(...) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3199'>DRILL-3199</a>] - GenericAccessor doesn't support isNull +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3204'>DRILL-3204</a>] - Problem in name resolution with window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3206'>DRILL-3206</a>] - Memory leak in window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3207'>DRILL-3207</a>] - Memory leak in window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3208'>DRILL-3208</a>] - Hive : Tpch (SF 0.01) query 10 fails with a system error when the data is backed by hive tables +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3210'>DRILL-3210</a>] - Star is not expanded correctly in projection list when used with window function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3211'>DRILL-3211</a>] - Assert in a query with window function and group by clause +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3214'>DRILL-3214</a>] - Config option to cast empty string to null does not cast empty string to null +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3215'>DRILL-3215</a>] - Describe table from hive storage does not connect to "default" database +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3216'>DRILL-3216</a>] - Fix existing(+) INFORMATION_SCHEMA.COLUMNS columns +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3218'>DRILL-3218</a>] - Window function usage throws CompileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3220'>DRILL-3220</a>] - IOB Exception when using constants in window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3245'>DRILL-3245</a>] - Error message needs to be fixed. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3247'>DRILL-3247</a>] - Query without casting results in CompileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3254'>DRILL-3254</a>] - Average over window functions returns wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3255'>DRILL-3255</a>] - Queries must fail when invalid-positions are specified in order by clause of a window function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3260'>DRILL-3260</a>] - Conflicting servlet-api jar causing web UI to be slow +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3262'>DRILL-3262</a>] - Rename missed .impl.DrillDatabaseMetaData to .impl.DrillDatabaseMetaDataImpl to +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3263'>DRILL-3263</a>] - Read smallint and tinyint data from hive as integer until these types are well supported throughout Drill +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3265'>DRILL-3265</a>] - Query with window function and sort below that spills to disk runs out of memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3266'>DRILL-3266</a>] - Drill's hive storage plugin cannot find RegexSerDe +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3268'>DRILL-3268</a>] - queries with empty OVER() clause return empty result set +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3269'>DRILL-3269</a>] - Window function query takes too long to complete and return results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3273'>DRILL-3273</a>] - Hive function 'concat_ws' not working from drill +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3275'>DRILL-3275</a>] - Difference in expected results - query that uses window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3285'>DRILL-3285</a>] - Split DrillCursor.next(), clean up DrillCursor for clarity +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3286'>DRILL-3286</a>] - IN clause with null in it results in AssertionError: Error while applying rule DrillValuesRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3293'>DRILL-3293</a>] - CTAS with window function fails with UnsupportedOperationException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3294'>DRILL-3294</a>] - False schema change exception in CTAS with AVG window function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3296'>DRILL-3296</a>] - Group By Union Distinct fails at planning +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3298'>DRILL-3298</a>] - Wrong result with SUM window function and order by without partition by in the OVER clause +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3305'>DRILL-3305</a>] - DrillOptiq should raise appropriate error message while dealing with unknown RexNode +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3306'>DRILL-3306</a>] - Concurrently Running hive queries results in a deadlock situation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3307'>DRILL-3307</a>] - Query with window function runs out of memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3311'>DRILL-3311</a>] - sqlline hangs when query is cancelled while results are returned from the server +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3316'>DRILL-3316</a>] - Different SQLHandlers should go through the same planning logics for the same SELECT statement. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3318'>DRILL-3318</a>] - SUM(CAST(col as INT)) shows different results when used in window functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3321'>DRILL-3321</a>] - ZK PStore configuration needed to prevent Drill Web UI problems +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3324'>DRILL-3324</a>] - CTAS broken with the new auto partition feature ( Not in master yet) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3326'>DRILL-3326</a>] - Query with unsupported windows function containing "AS" blocks correct error message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3327'>DRILL-3327</a>] - row_number function returns incorrect result when only order by clause is specified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3328'>DRILL-3328</a>] - Cannot cast hive binary to varchar +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3333'>DRILL-3333</a>] - Add support for auto-partitioning in parquet writer +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3337'>DRILL-3337</a>] - Queries with Window Function DENSE_RANK fail with SchemaChangeException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3343'>DRILL-3343</a>] - Seemingly incorrect result with SUM window functions and float data type +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3344'>DRILL-3344</a>] - When Group By clause is present, the argument in window function should not refer to any column outside Group By +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3345'>DRILL-3345</a>] - TestWindowFrame fails to properly check cases involving multiple batches +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3346'>DRILL-3346</a>] - Windowing query over View should display a better error message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3351'>DRILL-3351</a>] - Invalid query must be caught earlier +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3357'>DRILL-3357</a>] - Error when adding 2 columns together +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3358'>DRILL-3358</a>] - CUME_DIST window function provides wrong result when only ORDER BY clause is specified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3359'>DRILL-3359</a>] - Drill should throw and error when window function defined using WINDOW AS uses ROWS UNBOUNDED PRECEDING +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3361'>DRILL-3361</a>] - CTAS Auto Partitioning: Fails when we use boolean as the partition type +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3370'>DRILL-3370</a>] - FLATTEN error with a where clause +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3373'>DRILL-3373</a>] - CTAS partition by with empty list of partitioning columns should be blocked in parser +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3374'>DRILL-3374</a>] - CTAS with PARTITION BY, partition column name from view can not be resolved +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3376'>DRILL-3376</a>] - Reading individual files created by CTAS with partition causes an exception +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3377'>DRILL-3377</a>] - Can't partition by expression when columns are explicitly specified in the CTAS column list +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3378'>DRILL-3378</a>] - Average over window on a view returns wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3379'>DRILL-3379</a>] - Passing references when cloning ParquetGroupScan causes incorrect planning +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3380'>DRILL-3380</a>] - CTAS Auto Partitioning : We are not pruning when we use functions in the select list +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3398'>DRILL-3398</a>] - WebServer is leaking memory for queries submitted through REST API or WebUI +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3400'>DRILL-3400</a>] - After shifting CTAS's data, query on CTAS table failed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3404'>DRILL-3404</a>] - Filter on window function does not appear in query plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3408'>DRILL-3408</a>] - CTAS partition by columns[i] from csv fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3410'>DRILL-3410</a>] - Partition Pruning : We are doing a prune when we shouldn't +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3411'>DRILL-3411</a>] - CTAS Partition by column in deeper layer fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3413'>DRILL-3413</a>] - Use DIGEST mechanism in creating Hive MetaStoreClient for proxy users when SASL authentication is enabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3418'>DRILL-3418</a>] - Partition Pruning : We are over-pruning and this leads to wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3422'>DRILL-3422</a>] - Multiple unit test failures on Windows with current master +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3424'>DRILL-3424</a>] - Hive Views are not accessible through Drill Query +</li> +</ul> + +<h2>Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-745'>DRILL-745</a>] - Drill fails to read the schema of avro tables from hive when the schema is in a separate file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-959'>DRILL-959</a>] - drill fails to display binary in hive correctly +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1616'>DRILL-1616</a>] - Add support for count() on maps and arrays +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1862'>DRILL-1862</a>] - over clause with only order by clause throws an exception +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2086'>DRILL-2086</a>] - mapr profile - use MapR 4.0.2 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2272'>DRILL-2272</a>] - Tibco Spotfire Desktop configuration for Drill documentation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2405'>DRILL-2405</a>] - Generate test data for window function instead of downloading it from S3 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2746'>DRILL-2746</a>] - Filter is not pushed into subquery past UNION ALL +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2764'>DRILL-2764</a>] - REST API should return exception details on error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2839'>DRILL-2839</a>] - ODBC Driver Doc to point to latest available Driver, also provide compatibility matrix for Drill and ODBC version +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2997'>DRILL-2997</a>] - Remove references to groupCount from SerializedField +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3024'>DRILL-3024</a>] - UserException should have a getErrorType() method +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3025'>DRILL-3025</a>] - Tibco Spotfire Server - JDBC - Configuration Document +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3108'>DRILL-3108</a>] - Replace templated returns with covariant return overrides +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3130'>DRILL-3130</a>] - Project can be pushed below union all / union to improve performance +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3148'>DRILL-3148</a>] - JReport enablement document for Drill +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3200'>DRILL-3200</a>] - Add Window functions: ROW_NUMBER, RANK, PERCENT_RANK, DENSE_RANK and CUME_DIST +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3240'>DRILL-3240</a>] - Fetch hadoop maven profile specific Hive version in Hive storage plugin +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3304'>DRILL-3304</a>] - improve org.apache.drill.exec.expr.TypeHelper error messages when UnsupportedOprationException is thrown +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3319'>DRILL-3319</a>] - UserExceptions should be logged from the right class +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3320'>DRILL-3320</a>] - Do away with "rebuffing" Drill jar +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3421'>DRILL-3421</a>] - Add new outputformat=json +</li> +</ul> + +<h2> New Feature +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1169'>DRILL-1169</a>] - Add support for UNION (distinct type) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3246'>DRILL-3246</a>] - Query planning support for partition by clause in Drill's CTAS statement +</li> +</ul> + +<h2> Task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2420'>DRILL-2420</a>] - [umbrella] Identify, fix DatabaseMetaData.getColumns() bugs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2952'>DRILL-2952</a>] - Hive 1.0 plugin for Drill +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2983'>DRILL-2983</a>] - Bridget's User Auth Doc +</li> +</ul> + + + http://git-wip-us.apache.org/repos/asf/drill/blob/219fb0bc/_docs/rn/020-0.4.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/020-0.4.0rn.md b/_docs/rn/020-0.4.0rn.md deleted file mode 100644 index d0dac30..0000000 --- a/_docs/rn/020-0.4.0rn.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: "Apache Drill 0.4.0 Release Notes" -parent: "Release Notes" ---- -The 0.4.0 release is a developer preview release, designed to help enthusiasts -start to work with and experiment with Drill. It is the first Drill release -that provides distributed query execution. - -This release is built upon [more than 800 -JIRAs](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324963/). -It is a pre-beta release on the way towards Drill. As a developer snapshot, -the release contains a large number of outstanding bugs that will make some -use cases challenging. Feel free to consult outstanding issues [targeted for -the 0.5.0 -release](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324880/) -to see whether your use case is affected. - -To read more about this release and new features introduced, please view the -[0.4.0 announcement blog -entry](https://blogs.apache.org/drill/entry/announcing_apache_drill_0_4). - -The release is available as both [binary](http://www.apache.org/dyn/closer.cgi -/incubator/drill/drill-0.4.0-incubating/apache-drill-0.4.0-incubating.tar.gz) -and [source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.4.0- -incubating/apache-drill-0.4.0-incubating-src.tar.gz) tarballs. In both cases, -these are compiled against Apache Hadoop. Drill has also been tested against -MapR, Cloudera and Hortonworks Hadoop distributions and there are associated -build profiles or JIRAs that can help you run against your preferred -distribution. - -Some Key Notes & Limitations - - * The current release supports in memory and beyond memory execution. However, users must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. - * In many cases,merge join operations return incorrect results. - * Use of a local filter in a join âonâ clause when using left, right or full outer joins may result in incorrect results. - * Because of known memory leaks and memory overrun issues you may need more memory and you may need to restart the system in some cases. - * Some types of complex expressions, especially those involving empty arrays may fail or return incorrect results. - * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior (such as Sort). Others operations (such as streaming aggregate) may have partial support that leads to unexpected results. - * Protobuf, UDF, query plan interfaces and all interfaces are subject to change in incompatible ways. - * Multiplication of some types of DECIMAL(28+,*) will return incorrect result. - -
