[jira] [Assigned] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-2593: --- Assignee: Arina Ielchiieva > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: Future > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4716) status.json doesn't work in drill ui
[
https://issues.apache.org/jira/browse/DRILL-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4716:
Description:
1. http://localhost:8047/status returns "Running!"
But http://localhost:8047/status.json gives error.
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
2. Link to System Options on page http://localhost:8047/status is corrupted.
was:
1. http://localhost:8047/status returns "Running!"
But http://localhost5:8047/status.json gives error.
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
2. Link to System Options on page http://localhost:8047/status is corrupted.
> status.json doesn't work in drill ui
>
>
> Key: DRILL-4716
> URL: https://issues.apache.org/jira/browse/DRILL-4716
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.7.0
>
>
> 1. http://localhost:8047/status returns "Running!"
> But http://localhost:8047/status.json gives error.
> {code}
> {
> "errorMessage" : "HTTP 404 Not Found"
> }
> {code}
> 2. Link to System Options on page http://localhost:8047/status is corrupted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4716) status.json doesn't work in drill ui
[
https://issues.apache.org/jira/browse/DRILL-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4716:
Description:
1. http://localhost:8047/status returns "Running!"
But http://localhost:8047/status.json gives error.
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
2. Remove link to System Options on page http://localhost:8047/status as
redundant.
was:
1. http://localhost:8047/status returns "Running!"
But http://localhost:8047/status.json gives error.
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
2. Link to System Options on page http://localhost:8047/status is corrupted.
> status.json doesn't work in drill ui
>
>
> Key: DRILL-4716
> URL: https://issues.apache.org/jira/browse/DRILL-4716
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.7.0
>
>
> 1. http://localhost:8047/status returns "Running!"
> But http://localhost:8047/status.json gives error.
> {code}
> {
> "errorMessage" : "HTTP 404 Not Found"
> }
> {code}
> 2. Remove link to System Options on page http://localhost:8047/status as
> redundant.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-2593: Fix Version/s: (was: Future) 1.7.0 > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > Attachments: warning1.JPG, warning2.JPG > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4716) status.json doesn't work in drill ui
Arina Ielchiieva created DRILL-4716:
---
Summary: status.json doesn't work in drill ui
Key: DRILL-4716
URL: https://issues.apache.org/jira/browse/DRILL-4716
Project: Apache Drill
Issue Type: Bug
Components: Client - HTTP
Affects Versions: 1.6.0
Reporter: Arina Ielchiieva
Assignee: Arina Ielchiieva
Priority: Minor
Fix For: 1.7.0
1. http://localhost:8047/status returns "Running!"
But http://localhost5:8047/status.json gives error.
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
2. Link to System Options on page http://localhost:8047/status is corrupted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326489#comment-15326489 ] Arina Ielchiieva commented on DRILL-2593: - In case we have corrupted profile, it will be skipped but dismissable warning will be generated. Screenshots - warning1.JPG, warning2.JPG. > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: Future > > Attachments: warning1.JPG, warning2.JPG > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-2593: Attachment: warning2.JPG warning1.JPG > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: Future > > Attachments: warning1.JPG, warning2.JPG > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4514) Add describe schema command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331419#comment-15331419
]
Arina Ielchiieva commented on DRILL-4514:
-
h4. Implementation approach:
If schema doesn't have workspaces, all schema properties will be returned
(information shown in webui storage tab).
If schema has workspaces, all schema properties will be returned + properties
for indicated workspace ("location", "writable", "defaultInputFormat").
Example:
*1. Schema with workspaces*
When we indicate dfs without workspace, default workspace is implied. So result
for both commands below will be the same.
{code:sql}
describe schema dfs;
describe schema dfs.default;
{code}
{noformat}
{
"type" : "file",
"enabled" : true,
"connection" : "file:///",
"config" : null,
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv", "bcp" ],
"delimiter" : ","
},
... etc
},
"location" : "/",
"writable" : false,
"defaultInputFormat" : null
}
{noformat}
Results for dfs.tmp and dfs.default will slightly differ. All common properties
that belong to schema will be the same ("type", "connection", "formats" etc).
Properties "location", "writable", "defaultInputFormat" will differ since they
belong to workspace.
{code:sql}
describe schema dfs.tmp;
{code}
{noformat}
{
"type" : "file",
"enabled" : true,
"connection" : "file:///",
"config" : null,
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv", "bcp" ],
"delimiter" : ","
},
... etc
},
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : null
}
{noformat}
*2. Schema without workspaces*
Hive schema doesn't have workspaces so we return all schema properties.
{code:sql}
describe schema hive;
{code}
{noformat}
{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "",
"javax.jdo.option.ConnectionURL":
"jdbc:derby:;databaseName=../sample-data/drill_hive_db;create=true",
"hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
"fs.default.name": "file:///",
"hive.metastore.sasl.enabled": "false"
}
}
{noformat}
> Add describe schema command
> -
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {noformat}
> DESCRIBE SCHEMA xdf.proc;
> +-++
> |name | location |
> +-++
> | xdf.proc| maprfs://dl.data/processed |
> +-++
> {noformat}
> Current implementation covers only dfs schema.
> For all other "" will be returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe schema command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Description:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{code:sql}
DESCRIBE SCHEMA dfs.tmp;
{code}
{noformat}
++
| properties |
++
| {
"type" : "file",
"enabled" : true,
"connection" : "file:///",
"config" : null,
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json",
"extensions" : [ "json" ]
},
"avro" : {
"type" : "avro"
},
"sequencefile" : {
"type" : "sequencefile",
"extensions" : [ "seq" ]
},
"csvh" : {
"type" : "text",
"extensions" : [ "csvh" ],
"extractHeader" : true,
"delimiter" : ","
}
},
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : null
} |
++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
was:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE SCHEMA xdf.proc;
+-++
|name | location |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
> Add describe schema command
> -
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {code:sql}
> DESCRIBE SCHEMA dfs.tmp;
> {code}
> {noformat}
> ++
> | properties |
> ++
> | {
> "type" : "file",
> "enabled" : true,
> "connection" : "file:///",
> "config" : null,
> "formats" : {
> "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
> },
> "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
> },
> "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
> },
> "parquet" : {
> "type" : "parquet"
> },
> "json" : {
> "type" : "json",
> "extensions" : [ "json" ]
> },
> "avro" : {
> "type" : "avro"
> },
> "sequencefile" : {
> "type" : "sequencefile",
> "extensions" : [ "seq" ]
> },
> "csvh" : {
> "type" : "text",
> "extensions" : [ "csvh" ],
> "extractHeader" : true,
> "delimiter" : ","
> }
> },
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : null
> } |
> ++
> {noformat}
> Current implementation covers only dfs schema.
> For all other "" will be returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe schema command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Description:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{code:sql}
DESCRIBE SCHEMA dfs.tmp;
{code}
{noformat}
++
| properties |
++
| {
"type" : "file",
"enabled" : true,
"connection" : "file:///",
"config" : null,
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json",
"extensions" : [ "json" ]
},
"avro" : {
"type" : "avro"
},
"sequencefile" : {
"type" : "sequencefile",
"extensions" : [ "seq" ]
},
"csvh" : {
"type" : "text",
"extensions" : [ "csvh" ],
"extractHeader" : true,
"delimiter" : ","
}
},
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : null
} |
++
{noformat}
was:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{code:sql}
DESCRIBE SCHEMA dfs.tmp;
{code}
{noformat}
++
| properties |
++
| {
"type" : "file",
"enabled" : true,
"connection" : "file:///",
"config" : null,
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json",
"extensions" : [ "json" ]
},
"avro" : {
"type" : "avro"
},
"sequencefile" : {
"type" : "sequencefile",
"extensions" : [ "seq" ]
},
"csvh" : {
"type" : "text",
"extensions" : [ "csvh" ],
"extractHeader" : true,
"delimiter" : ","
}
},
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : null
} |
++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
> Add describe schema command
> -
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {code:sql}
> DESCRIBE SCHEMA dfs.tmp;
> {code}
> {noformat}
> ++
> | properties |
> ++
> | {
> "type" : "file",
> "enabled" : true,
> "connection" : "file:///",
> "config" : null,
> "formats" : {
> "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
> },
> "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
> },
> "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
> },
> "parquet" : {
> "type" : "parquet"
> },
> "json" : {
> "type" : "json",
> "extensions" : [ "json" ]
> },
> "avro" : {
> "type" : "avro"
> },
> "sequencefile" : {
> "type" : "sequencefile",
> "extensions" : [ "seq" ]
> },
> "csvh" : {
> "type" : "text",
> "extensions" : [ "csvh" ],
> "extractHeader" : true,
> "delimiter" : ","
> }
> },
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : null
> } |
> ++
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (DRILL-4309) Make this option store.hive.optimize_scan_with_native_readers=true default
[ https://issues.apache.org/jira/browse/DRILL-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-4309: --- Assignee: Arina Ielchiieva > Make this option store.hive.optimize_scan_with_native_readers=true default > -- > > Key: DRILL-4309 > URL: https://issues.apache.org/jira/browse/DRILL-4309 > Project: Apache Drill > Issue Type: Improvement > Components: Query Planning & Optimization >Reporter: Sean Hsuan-Yi Chu >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > > This new feature has been around and used/tests in many scenarios. > We should enable this feature by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4571: Attachment: drillbit_queries_json_screenshot.jpg > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Labels: doc-impacting > Fix For: 1.7.0 > > Attachments: display_log.JPG, drillbit_download.log.gz, > drillbit_queries_json_screenshot.jpg, drillbit_ui.log, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306529#comment-15306529 ] Arina Ielchiieva commented on DRILL-4571: - [~knguyen], thanks for verification. I have fixed point 1 and 3. Point 2 regarding the content of the drillbit_queries.json display in Chrome is not reproducible. Actually, before my changes drillbit_queries.json log was writing all info in file in one line. I have modified logback.xml so query log started to write and thus and display in more readable format. So if you had drillbit_queries.json with previous info, it might show you all in one line. But after my changes all info should be written in file line by line. Attaching screenshot of drillbit_queries.json display (drillbit_queries_json_screenshot.jpg). Could you please re-check? All changes are in "DRILL-4571-fix" branch in my repo. (https://github.com/arina-ielchiieva/drill/commit/7def77b99d61a1c0e4810ba2a073bbc770d2d161). > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Labels: doc-impacting > Fix For: 1.7.0 > > Attachments: display_log.JPG, drillbit_download.log.gz, > drillbit_ui.log, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4701) Fix log name and missing lines in logs on Web UI
Arina Ielchiieva created DRILL-4701: --- Summary: Fix log name and missing lines in logs on Web UI Key: DRILL-4701 URL: https://issues.apache.org/jira/browse/DRILL-4701 Project: Apache Drill Issue Type: Bug Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Fix For: 1.7.0 1. When the log files are downloaded from the ui, the name of the downloaded file is "download". We should save the file with the same name as the log file (ie. drillbit.log) 2. The last N lines of the log file displayed in the web UI do not match the log file itself. Some lines are missing compared with actual log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15309917#comment-15309917 ] Arina Ielchiieva commented on DRILL-4571: - Created Jira DRILL-4701 to address changes made to fix point 1 and 3. > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Labels: doc-impacting > Fix For: 1.7.0 > > Attachments: display_log.JPG, drillbit_download.log.gz, > drillbit_queries_json_screenshot.jpg, drillbit_ui.log, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4726) Dynamic UDFs support
[ https://issues.apache.org/jira/browse/DRILL-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4726: Description: Allow register UDFs without restart of Drillbits. Design is described in document below: https://docs.google.com/document/d/1MluM17EKajvNP_x8U4aymcOihhUm8BMm8t_hM0jEFWk/edit was: Allow register UDFs without restart of Drillbits. Approach is described in document below: https://docs.google.com/document/d/1MluM17EKajvNP_x8U4aymcOihhUm8BMm8t_hM0jEFWk/edit > Dynamic UDFs support > > > Key: DRILL-4726 > URL: https://issues.apache.org/jira/browse/DRILL-4726 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: Future > > > Allow register UDFs without restart of Drillbits. > Design is described in document below: > https://docs.google.com/document/d/1MluM17EKajvNP_x8U4aymcOihhUm8BMm8t_hM0jEFWk/edit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4726) Dynamic UDFs support
Arina Ielchiieva created DRILL-4726: --- Summary: Dynamic UDFs support Key: DRILL-4726 URL: https://issues.apache.org/jira/browse/DRILL-4726 Project: Apache Drill Issue Type: New Feature Affects Versions: 1.6.0 Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Fix For: Future Allow register UDFs without restart of Drillbits. Approach is described in document below: https://docs.google.com/document/d/1MluM17EKajvNP_x8U4aymcOihhUm8BMm8t_hM0jEFWk/edit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4733) max(dir0) reading more columns than necessary
[
https://issues.apache.org/jira/browse/DRILL-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4733:
Fix Version/s: 1.7.0
> max(dir0) reading more columns than necessary
> -
>
> Key: DRILL-4733
> URL: https://issues.apache.org/jira/browse/DRILL-4733
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization, Storage - Parquet
>Affects Versions: 1.7.0
>Reporter: Rahul Challapalli
>Assignee: Arina Ielchiieva
>Priority: Critical
> Fix For: 1.7.0
>
> Attachments: bug.tgz
>
>
> The below query started to fail from this commit :
> 3209886a8548eea4a2f74c059542672f8665b8d2
> {code}
> select max(dir0) from dfs.`/drill/testdata/bug/2016`;
> Error: UNSUPPORTED_OPERATION ERROR: Streaming aggregate does not support
> schema changes
> Fragment 0:0
> [Error Id: b0060205-e9a6-428a-9803-7b4312b2c6f4 on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> The sub-folders contains files which do have schema change for one column
> "contributions" (int32 vs double). However prior to this commit we did not
> fail in the scenario. Log files and test data are attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-3149) TextReader should support multibyte line delimiters
[ https://issues.apache.org/jira/browse/DRILL-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-3149. - Resolution: Fixed Fix Version/s: (was: Future) 1.7.0 > TextReader should support multibyte line delimiters > --- > > Key: DRILL-3149 > URL: https://issues.apache.org/jira/browse/DRILL-3149 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Text & CSV >Affects Versions: 1.0.0, 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva >Priority: Minor > Labels: doc-impacting > Fix For: 1.7.0 > > > lineDelimiter in the TextFormatConfig doesn't support \r\n for record > delimiters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-3726) Drill is not properly interpreting CRLF (0d0a). CR gets read as content.
[ https://issues.apache.org/jira/browse/DRILL-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-3726. - Resolution: Fixed Fix merged into master with commit id 223507b76ff6c2227e667ae4a53f743c92edd295 > Drill is not properly interpreting CRLF (0d0a). CR gets read as content. > > > Key: DRILL-3726 > URL: https://issues.apache.org/jira/browse/DRILL-3726 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Text & CSV >Affects Versions: 1.1.0 > Environment: Linux RHEL 6.6, OSX 10.9 >Reporter: Edmon Begoli > Fix For: 1.7.0 > > Original Estimate: 120h > Remaining Estimate: 120h > > When we query the last attribute of a text file, we get missing characters. > Looking at the row through Drill, a \r is included at the end of the last > attribute. > Looking in a text editor, it's not embedded into that attribute. > I'm thinking that Drill is not interpreting CRLF (0d0a) as a new line, only > the LF, resulting in the CR becoming part of the last attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3149) TextReader should support multibyte line delimiters
[ https://issues.apache.org/jira/browse/DRILL-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-3149: Labels: doc-impacting (was: ) > TextReader should support multibyte line delimiters > --- > > Key: DRILL-3149 > URL: https://issues.apache.org/jira/browse/DRILL-3149 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Text & CSV >Affects Versions: 1.0.0, 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva >Priority: Minor > Labels: doc-impacting > Fix For: 1.7.0 > > > lineDelimiter in the TextFormatConfig doesn't support \r\n for record > delimiters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-2593. - Resolution: Fixed > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > Attachments: warning1.JPG, warning2.JPG > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4716) status.json doesn't work in drill ui
[
https://issues.apache.org/jira/browse/DRILL-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-4716.
-
Resolution: Fixed
> status.json doesn't work in drill ui
>
>
> Key: DRILL-4716
> URL: https://issues.apache.org/jira/browse/DRILL-4716
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.7.0
>
>
> 1. http://localhost:8047/status returns "Running!"
> But http://localhost:8047/status.json gives error.
> {code}
> {
> "errorMessage" : "HTTP 404 Not Found"
> }
> {code}
> 2. Remove link to System Options on page http://localhost:8047/status as
> redundant.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3726) Drill is not properly interpreting CRLF (0d0a). CR gets read as content.
[ https://issues.apache.org/jira/browse/DRILL-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341825#comment-15341825 ] Arina Ielchiieva commented on DRILL-3726: - User will have two options: 1. specify delimiter in select clause: select * from table(dfs.`my_table`(type=>'text', 'lineDelimiter'=>'\r\n')) 2. update storage plugin lineDelimiter value to '\r\n' on web UI. > Drill is not properly interpreting CRLF (0d0a). CR gets read as content. > > > Key: DRILL-3726 > URL: https://issues.apache.org/jira/browse/DRILL-3726 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Text & CSV >Affects Versions: 1.1.0 > Environment: Linux RHEL 6.6, OSX 10.9 >Reporter: Edmon Begoli > Fix For: 1.7.0 > > Original Estimate: 120h > Remaining Estimate: 120h > > When we query the last attribute of a text file, we get missing characters. > Looking at the row through Drill, a \r is included at the end of the last > attribute. > Looking in a text editor, it's not embedded into that attribute. > I'm thinking that Drill is not interpreting CRLF (0d0a) as a new line, only > the LF, resulting in the CR becoming part of the last attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4701) Fix log name and missing lines in logs on Web UI
[ https://issues.apache.org/jira/browse/DRILL-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341787#comment-15341787 ] Arina Ielchiieva commented on DRILL-4701: - Merged into master with commit id 4123ed2a539cd3f9812f22f96d56aa4709828acd > Fix log name and missing lines in logs on Web UI > > > Key: DRILL-4701 > URL: https://issues.apache.org/jira/browse/DRILL-4701 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > > 1. When the log files are downloaded from the ui, the name of the downloaded > file is "download". We should save the file with the same name as the log > file (ie. drillbit.log) > 2. The last N lines of the log file displayed in the web UI do not match the > log file itself. Some lines are missing compared with actual log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4701) Fix log name and missing lines in logs on Web UI
[ https://issues.apache.org/jira/browse/DRILL-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4701. - Resolution: Fixed > Fix log name and missing lines in logs on Web UI > > > Key: DRILL-4701 > URL: https://issues.apache.org/jira/browse/DRILL-4701 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > > 1. When the log files are downloaded from the ui, the name of the downloaded > file is "download". We should save the file with the same name as the log > file (ie. drillbit.log) > 2. The last N lines of the log file displayed in the web UI do not match the > log file itself. Some lines are missing compared with actual log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4571. - Resolution: Fixed Fix was merged into master with commit id 4123ed2a539cd3f9812f22f96d56aa4709828acd > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Labels: doc-impacting > Fix For: 1.7.0 > > Attachments: display_log.JPG, drillbit_download.log.gz, > drillbit_queries_json_screenshot.jpg, drillbit_ui.log, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4658) cannot specify tab as a fieldDelimiter in table function
[
https://issues.apache.org/jira/browse/DRILL-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-4658.
-
Resolution: Fixed
Fix Version/s: 1.7.0
Fix merged into master with commit id - 223507b76ff6c2227e667ae4a53f743c92edd295
> cannot specify tab as a fieldDelimiter in table function
>
>
> Key: DRILL-4658
> URL: https://issues.apache.org/jira/browse/DRILL-4658
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Affects Versions: 1.6.0
> Environment: Mac OS X, Java 8
>Reporter: Vince Gonzalez
>Assignee: Arina Ielchiieva
> Fix For: 1.7.0
>
>
> I can't specify a tab delimiter in the table function because it maybe counts
> the characters rather than trying to interpret as a character escape code?
> {code}
> 0: jdbc:drill:zk=local> select columns[0] as a, cast(columns[1] as bigint) as
> b from table(dfs.tmp.`sample_cast.tsv`(type => 'text', fieldDelimiter =>
> '\t', skipFirstLine => true));
> Error: PARSE ERROR: Expected single character but was String: \t
> table sample_cast.tsv
> parameter fieldDelimiter
> SQL Query null
> [Error Id: 3efa82e1-3810-4d4a-b23c-32d6658dffcf on 172.30.1.144:31010]
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2593) 500 error when crc for a query profile is out of sync
[ https://issues.apache.org/jira/browse/DRILL-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341763#comment-15341763 ] Arina Ielchiieva commented on DRILL-2593: - Merged into master with commit 2862beaf5c72ccaafc6c52b9956f2d0414948b67 > 500 error when crc for a query profile is out of sync > - > > Key: DRILL-2593 > URL: https://issues.apache.org/jira/browse/DRILL-2593 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 0.7.0 >Reporter: Jason Altekruse >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > Attachments: warning1.JPG, warning2.JPG > > > To reproduce, on a machine where an embedded drillbit has been run, edit one > of the profiles stored in /tmp/drill/profiles and try to navigate to the > profiles page on the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-4733) max(dir0) reading more columns than necessary
[
https://issues.apache.org/jira/browse/DRILL-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-4733:
---
Assignee: Arina Ielchiieva
> max(dir0) reading more columns than necessary
> -
>
> Key: DRILL-4733
> URL: https://issues.apache.org/jira/browse/DRILL-4733
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization, Storage - Parquet
>Affects Versions: 1.7.0
>Reporter: Rahul Challapalli
>Assignee: Arina Ielchiieva
>Priority: Critical
> Fix For: 1.7.0
>
> Attachments: bug.tgz
>
>
> The below query started to fail from this commit :
> 3209886a8548eea4a2f74c059542672f8665b8d2
> {code}
> select max(dir0) from dfs.`/drill/testdata/bug/2016`;
> Error: UNSUPPORTED_OPERATION ERROR: Streaming aggregate does not support
> schema changes
> Fragment 0:0
> [Error Id: b0060205-e9a6-428a-9803-7b4312b2c6f4 on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> The sub-folders contains files which do have schema change for one column
> "contributions" (int32 vs double). However prior to this commit we did not
> fail in the scenario. Log files and test data are attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3149) TextReader should support multibyte line delimiters
[ https://issues.apache.org/jira/browse/DRILL-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341807#comment-15341807 ] Arina Ielchiieva commented on DRILL-3149: - Merged into master with commit id 223507b76ff6c2227e667ae4a53f743c92edd295 > TextReader should support multibyte line delimiters > --- > > Key: DRILL-3149 > URL: https://issues.apache.org/jira/browse/DRILL-3149 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Text & CSV >Affects Versions: 1.0.0, 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva >Priority: Minor > Labels: doc-impacting > Fix For: 1.7.0 > > > lineDelimiter in the TextFormatConfig doesn't support \r\n for record > delimiters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4177) select * from table;Node ran out of Heap memory, exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded
[
https://issues.apache.org/jira/browse/DRILL-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351280#comment-15351280
]
Arina Ielchiieva commented on DRILL-4177:
-
Adding configuration properties ?useCursorFetch=true=1 [1]
helps to solve OOM
Here is some guidelines:
1. Before setting up storage plugin for MySql user needs to add
mysql-connector-java-XXX-bin.jar to /jars/3rdparty/. Please mind cursor fetch configuration works from
5.0.0 version [2].
2. MySql version should be > 5.0.2.
3. Add properties to storage plugin configuration:
{
"type": "jdbc",
"driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://host:port?useCursorFetch=true=1",
"username": "XXX",
"password": "XXX",
"enabled": true
}
Fetch size can be adjusted.
[1]
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
[2] https://dev.mysql.com/downloads/connector/j/
> select * from table;Node ran out of Heap memory,
> exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded
> ---
>
> Key: DRILL-4177
> URL: https://issues.apache.org/jira/browse/DRILL-4177
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: drill1.3 jdk7
>Reporter: david_hudavy
> Labels: patch
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 0: jdbc:drill:zk=local> select * from table;
> Node ran out of Heap memory, exiting.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2149)
> at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1956)
> at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3308)
> at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463)
> at
> com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3032)
> at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2280)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2546)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2504)
> at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177)
> at
> org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:101)
> at
> org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:128)
> at
> org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40)
> at
> org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4171) When exceeded scripts "select * from mscIdentities limit 1;"throwException in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC overhead limit exceeded
[ https://issues.apache.org/jira/browse/DRILL-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351297#comment-15351297 ] Arina Ielchiieva commented on DRILL-4171: - Problems seems the same as indicated in Jira DRILL-4177. Please take a look at comment in DRILL-4177 about adding configuration properties to enable batch read from MySql. > When exceeded scripts "select * from mscIdentities limit 1;"throwException > in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC overhead > limit exceeded > --- > > Key: DRILL-4171 > URL: https://issues.apache.org/jira/browse/DRILL-4171 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.3.0 > Environment: drill 1.3 jdk7 x86_64 GNU/Linux >Reporter: david_hudavy > > 0: jdbc:drill:zk=local> select * from mscIdentities limit 1; > Exception in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC > overhead limit exceeded > at > java.util.concurrent.ConcurrentHashMap$Values.iterator(ConcurrentHashMap.java:1449) > at > org.apache.drill.exec.work.WorkManager$StatusThread.run(WorkManager.java:309) > Node ran out of Heap memory, exiting. > java.lang.OutOfMemoryError: GC overhead limit exceeded > at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2166) > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1956) > at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3308) > at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) > at > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3032) > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2280) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2546) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2504) > at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:101) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:128) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105) > at > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230) > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4696) select four table inner join result.Waiting for a long time after drill report error java.lang.OutOfMemoryError: Java heap space
[ https://issues.apache.org/jira/browse/DRILL-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351403#comment-15351403 ] Arina Ielchiieva commented on DRILL-4696: - Queries issued via jdbc in fact being pushed down completely to MySQL. Using batches to return data might resolve the issue. Solution is described in Jira DRILL-4177. > select four table inner join result.Waiting for a long time after drill > report error java.lang.OutOfMemoryError: Java heap space > > > Key: DRILL-4696 > URL: https://issues.apache.org/jira/browse/DRILL-4696 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.6.0 > Environment: Test Environment: > SUSE Linux Enterprise Server 11 SP3 (x86_64) cluster > MySQL 5.7.11 Enterprise Server - Advanced Edition > Drill cluster >Reporter: david_hudavy > > Test Environment: > cluster 10-3 > MySQL 5.7.11 Enterprise Server - Advanced Edition > Drill cluster > Test Scope: > select performance of huge table(30M records). > MySQL table: a b c d inner join (Four table each have 30M records) > -- four table inner join: (take time Drill Crash) > 0: jdbc:drill:zk=SC-1:6181,SC-2:6181,PL-3:618> select > . . . . . . . . . . . . . . . . . . . . . . .> d.d_5, > . . . . . . . . . . . . . . . . . . . . . . .> c.c_3, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_4, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_5, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_6, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_7, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_8 , > . . . . . . . . . . . . . . . . . . . . . . .> a.a_9 , > . . . . . . . . . . . . . . . . . . . . . . .> a.a_10, > . . . . . . . . . . . . . . . . . . . . . . .> a.a_11 , > . . . . . . . . . . . . . . . . . . . . . . .> a.a_12, > . . . . . . . . . . . . . . . . . . . . . . .> b.b_4 > . . . . . . . . . . . . . . . . . . . . . . .> from > mysql.user_data.a,mysql.user_data.b,mysql.user_data.c,mysql.user_data.d > . . . . . . . . . . . . . . . . . . . . . . .> where c.c_3=a.a_3 and > a.a_3 =b.b_3 and b.b_3=d.d_3 > . . . . . . . . . . . . . . . . . . . . . . .> and c.c_3='0'; > Drill Crash > 2016-05-13 09:52:35,131 [28cacd19-0f04-cbb1-b418-73a76dcd6ebe:frag:0:0] ERROR > o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, > exiting. Information message: Unable to handle out of memory condition in > FragmentExecutor. > java.lang.OutOfMemoryError: Java heap space > at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2157) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1964) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3316) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3040) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2288) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2681) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2547) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2505) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) > ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > ~[commons-dbcp-1.4.jar:1.4] > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > ~[commons-dbcp-1.4.jar:1.4] > at > org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) > ~[drill-jdbc-storage-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:108) > ~[drill-java-exec-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:136) > ~[drill-java-exec-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) > ~[drill-jdbc-storage-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) >
[jira] [Assigned] (DRILL-3688) Drill should honor "skip.header.line.count" attribute of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-3688:
---
Assignee: Arina Ielchiieva
> Drill should honor "skip.header.line.count" attribute of Hive table
> ---
>
> Key: DRILL-3688
> URL: https://issues.apache.org/jira/browse/DRILL-3688
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Affects Versions: 1.1.0
> Environment: 1.1
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Currently Drill does not honor the "skip.header.line.count" attribute of Hive
> table.
> It may cause some other format conversion issue.
> Reproduce:
> 1. Create a Hive table
> {code}
> create table h1db.testheader(col0 string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
> STORED AS TEXTFILE
> tblproperties("skip.header.line.count"="1");
> {code}
> 2. Prepare a sample data:
> {code}
> # cat test.data
> col0
> 2015-01-01
> {code}
> 3. Load sample data into Hive
> {code}
> LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
> {code}
> 4. Hive
> {code}
> hive> select * from h1db.testheader ;
> OK
> 2015-01-01
> Time taken: 0.254 seconds, Fetched: 1 row(s)
> {code}
> 5. Drill
> {code}
> > select * from hive.h1db.testheader ;
> +-+
> |col0 |
> +-+
> | col0|
> | 2015-01-01 |
> +-+
> 2 rows selected (0.257 seconds)
> > select cast(col0 as date) from hive.h1db.testheader ;
> Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
> be in the range [1,12]
> Fragment 0:0
> [Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
> (org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be
> in the range [1,12]
> org.joda.time.field.FieldUtils.verifyValueBounds():236
> org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
> org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
> org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
> org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
> org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():147
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3944) Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
[
https://issues.apache.org/jira/browse/DRILL-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123653#comment-15123653
]
Arina Ielchiieva commented on DRILL-3944:
-
[Jason
Altekruse|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jaltekruse],
you are right, there is no such concept in sql, like banning scalar function
from select list. Maxdir looks like *scalar function* (meaning, it takes #
parameters and returns one value per row), so it doesn’t seem right to ban it
from there.
Taking into account that it's not allowed anywhere else except from clause, in
a way it behaves like *table-value function*.
Currently function syntax is the following:
{code}
select * from vspace.wspace.`freemat2` where dir0 = maxdir('vspace.wspace',
'freemat2');
{code}
Table repeats two times in query. Is there a case when maxdir IN parameters
will differ from those in from clause?
Because table-value design could look like:
{code}
select * from maxdir('vspace.wspace', 'freemat2', ‘dir0’);
{code}
_I am not suggesting changing current design, it's just an observation_
> Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
> --
>
> Key: DRILL-3944
> URL: https://issues.apache.org/jira/browse/DRILL-3944
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: 1.2.0
>Reporter: Jitendra
>Assignee: Arina Ielchiieva
> Attachments: newStackTrace.txt
>
>
> We are facing issue with MAXDIR function, below is the query we are using to
> reproduce this issue.
> 0: jdbc:drill:drillbit=localhost> select maxdir('vspace.wspace', 'freemat2')
> from vspace.wspace.`freemat2`;
> Error: SYSTEM ERROR: CompileException: Line 75, Column 70: Unknown variable
> or type "FILE_SEPARATOR"
> Fragment 0:0
> [Error Id: d17c6e48-554d-4934-bc4d-783ca3dc6f51 on 10.10.99.71:31010]
> (state=,code=0);
> Below are the drillbit logs.
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: RUNNING
> 2015-10-09 21:26:22,038 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested RUNNING -->
> FINISHED
> 2015-10-09 21:26:22,039 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: FINISHED
> 2015-10-09 21:29:59,281 [29e7ce27-9cad-9d8a-a482-39f54cc7deda:foreman] INFO
> o.a.d.e.store.mock.MockStorageEngine - Failure while attempting to check for
> Parquet metadata file.
> java.io.IOException: Open failed for file: /vspace/wspace/freemat2/20151005,
> error: Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:212)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:862)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:132)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:142)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isFileReadable(BasicFormatMatcher.java:112)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isDirReadable(ParquetFormatPlugin.java:256)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:210)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:326)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:153)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:276)
>
[jira] [Assigned] (DRILL-3944) Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
[
https://issues.apache.org/jira/browse/DRILL-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-3944:
---
Assignee: Arina Ielchiieva (was: Jason Altekruse)
> Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
> --
>
> Key: DRILL-3944
> URL: https://issues.apache.org/jira/browse/DRILL-3944
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: 1.2.0
>Reporter: Jitendra
>Assignee: Arina Ielchiieva
> Attachments: newStackTrace.txt
>
>
> We are facing issue with MAXDIR function, below is the query we are using to
> reproduce this issue.
> 0: jdbc:drill:drillbit=localhost> select maxdir('vspace.wspace', 'freemat2')
> from vspace.wspace.`freemat2`;
> Error: SYSTEM ERROR: CompileException: Line 75, Column 70: Unknown variable
> or type "FILE_SEPARATOR"
> Fragment 0:0
> [Error Id: d17c6e48-554d-4934-bc4d-783ca3dc6f51 on 10.10.99.71:31010]
> (state=,code=0);
> Below are the drillbit logs.
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: RUNNING
> 2015-10-09 21:26:22,038 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested RUNNING -->
> FINISHED
> 2015-10-09 21:26:22,039 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: FINISHED
> 2015-10-09 21:29:59,281 [29e7ce27-9cad-9d8a-a482-39f54cc7deda:foreman] INFO
> o.a.d.e.store.mock.MockStorageEngine - Failure while attempting to check for
> Parquet metadata file.
> java.io.IOException: Open failed for file: /vspace/wspace/freemat2/20151005,
> error: Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:212)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:862)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:132)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:142)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isFileReadable(BasicFormatMatcher.java:112)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isDirReadable(ParquetFormatPlugin.java:256)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:210)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:326)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:153)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:276)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.calcite.jdbc.SimpleCalciteSchema.getTable(SimpleCalciteSchema.java:83)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTableFrom(CalciteCatalogReader.java:116)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:99)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:70)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:75)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.sql.validate.DelegatingScope.getTableNamespace(DelegatingScope.java:124)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
>
[jira] [Commented] (DRILL-3547) IndexOutOfBoundsException on directory with ~20 subdirectories
[
https://issues.apache.org/jira/browse/DRILL-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120310#comment-15120310
]
Arina Ielchiieva commented on DRILL-3547:
-
Can't reproduce. Please check if this issue is still valid.
Tested on:
Version 1.3.0, 1.5.0-SNAPSHOT (commit 3d0b4b02521f12e3871d6060c8f9bfce73b218a0)
> IndexOutOfBoundsException on directory with ~20 subdirectories
> --
>
> Key: DRILL-3547
> URL: https://issues.apache.org/jira/browse/DRILL-3547
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.1.0
> Environment: RHEL 7
>Reporter: Philip Deegan
>Assignee: Arina Ielchiieva
> Fix For: 1.5.0
>
>
> Tested on 1.1 with commit id:
> {noformat}
> 0: jdbc:drill:zk=local> select commit_id from sys.version;
> +---+
> | commit_id |
> +---+
> | e3fc7e97bfe712dc09d43a8a055a5135c96b7344 |
> +---+
> {noformat}
> Directory has child directories "a" to "u", each contain json files.
> Running the query on each subdirectory indivudually does not cause an error.
> {noformat}
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> Fragment 1:2
> [Error Id: 69a0879f-f718-4930-ae6f-c526de05528c on
> ip-172-31-29-60.eu-central-1.compute.internal:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (DRILL-3547) IndexOutOfBoundsException on directory with ~20 subdirectories
[
https://issues.apache.org/jira/browse/DRILL-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-3547:
---
Assignee: Arina Ielchiieva
> IndexOutOfBoundsException on directory with ~20 subdirectories
> --
>
> Key: DRILL-3547
> URL: https://issues.apache.org/jira/browse/DRILL-3547
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.1.0
> Environment: RHEL 7
>Reporter: Philip Deegan
>Assignee: Arina Ielchiieva
> Fix For: 1.5.0
>
>
> Tested on 1.1 with commit id:
> {noformat}
> 0: jdbc:drill:zk=local> select commit_id from sys.version;
> +---+
> | commit_id |
> +---+
> | e3fc7e97bfe712dc09d43a8a055a5135c96b7344 |
> +---+
> {noformat}
> Directory has child directories "a" to "u", each contain json files.
> Running the query on each subdirectory indivudually does not cause an error.
> {noformat}
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> Fragment 1:2
> [Error Id: 69a0879f-f718-4930-ae6f-c526de05528c on
> ip-172-31-29-60.eu-central-1.compute.internal:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (DRILL-3610) TimestampAdd/Diff (SQL_TSI_) functions
[
https://issues.apache.org/jira/browse/DRILL-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-3610:
---
Assignee: Arina Ielchiieva
> TimestampAdd/Diff (SQL_TSI_) functions
> --
>
> Key: DRILL-3610
> URL: https://issues.apache.org/jira/browse/DRILL-3610
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Reporter: Andries Engelbrecht
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Add TimestampAdd and TimestampDiff (SQL_TSI) functions for year, quarter,
> month, week, day, hour, minute, second.
> Examples
> SELECT CAST(TIMESTAMPADD(SQL_TSI_QUARTER,1,Date('2013-03-31'), SQL_DATE) AS
> `column_quarter`
> FROM `table_in`
> HAVING (COUNT(1) > 0)
> SELECT `table_in`.`datetime` AS `column1`,
> `table`.`Key` AS `column_Key`,
> TIMESTAMPDIFF(SQL_TSI_MINUTE,to_timestamp('2004-07-04',
> '-MM-dd'),`table_in`.`datetime`) AS `sum_datediff_minute`
> FROM `calcs`
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (DRILL-3547) IndexOutOfBoundsException on directory with ~20 subdirectories
[
https://issues.apache.org/jira/browse/DRILL-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120310#comment-15120310
]
Arina Ielchiieva edited comment on DRILL-3547 at 1/31/16 2:56 PM:
--
Can't reproduce. Please check if this issue is still valid.
Tested on:
Version 1.3.0, 1.4.0, 1.5.0-SNAPSHOT (commit
3d0b4b02521f12e3871d6060c8f9bfce73b218a0)
was (Author: arina):
Can't reproduce. Please check if this issue is still valid.
Tested on:
Version 1.3.0, 1.5.0-SNAPSHOT (commit 3d0b4b02521f12e3871d6060c8f9bfce73b218a0)
> IndexOutOfBoundsException on directory with ~20 subdirectories
> --
>
> Key: DRILL-3547
> URL: https://issues.apache.org/jira/browse/DRILL-3547
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.1.0
> Environment: RHEL 7
>Reporter: Philip Deegan
>Assignee: Arina Ielchiieva
> Fix For: 1.5.0
>
>
> Tested on 1.1 with commit id:
> {noformat}
> 0: jdbc:drill:zk=local> select commit_id from sys.version;
> +---+
> | commit_id |
> +---+
> | e3fc7e97bfe712dc09d43a8a055a5135c96b7344 |
> +---+
> {noformat}
> Directory has child directories "a" to "u", each contain json files.
> Running the query on each subdirectory indivudually does not cause an error.
> {noformat}
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> Fragment 1:2
> [Error Id: 69a0879f-f718-4930-ae6f-c526de05528c on
> ip-172-31-29-60.eu-central-1.compute.internal:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3944) Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
[
https://issues.apache.org/jira/browse/DRILL-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120240#comment-15120240
]
Arina Ielchiieva commented on DRILL-3944:
-
Making change in DirectoryExplorers.java applies to all such functions, since
they all are generated from this template.
Exception is generated in on the fly created class ProjectorGen0 in method
doSetup:
work1 = (incoming).getContext().getPartitionExplorer();
PartitionExplorer is injected into it (@Inject
org.apache.drill.exec.store.PartitionExplorer partitionExplorer).
Catching this exception is possible in two classes:
1. org.apache.drill.exec.physical.impl.project.ProjectorTemplate setup method
when it calls doSetup;
2. org.apache.drill.exec.physical.impl.project.ProjectRecordBatch in method
setupNewSchema() where Projector setup method is called.
_But to my opinion neither of these two classes should care for such exception._
Probably we can enhance exception message in FragmentContext but it seems to be
pretty valid. Currently it's the following:
{code}
@Override
public PartitionExplorer getPartitionExplorer() {
throw new UnsupportedOperationException(String.format("The partition
explorer interface can only be used " +
"in functions that can be evaluated at planning time. Make sure that
the %s configuration " +
"option is set to true.",
PlannerSettings.CONSTANT_FOLDING.getOptionName()));
}
{code}
But we still need [Jason
Altekruse|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jaltekruse]
opinion on planner.enable_constant_folding.
> Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
> --
>
> Key: DRILL-3944
> URL: https://issues.apache.org/jira/browse/DRILL-3944
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: 1.2.0
>Reporter: Jitendra
>Assignee: Jason Altekruse
> Attachments: newStackTrace.txt
>
>
> We are facing issue with MAXDIR function, below is the query we are using to
> reproduce this issue.
> 0: jdbc:drill:drillbit=localhost> select maxdir('vspace.wspace', 'freemat2')
> from vspace.wspace.`freemat2`;
> Error: SYSTEM ERROR: CompileException: Line 75, Column 70: Unknown variable
> or type "FILE_SEPARATOR"
> Fragment 0:0
> [Error Id: d17c6e48-554d-4934-bc4d-783ca3dc6f51 on 10.10.99.71:31010]
> (state=,code=0);
> Below are the drillbit logs.
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: RUNNING
> 2015-10-09 21:26:22,038 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested RUNNING -->
> FINISHED
> 2015-10-09 21:26:22,039 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: FINISHED
> 2015-10-09 21:29:59,281 [29e7ce27-9cad-9d8a-a482-39f54cc7deda:foreman] INFO
> o.a.d.e.store.mock.MockStorageEngine - Failure while attempting to check for
> Parquet metadata file.
> java.io.IOException: Open failed for file: /vspace/wspace/freemat2/20151005,
> error: Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:212)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:862)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:132)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:142)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isFileReadable(BasicFormatMatcher.java:112)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isDirReadable(ParquetFormatPlugin.java:256)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:210)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:326)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
>
[jira] [Commented] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15163837#comment-15163837 ] Arina Ielchiieva commented on DRILL-3745: - Since Drill basically handles all chars as varchars to work correctly with hive chars, we can return them as varchars but trimmed beforehand so they can be compared to each other. Proposed solution - [https://github.com/arina-ielchiieva/drill/commit/0e3821e5d100d295e163d7d03d94a064329a4982] Can anybody look at it? > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-3745: --- Assignee: Arina Ielchiieva > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-2805) Throw clear error message when maxdir/mindir UDFs are used in any other part of SQL, but where clause
[
https://issues.apache.org/jira/browse/DRILL-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva closed DRILL-2805.
---
Resolution: Fixed
Assignee: Arina Ielchiieva
Fixed in cca17cb in sope of DRILL-3944
> Throw clear error message when maxdir/mindir UDFs are used in any other part
> of SQL, but where clause
> -
>
> Key: DRILL-2805
> URL: https://issues.apache.org/jira/browse/DRILL-2805
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> This error message is not right. I think the problem here is that we are not
> allowing to use these UDFs in projection list, group by or order by.
> Can we please have an error message that says that ?
> {code}
> 0: jdbc:drill:schema=dfs> select maxdir('dfs.test', 'bigtable') from
> smalltable group by maxdir('dfs.test', 'bigtable');
> Query failed: SYSTEM ERROR: The partition explorer interface can only be used
> in functions that can be evaluated at planning time. Make sure that the
> planner.enable_constant_folding configuration option is set to true.
> [6c9dd988-0960-4a3a-9495-535e245f0daa on atsqa4-134.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> This is the correct way this error message is used:
> {code}
> 0: jdbc:drill:schema=dfs> select * from smalltable where dir0 =
> maxdir('dfs.test', 'bigtable') and dir1 = mindir('dfs.test', 'bigtable/2016');
> ++++
> | columns |dir0|dir1|
> ++++
> ++++
> No rows selected (0.943 seconds)
> 0: jdbc:drill:schema=dfs> alter session set `planner.enable_constant_folding`
> = false;
> +++
> | ok | summary |
> +++
> | true | planner.enable_constant_folding updated. |
> +++
> 1 row selected (0.108 seconds)
> 0: jdbc:drill:schema=dfs> select * from smalltable where dir0 =
> maxdir('dfs.test', 'bigtable') and dir1 = mindir('dfs.test', 'bigtable/2016');
> Query failed: SYSTEM ERROR: The partition explorer interface can only be used
> in functions that can be evaluated at planning time. Make sure that the
> planner.enable_constant_folding configuration option is set to true.
> [c22bad01-f9c7-43ba-b76c-8a11be518b40 on atsqa4-134.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Btw, I'm not sure I understand why we want to make constant folding
> configurable. Is the any benefit to not have it turned on ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172152#comment-15172152 ] Arina Ielchiieva commented on DRILL-3745: - [Venki Korukanti|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vkorukanti], I have added char support in Hive UDFs but it's only applicable for udf return type. When char is udf return type, we convert it to varchar, that's simple here. But we'll get an error if udf expects char as input parameter since Drill has varchar holder for both char and varchar and it's impossible to determine which one it is. Link - https://github.com/arina-ielchiieva/drill/commit/1a167efecf73b2b6e91585798914a79237b68534 > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3944) Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
[
https://issues.apache.org/jira/browse/DRILL-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117535#comment-15117535
]
Arina Ielchiieva commented on DRILL-3944:
-
As part of solution we can update the template
src/main/codegen/templates/DirectoryExplorers.java: moving FILE_SEPARATOR from
class to method level, which will result in different error, probably more
obvious:
*org.apache.drill.exec.rpc.RpcException:
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
UnsupportedOperationException: The partition explorer interface can only be
used in functions that can be evaluated at planning time. Make sure that the
planner.enable_constant_folding configuration option is set to true.*
Full stack trace is attached (newStackTrace.txt).
> Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
> --
>
> Key: DRILL-3944
> URL: https://issues.apache.org/jira/browse/DRILL-3944
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: 1.2.0
>Reporter: Jitendra
>Assignee: Jason Altekruse
>
> We are facing issue with MAXDIR function, below is the query we are using to
> reproduce this issue.
> 0: jdbc:drill:drillbit=localhost> select maxdir('vspace.wspace', 'freemat2')
> from vspace.wspace.`freemat2`;
> Error: SYSTEM ERROR: CompileException: Line 75, Column 70: Unknown variable
> or type "FILE_SEPARATOR"
> Fragment 0:0
> [Error Id: d17c6e48-554d-4934-bc4d-783ca3dc6f51 on 10.10.99.71:31010]
> (state=,code=0);
> Below are the drillbit logs.
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: RUNNING
> 2015-10-09 21:26:22,038 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested RUNNING -->
> FINISHED
> 2015-10-09 21:26:22,039 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: FINISHED
> 2015-10-09 21:29:59,281 [29e7ce27-9cad-9d8a-a482-39f54cc7deda:foreman] INFO
> o.a.d.e.store.mock.MockStorageEngine - Failure while attempting to check for
> Parquet metadata file.
> java.io.IOException: Open failed for file: /vspace/wspace/freemat2/20151005,
> error: Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:212)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:862)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:132)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:142)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isFileReadable(BasicFormatMatcher.java:112)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isDirReadable(ParquetFormatPlugin.java:256)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:210)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:326)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:153)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:276)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.calcite.jdbc.SimpleCalciteSchema.getTable(SimpleCalciteSchema.java:83)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTableFrom(CalciteCatalogReader.java:116)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
>
[jira] [Updated] (DRILL-3944) Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
[
https://issues.apache.org/jira/browse/DRILL-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-3944:
Attachment: newStackTrace.txt
> Drill MAXDIR Unknown variable or type "FILE_SEPARATOR"
> --
>
> Key: DRILL-3944
> URL: https://issues.apache.org/jira/browse/DRILL-3944
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: 1.2.0
>Reporter: Jitendra
>Assignee: Jason Altekruse
> Attachments: newStackTrace.txt
>
>
> We are facing issue with MAXDIR function, below is the query we are using to
> reproduce this issue.
> 0: jdbc:drill:drillbit=localhost> select maxdir('vspace.wspace', 'freemat2')
> from vspace.wspace.`freemat2`;
> Error: SYSTEM ERROR: CompileException: Line 75, Column 70: Unknown variable
> or type "FILE_SEPARATOR"
> Fragment 0:0
> [Error Id: d17c6e48-554d-4934-bc4d-783ca3dc6f51 on 10.10.99.71:31010]
> (state=,code=0);
> Below are the drillbit logs.
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-10-09 21:26:21,972 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: RUNNING
> 2015-10-09 21:26:22,038 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State change requested RUNNING -->
> FINISHED
> 2015-10-09 21:26:22,039 [29e7cf02-02bf-b007-72f2-52c67c80ea1c:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29e7cf02-02bf-b007-72f2-52c67c80ea1c:0:0: State to report: FINISHED
> 2015-10-09 21:29:59,281 [29e7ce27-9cad-9d8a-a482-39f54cc7deda:foreman] INFO
> o.a.d.e.store.mock.MockStorageEngine - Failure while attempting to check for
> Parquet metadata file.
> java.io.IOException: Open failed for file: /vspace/wspace/freemat2/20151005,
> error: Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:212)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:862)
> ~[maprfs-4.1.0-mapr.jar:4.1.0-mapr]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:800)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:132)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:142)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isFileReadable(BasicFormatMatcher.java:112)
> ~[drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isDirReadable(ParquetFormatPlugin.java:256)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:210)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:326)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:153)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:276)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.calcite.jdbc.SimpleCalciteSchema.getTable(SimpleCalciteSchema.java:83)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTableFrom(CalciteCatalogReader.java:116)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:99)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:70)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:75)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.sql.validate.DelegatingScope.getTableNamespace(DelegatingScope.java:124)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
>
[jira] [Resolved] (DRILL-3547) IndexOutOfBoundsException on directory with ~20 subdirectories
[
https://issues.apache.org/jira/browse/DRILL-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-3547.
-
Resolution: Cannot Reproduce
> IndexOutOfBoundsException on directory with ~20 subdirectories
> --
>
> Key: DRILL-3547
> URL: https://issues.apache.org/jira/browse/DRILL-3547
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.1.0
> Environment: RHEL 7
>Reporter: Philip Deegan
>Assignee: Arina Ielchiieva
> Fix For: 1.6.0
>
>
> Tested on 1.1 with commit id:
> {noformat}
> 0: jdbc:drill:zk=local> select commit_id from sys.version;
> +---+
> | commit_id |
> +---+
> | e3fc7e97bfe712dc09d43a8a055a5135c96b7344 |
> +---+
> {noformat}
> Directory has child directories "a" to "u", each contain json files.
> Running the query on each subdirectory indivudually does not cause an error.
> {noformat}
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> Fragment 1:2
> [Error Id: 69a0879f-f718-4930-ae6f-c526de05528c on
> ip-172-31-29-60.eu-central-1.compute.internal:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3688) Drill should honor "skip.header.line.count" attribute of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147809#comment-15147809
]
Arina Ielchiieva commented on DRILL-3688:
-
Since Hive table properties take String, such property can be set as with
non-numeric value.
Ex:
{code}
create table h1db.testheader(col0 string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
tblproperties("skip.header.line.count"="");
{code}
Hive allows to create such table, and even load data into it. But when you
query such table, Hive throws NumberFormatException: For input string: ""
Since Drill doesn't select from Hive directly, we can handle such case and
ignore incorrect property, i.e. replace incorrect property value to 0.
Which behavior is better to choose: fail as Hive or let user query the data?
> Drill should honor "skip.header.line.count" attribute of Hive table
> ---
>
> Key: DRILL-3688
> URL: https://issues.apache.org/jira/browse/DRILL-3688
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Affects Versions: 1.1.0
> Environment: 1.1
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Currently Drill does not honor the "skip.header.line.count" attribute of Hive
> table.
> It may cause some other format conversion issue.
> Reproduce:
> 1. Create a Hive table
> {code}
> create table h1db.testheader(col0 string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
> STORED AS TEXTFILE
> tblproperties("skip.header.line.count"="1");
> {code}
> 2. Prepare a sample data:
> {code}
> # cat test.data
> col0
> 2015-01-01
> {code}
> 3. Load sample data into Hive
> {code}
> LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
> {code}
> 4. Hive
> {code}
> hive> select * from h1db.testheader ;
> OK
> 2015-01-01
> Time taken: 0.254 seconds, Fetched: 1 row(s)
> {code}
> 5. Drill
> {code}
> > select * from hive.h1db.testheader ;
> +-+
> |col0 |
> +-+
> | col0|
> | 2015-01-01 |
> +-+
> 2 rows selected (0.257 seconds)
> > select cast(col0 as date) from hive.h1db.testheader ;
> Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
> be in the range [1,12]
> Fragment 0:0
> [Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
> (org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be
> in the range [1,12]
> org.joda.time.field.FieldUtils.verifyValueBounds():236
> org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
> org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
> org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
> org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
> org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():147
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3688) Drill should honor "skip.header.line.count" attribute of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147813#comment-15147813
]
Arina Ielchiieva commented on DRILL-3688:
-
Hive also has similar property "skip.footer.line.count", do we need to support
it, too?
> Drill should honor "skip.header.line.count" attribute of Hive table
> ---
>
> Key: DRILL-3688
> URL: https://issues.apache.org/jira/browse/DRILL-3688
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Affects Versions: 1.1.0
> Environment: 1.1
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Currently Drill does not honor the "skip.header.line.count" attribute of Hive
> table.
> It may cause some other format conversion issue.
> Reproduce:
> 1. Create a Hive table
> {code}
> create table h1db.testheader(col0 string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
> STORED AS TEXTFILE
> tblproperties("skip.header.line.count"="1");
> {code}
> 2. Prepare a sample data:
> {code}
> # cat test.data
> col0
> 2015-01-01
> {code}
> 3. Load sample data into Hive
> {code}
> LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
> {code}
> 4. Hive
> {code}
> hive> select * from h1db.testheader ;
> OK
> 2015-01-01
> Time taken: 0.254 seconds, Fetched: 1 row(s)
> {code}
> 5. Drill
> {code}
> > select * from hive.h1db.testheader ;
> +-+
> |col0 |
> +-+
> | col0|
> | 2015-01-01 |
> +-+
> 2 rows selected (0.257 seconds)
> > select cast(col0 as date) from hive.h1db.testheader ;
> Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
> be in the range [1,12]
> Fragment 0:0
> [Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
> (org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be
> in the range [1,12]
> org.joda.time.field.FieldUtils.verifyValueBounds():236
> org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
> org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
> org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
> org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
> org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():147
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3688) Drill should honor "skip.header.line.count" and "skip.footer.line.count" attributes of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-3688:
Description:
Currently Drill does not honor the "skip.header.line.count" attribute of Hive
table.
It may cause some other format conversion issue.
Reproduce:
1. Create a Hive table
{code}
create table h1db.testheader(col0 string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
tblproperties("skip.header.line.count"="1");
{code}
2. Prepare a sample data:
{code}
# cat test.data
col0
2015-01-01
{code}
3. Load sample data into Hive
{code}
LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
{code}
4. Hive
{code}
hive> select * from h1db.testheader ;
OK
2015-01-01
Time taken: 0.254 seconds, Fetched: 1 row(s)
{code}
5. Drill
{code}
> select * from hive.h1db.testheader ;
+-+
|col0 |
+-+
| col0|
| 2015-01-01 |
+-+
2 rows selected (0.257 seconds)
> select cast(col0 as date) from hive.h1db.testheader ;
Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
be in the range [1,12]
Fragment 0:0
[Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
(org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be in
the range [1,12]
org.joda.time.field.FieldUtils.verifyValueBounds():236
org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
org.apache.drill.exec.record.AbstractRecordBatch.next():147
org.apache.drill.exec.physical.impl.BaseRootExec.next():83
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
org.apache.drill.exec.physical.impl.BaseRootExec.next():73
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1566
org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1142
java.util.concurrent.ThreadPoolExecutor$Worker.run():617
java.lang.Thread.run():745 (state=,code=0)
{code}
Also "skip.footer.line.count" should be taken into account.
If "skip.header.line.count" or "skip.footer.line.count" has incorrect value in
Hive, throw appropriate exception in Drill.
Ex: Hive table property skip.header.line.count value 'someValue' is non-numeric
was:
Currently Drill does not honor the "skip.header.line.count" attribute of Hive
table.
It may cause some other format conversion issue.
Reproduce:
1. Create a Hive table
{code}
create table h1db.testheader(col0 string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
tblproperties("skip.header.line.count"="1");
{code}
2. Prepare a sample data:
{code}
# cat test.data
col0
2015-01-01
{code}
3. Load sample data into Hive
{code}
LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
{code}
4. Hive
{code}
hive> select * from h1db.testheader ;
OK
2015-01-01
Time taken: 0.254 seconds, Fetched: 1 row(s)
{code}
5. Drill
{code}
> select * from hive.h1db.testheader ;
+-+
|col0 |
+-+
| col0|
| 2015-01-01 |
+-+
2 rows selected (0.257 seconds)
> select cast(col0 as date) from hive.h1db.testheader ;
Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
be in the range [1,12]
Fragment 0:0
[Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
(org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be in
the range [1,12]
org.joda.time.field.FieldUtils.verifyValueBounds():236
org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
[jira] [Created] (DRILL-4456) Hive translate function is not working
Arina Ielchiieva created DRILL-4456:
---
Summary: Hive translate function is not working
Key: DRILL-4456
URL: https://issues.apache.org/jira/browse/DRILL-4456
Project: Apache Drill
Issue Type: Improvement
Components: Functions - Hive
Affects Versions: 1.5.0
Reporter: Arina Ielchiieva
Fix For: Future
In Hive "select translate(name, 'A', 'B') from users" works fine.
But in Drill "select translate(name, 'A', 'B') from hive.`users`" returns the
following error:
org.apache.drill.common.exceptions.UserRemoteException: PARSE ERROR:
Encountered "," at line 1, column 22. Was expecting one of: "USING" ... "NOT"
... "IN" ... "BETWEEN" ... "LIKE" ... "SIMILAR" ... "=" ... ">" ... "<" ...
"<=" ... ">=" ... "<>" ... "+" ... "-" ... "*" ... "/" ... "||" ... "AND" ...
"OR" ... "IS" ... "MEMBER" ... "SUBMULTISET" ... "MULTISET" ... "[" ... "." ...
"(" ... while parsing SQL query: select translate(name, 'A', 'B') from
hive.users ^ [Error Id: ba21956b-3285-4544-b3b2-fab68b95be1f on localhost:31010]
Root cause:
Calcite follows the standard SQL reference.
SQL reference, ISO/IEC 9075-2:2011(E), section 6.30
::=
TRANSLATE
USING
To fix:
1. add support to translate (expession, from_string, to_string) alternative
syntax
2. add unit test in org.apache.drill.exec.fn.hive.TestInbuiltHiveUDFs
Changes can be made directly in Calcite and then upgrade to appropriate Calcite
version.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173385#comment-15173385 ] Arina Ielchiieva commented on DRILL-3745: - Totally agree, usually in Hive for strings check is done either for PrimitiveGrouping.STRING_GROUP (ex; concat_ws function), or explicitly for each string type (ex: PrimitiveCategory.STRING, PrimitiveCategory.CHAR, PrimitiveCategory.VARCHAR in translate function). If code looks good, I'll create pull request then. > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3688) Drill should honor "skip.header.line.count" and "skip.footer.line.count" attributes of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173392#comment-15173392
]
Arina Ielchiieva commented on DRILL-3688:
-
Fixed in 84ce21c.
> Drill should honor "skip.header.line.count" and "skip.footer.line.count"
> attributes of Hive table
> -
>
> Key: DRILL-3688
> URL: https://issues.apache.org/jira/browse/DRILL-3688
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Affects Versions: 1.1.0
> Environment: 1.1
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: 1.6.0
>
>
> Currently Drill does not honor the "skip.header.line.count" attribute of Hive
> table.
> It may cause some other format conversion issue.
> Reproduce:
> 1. Create a Hive table
> {code}
> create table h1db.testheader(col0 string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
> STORED AS TEXTFILE
> tblproperties("skip.header.line.count"="1");
> {code}
> 2. Prepare a sample data:
> {code}
> # cat test.data
> col0
> 2015-01-01
> {code}
> 3. Load sample data into Hive
> {code}
> LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
> {code}
> 4. Hive
> {code}
> hive> select * from h1db.testheader ;
> OK
> 2015-01-01
> Time taken: 0.254 seconds, Fetched: 1 row(s)
> {code}
> 5. Drill
> {code}
> > select * from hive.h1db.testheader ;
> +-+
> |col0 |
> +-+
> | col0|
> | 2015-01-01 |
> +-+
> 2 rows selected (0.257 seconds)
> > select cast(col0 as date) from hive.h1db.testheader ;
> Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
> be in the range [1,12]
> Fragment 0:0
> [Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
> (org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be
> in the range [1,12]
> org.joda.time.field.FieldUtils.verifyValueBounds():236
> org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
> org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
> org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
> org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
> org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():147
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
> Also "skip.footer.line.count" should be taken into account.
> If "skip.header.line.count" or "skip.footer.line.count" has incorrect value
> in Hive, throw appropriate exception in Drill.
> Ex: Hive table property skip.header.line.count value 'someValue' is
> non-numeric
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-3688) Drill should honor "skip.header.line.count" and "skip.footer.line.count" attributes of Hive table
[
https://issues.apache.org/jira/browse/DRILL-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-3688.
-
Resolution: Fixed
Fix Version/s: (was: Future)
1.6.0
> Drill should honor "skip.header.line.count" and "skip.footer.line.count"
> attributes of Hive table
> -
>
> Key: DRILL-3688
> URL: https://issues.apache.org/jira/browse/DRILL-3688
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Affects Versions: 1.1.0
> Environment: 1.1
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: 1.6.0
>
>
> Currently Drill does not honor the "skip.header.line.count" attribute of Hive
> table.
> It may cause some other format conversion issue.
> Reproduce:
> 1. Create a Hive table
> {code}
> create table h1db.testheader(col0 string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
> STORED AS TEXTFILE
> tblproperties("skip.header.line.count"="1");
> {code}
> 2. Prepare a sample data:
> {code}
> # cat test.data
> col0
> 2015-01-01
> {code}
> 3. Load sample data into Hive
> {code}
> LOAD DATA LOCAL INPATH '/xxx/test.data' OVERWRITE INTO TABLE h1db.testheader;
> {code}
> 4. Hive
> {code}
> hive> select * from h1db.testheader ;
> OK
> 2015-01-01
> Time taken: 0.254 seconds, Fetched: 1 row(s)
> {code}
> 5. Drill
> {code}
> > select * from hive.h1db.testheader ;
> +-+
> |col0 |
> +-+
> | col0|
> | 2015-01-01 |
> +-+
> 2 rows selected (0.257 seconds)
> > select cast(col0 as date) from hive.h1db.testheader ;
> Error: SYSTEM ERROR: IllegalFieldValueException: Value 0 for monthOfYear must
> be in the range [1,12]
> Fragment 0:0
> [Error Id: 34353702-ca27-440b-a4f4-0c9f79fc8ccd on h1.poc.com:31010]
> (org.joda.time.IllegalFieldValueException) Value 0 for monthOfYear must be
> in the range [1,12]
> org.joda.time.field.FieldUtils.verifyValueBounds():236
> org.joda.time.chrono.BasicChronology.getDateMidnightMillis():613
> org.joda.time.chrono.BasicChronology.getDateTimeMillis():159
> org.joda.time.chrono.AssembledChronology.getDateTimeMillis():120
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.memGetDate():261
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.getDate():218
> org.apache.drill.exec.test.generated.ProjectorGen0.doEval():67
> org.apache.drill.exec.test.generated.ProjectorGen0.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():172
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():147
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():79
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():261
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():255
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():255
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
> Also "skip.footer.line.count" should be taken into account.
> If "skip.header.line.count" or "skip.footer.line.count" has incorrect value
> in Hive, throw appropriate exception in Drill.
> Ex: Hive table property skip.header.line.count value 'someValue' is
> non-numeric
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3610) TimestampAdd/Diff (SQL_TSI_) functions
[
https://issues.apache.org/jira/browse/DRILL-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189656#comment-15189656
]
Arina Ielchiieva commented on DRILL-3610:
-
Pending on merge with Calcite latest version.
> TimestampAdd/Diff (SQL_TSI_) functions
> --
>
> Key: DRILL-3610
> URL: https://issues.apache.org/jira/browse/DRILL-3610
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Reporter: Andries Engelbrecht
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Add TimestampAdd and TimestampDiff (SQL_TSI) functions for year, quarter,
> month, week, day, hour, minute, second.
> Examples
> SELECT CAST(TIMESTAMPADD(SQL_TSI_QUARTER,1,Date('2013-03-31'), SQL_DATE) AS
> `column_quarter`
> FROM `table_in`
> HAVING (COUNT(1) > 0)
> SELECT `table_in`.`datetime` AS `column1`,
> `table`.`Key` AS `column_Key`,
> TIMESTAMPDIFF(SQL_TSI_MINUTE,to_timestamp('2004-07-04',
> '-MM-dd'),`table_in`.`datetime`) AS `sum_datediff_minute`
> FROM `calcs`
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (DRILL-4052) Allow variables in drill-override.conf
[ https://issues.apache.org/jira/browse/DRILL-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-4052: --- Assignee: Arina Ielchiieva > Allow variables in drill-override.conf > -- > > Key: DRILL-4052 > URL: https://issues.apache.org/jira/browse/DRILL-4052 > Project: Apache Drill > Issue Type: Improvement > Components: Tools, Build & Test >Affects Versions: 1.1.0 > Environment: All >Reporter: John Omernik >Assignee: Arina Ielchiieva > Labels: features > Fix For: Future > > > Basic variables, starting with "$HOSTNAME" if that simplified things would be > a huge help to managing clusters of drill bits, especially in multi-tenant > environments. > (Note: Perhaps being able to use all ENV variables through a $ENV:ENVVARNAME > would address the issue completely, giving ultimate flexibility). > The use case/story I have is a drill cluster where I am running the > individual drill bits in Marathon on Mesos. Since I am using MapR-FS > (although this could work without MapR through Marathon downloading the > config at run time) I point all drill bits to one config location. This > allows me to make a change in one place, making for easy changes and > configuration management. > For the drill-env.sh, since that is shell script, I found I could easily put > in items like specifying log locations that allowed me to use one shared log > location for all drillbits, and then specifying that each logfile had the > hostname of the drillbit, so I could keep all logs together and thus > troubleshoot much easier. > When it came to drill-override.conf, I wanted to specify spill directory > locations. Specifically, I wanted to use local volumes in MapR-FS. It allows > me to use a clustered filesystem, but have that data only exit on the node > that the drill bit is running on. So, Scheme: maprfs works, but what to > specify for the location? I didn't want to specify the same location, > /tmp/spill as then it would be shared by drillbits, and not guaranteed to be > local to the drill bit (giving up on performance). I wanted to create a > volume that could be mounted /tmp/$hostname/drillspill I could create the > volume in drill-env.sh, but I could not specify that in the drill-override > without creating a separate override file for each node. > Given scaling and multiple users/clusters, I really want to avoid that > situation (a config for each drill bit), hence creating this JIRA for the > ability to use variables in the drill-override. Happy to discuss other use > cases as well. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe database command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Description:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
| SCHEMA_NAME | LOCATION |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
was:
Add describe database command which will return directory associated
with a database on the fly.
Syntax: describe database
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
| SCHEMA_NAME | LOCATION |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
> Add describe database command
> ---
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {noformat}
> DESCRIBE DATABASE xdf.proc;
> +-++
> | SCHEMA_NAME | LOCATION |
> +-++
> | xdf.proc| maprfs://dl.data/processed |
> +-++
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4514) Add describe database command
Arina Ielchiieva created DRILL-4514:
---
Summary: Add describe database command
Key: DRILL-4514
URL: https://issues.apache.org/jira/browse/DRILL-4514
Project: Apache Drill
Issue Type: New Feature
Affects Versions: Future
Reporter: Arina Ielchiieva
Assignee: Arina Ielchiieva
Add describe database command which will return directory associated
with a database on the fly.
Syntax: describe database
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
| SCHEMA_NAME | LOCATION |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4596) Drill should do version check among drillbits
Arina Ielchiieva created DRILL-4596: --- Summary: Drill should do version check among drillbits Key: DRILL-4596 URL: https://issues.apache.org/jira/browse/DRILL-4596 Project: Apache Drill Issue Type: New Feature Affects Versions: 1.6.0 Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Fix For: Future Before registering new drillbit in zookeeper, we should do version check, and make sure all the running drillbits are in the same version. Using drillbits of different version can lead to unexpected results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4596) Drill should do version check among drillbits
[ https://issues.apache.org/jira/browse/DRILL-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233563#comment-15233563 ] Arina Ielchiieva commented on DRILL-4596: - [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], thanks for use case. So implementation approach can be the following: 1. we allow to register any drillbit (disregarding the version) in zookeeper. 2. version compatibility will be guaranteed during release, changes in modules (protobuf, rpc, etc) will influence main version number. 3. when issuing fragment, query to different drillbit, we will check if it's main version is compatible with our drillibit (N + 1). For example, drillbit with version 1.5 can work with 1.5 and 1.6. Drillbit with version 1.6 can work with 1.6 and 1.7 and so on. > Drill should do version check among drillbits > - > > Key: DRILL-4596 > URL: https://issues.apache.org/jira/browse/DRILL-4596 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: Future > > > Before registering new drillbit in zookeeper, we should do version check, and > make sure all the running drillbits are in the same version. > Using drillbits of different version can lead to unexpected results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-3559) Make filename available to sql statments just like dirN
[ https://issues.apache.org/jira/browse/DRILL-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-3559: --- Assignee: Arina Ielchiieva > Make filename available to sql statments just like dirN > --- > > Key: DRILL-3559 > URL: https://issues.apache.org/jira/browse/DRILL-3559 > Project: Apache Drill > Issue Type: Improvement > Components: SQL Parser >Affects Versions: 1.1.0 >Reporter: Stefán Baxter >Assignee: Arina Ielchiieva >Priority: Minor > Fix For: Future > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-3474) Filename should be an available column when querying a directory
[ https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-3474: --- Assignee: Arina Ielchiieva > Filename should be an available column when querying a directory > > > Key: DRILL-3474 > URL: https://issues.apache.org/jira/browse/DRILL-3474 > Project: Apache Drill > Issue Type: Bug > Components: Metadata >Affects Versions: 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva > Fix For: Future > > > I could not find another ticket which talks about this ... > The file name should be a column which can be selected or filtered when > querying a directory just like dir0, dir1 are available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-4596) Drill should do version check among drillbits
[ https://issues.apache.org/jira/browse/DRILL-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233563#comment-15233563 ] Arina Ielchiieva edited comment on DRILL-4596 at 4/11/16 12:55 PM: --- [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], thanks for use case. So implementation approach can be the following: 1. we allow to register any drillbit (disregarding the version). 2. instead of getting available endpoints from coordinator, we'll be receiving compatible endpoints. Version compatibility will be defined using main application version (major + minor (+ 1). For example, drillbit with version 1.8 can work with 1.8 and 1.9. Drillbit with version 1.9 can work with 1.9 and 2.0 and so on. Even during rolling upgrades, compatible endpoints will be able to work together. was (Author: arina): [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], thanks for use case. So implementation approach can be the following: 1. we allow to register any drillbit (disregarding the version). Only drillbits with UNKNOWN version is disallowed. We can get UNKNOWN version only if there is something incorrectly set in manifest file. 2. instead of getting available endpoints from coordinator, we'll be receiving compatible endpoints. Version compatibility will be defined using main application version (major + minor (+ 1). For example, drillbit with version 1.8 can work with 1.8 and 1.9. Drillbit with version 1.9 can work with 1.9 and 2.0 and so on. Even during rolling upgrades, compatible endpoints will be able to work together. > Drill should do version check among drillbits > - > > Key: DRILL-4596 > URL: https://issues.apache.org/jira/browse/DRILL-4596 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: Future > > > Before registering new drillbit in zookeeper, we should do version check, and > make sure all the running drillbits are in the same version. > Using drillbits of different version can lead to unexpected results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-4596) Drill should do version check among drillbits
[ https://issues.apache.org/jira/browse/DRILL-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233563#comment-15233563 ] Arina Ielchiieva edited comment on DRILL-4596 at 4/11/16 11:08 AM: --- [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], thanks for use case. So implementation approach can be the following: 1. we allow to register any drillbit (disregarding the version). Only drillbits with UNKNOWN version is disallowed. We can get UNKNOWN version only if there is something incorrectly set in manifest file. 2. instead of getting available endpoints from coordinator, we'll be receiving compatible endpoints. Version compatibility will be defined using main application version (major + minor (+ 1). For example, drillbit with version 1.8 can work with 1.8 and 1.9. Drillbit with version 1.9 can work with 1.9 and 2.0 and so on. Even during rolling upgrades, compatible endpoints will be able to work together. was (Author: arina): [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], thanks for use case. So implementation approach can be the following: 1. we allow to register any drillbit (disregarding the version) in zookeeper. 2. version compatibility will be guaranteed during release, changes in modules (protobuf, rpc, etc) will influence main version number. 3. when issuing fragment, query to different drillbit, we will check if it's main version is compatible with our drillibit (N + 1). For example, drillbit with version 1.5 can work with 1.5 and 1.6. Drillbit with version 1.6 can work with 1.6 and 1.7 and so on. > Drill should do version check among drillbits > - > > Key: DRILL-4596 > URL: https://issues.apache.org/jira/browse/DRILL-4596 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: Future > > > Before registering new drillbit in zookeeper, we should do version check, and > make sure all the running drillbits are in the same version. > Using drillbits of different version can lead to unexpected results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4585) Add ability to view logs from all drillbits in Web UI
Arina Ielchiieva created DRILL-4585: --- Summary: Add ability to view logs from all drillbits in Web UI Key: DRILL-4585 URL: https://issues.apache.org/jira/browse/DRILL-4585 Project: Apache Drill Issue Type: Improvement Reporter: Arina Ielchiieva Fix For: Future Currently we can only view logs in Web UI from local drillibit. It would be nice, if we could see logs from all active drillbits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4571: Summary: Add link to local Drill logs from the web UI (was: Add link to the Drill log from the web UI) > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Attachments: display_log.JPG, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to the log as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4571: Description: Now we have link to the profile from the web UI. It will be handy for the users to have the link to local logs as well. was: Now we have link to the profile from the web UI. It will be handy for the users to have the link to the log as well. > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Attachments: display_log.JPG, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4571) Add link to local Drill logs from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228626#comment-15228626 ] Arina Ielchiieva commented on DRILL-4571: - Splitting this Jira into two. This Jira will deliver ability to view local logs from Web UI. https://issues.apache.org/jira/browse/DRILL-4585 will add ability to view logs from all drillibits. > Add link to local Drill logs from the web UI > > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Attachments: display_log.JPG, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to local logs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4523) Disallow using loopback address in distributed mode
[
https://issues.apache.org/jira/browse/DRILL-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-4523.
-
Resolution: Fixed
Fixed in 9514cbe.
> Disallow using loopback address in distributed mode
> ---
>
> Key: DRILL-4523
> URL: https://issues.apache.org/jira/browse/DRILL-4523
> Project: Apache Drill
> Issue Type: Improvement
> Components: Server
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.7.0
>
>
> If we enable debug for org.apache.drill.exec.coord.zk in logback.xml, we only
> get the hostname and ports information. For example:
> {code}
> 2015-11-04 19:47:02,927 [ServiceCache-0] DEBUG
> o.a.d.e.c.zk.ZKClusterCoordinator - Cache changed, updating.
> 2015-11-04 19:47:02,932 [ServiceCache-0] DEBUG
> o.a.d.e.c.zk.ZKClusterCoordinator - Active drillbit set changed. Now
> includes 2 total bits. New active drillbits:
> h3.poc.com:31010:31011:31012
> h2.poc.com:31010:31011:31012
> {code}
> We need to know the IP address of each hostname to do further troubleshooting.
> Imagine if any drillbit registers itself as "localhost.localdomain" in
> zookeeper, we will never know where it comes from. Enabling IP address
> tracking can help this case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-4544) Improve error messages for REFRESH TABLE METADATA command
[ https://issues.apache.org/jira/browse/DRILL-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4544. - Resolution: Fixed Merged into master. Commit id - a759552. > Improve error messages for REFRESH TABLE METADATA command > - > > Key: DRILL-4544 > URL: https://issues.apache.org/jira/browse/DRILL-4544 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Minor > Fix For: 1.7.0 > > > Improve the error messages thrown by REFRESH TABLE METADATA command: > In the first case below, the error is maprfs.abc doesn't exist. It should > throw a Object not found or workspace not found. It is currently throwing a > non helpful message; > 0: jdbc:drill:> refresh table metadata maprfs.abc.`my_table`; > + > oksummary > + > false Error: null > + > 1 row selected (0.355 seconds) > In the second case below, it says refresh table metadata is supported only > for single-directory based Parquet tables. But the command works for nested > multi-directory Parquet files. > 0: jdbc:drill:> refresh table metadata maprfs.vnaranammalpuram.`rfm_sales_vw`; > ---+ > oksummary > ---+ > false Table rfm_sales_vw does not support metadata refresh. Support is > currently limited to single-directory-based Parquet tables. > ---+ > 1 row selected (0.418 seconds) > 0: jdbc:drill:> -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4544) Improve error messages for REFRESH TABLE METADATA command
[ https://issues.apache.org/jira/browse/DRILL-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4544: Labels: (was: doc-impacting) > Improve error messages for REFRESH TABLE METADATA command > - > > Key: DRILL-4544 > URL: https://issues.apache.org/jira/browse/DRILL-4544 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Minor > Fix For: 1.7.0 > > > Improve the error messages thrown by REFRESH TABLE METADATA command: > In the first case below, the error is maprfs.abc doesn't exist. It should > throw a Object not found or workspace not found. It is currently throwing a > non helpful message; > 0: jdbc:drill:> refresh table metadata maprfs.abc.`my_table`; > + > oksummary > + > false Error: null > + > 1 row selected (0.355 seconds) > In the second case below, it says refresh table metadata is supported only > for single-directory based Parquet tables. But the command works for nested > multi-directory Parquet files. > 0: jdbc:drill:> refresh table metadata maprfs.vnaranammalpuram.`rfm_sales_vw`; > ---+ > oksummary > ---+ > false Table rfm_sales_vw does not support metadata refresh. Support is > currently limited to single-directory-based Parquet tables. > ---+ > 1 row selected (0.418 seconds) > 0: jdbc:drill:> -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4596) Drill should do version check among drillbits
[ https://issues.apache.org/jira/browse/DRILL-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232774#comment-15232774 ] Arina Ielchiieva commented on DRILL-4596: - [Parth|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parthc], I can't think of use case when customer would like to have drillbits with different versions. If we want to test out new drillibit version, we can just simply set different cluster-id. Though, I agree that it makes sense to check each module for version compatibility. Let's say, compare each module major version only. Does it sound reasonable? > Drill should do version check among drillbits > - > > Key: DRILL-4596 > URL: https://issues.apache.org/jira/browse/DRILL-4596 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: Future > > > Before registering new drillbit in zookeeper, we should do version check, and > make sure all the running drillbits are in the same version. > Using drillbits of different version can lead to unexpected results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197552#comment-15197552 ] Arina Ielchiieva commented on DRILL-3745: - Commid id - dd4f03b. > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > Labels: doc-impacting > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-3745) Hive CHAR not supported
[ https://issues.apache.org/jira/browse/DRILL-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-3745. - Resolution: Fixed Fix Version/s: 1.6.0 > Hive CHAR not supported > --- > > Key: DRILL-3745 > URL: https://issues.apache.org/jira/browse/DRILL-3745 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Nathaniel Auvil >Assignee: Arina Ielchiieva > Labels: doc-impacting > Fix For: 1.6.0 > > > It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? > In Hive: > create table development.foo > ( > bad CHAR(10) > ); > And then in sqlline: > > use `hive.development`; > > select * from foo; > Error: PARSE ERROR: Unsupported Hive data type CHAR. > Following Hive data types are supported in Drill INFORMATION_SCHEMA: > BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, > BINARY, DECIMAL, STRING, VARCHAR, LIST, MAP, STRUCT and UNION > [Error Id: 58bf3940-3c09-4ad2-8f52-d052dffd4b17 on dtpg05:31010] > (state=,code=0) > This was originally found when getting failures trying to connect via JDBS > using Squirrel. We have the Hive plugin enabled with tables using CHAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4523) Add the IP address if we enable debug for org.apache.drill.exec.coord.zk
Arina Ielchiieva created DRILL-4523:
---
Summary: Add the IP address if we enable debug for
org.apache.drill.exec.coord.zk
Key: DRILL-4523
URL: https://issues.apache.org/jira/browse/DRILL-4523
Project: Apache Drill
Issue Type: Improvement
Components: Server
Affects Versions: 1.6.0
Reporter: Arina Ielchiieva
Assignee: Arina Ielchiieva
Fix For: 1.7.0
If we enable debug for org.apache.drill.exec.coord.zk in logback.xml, we only
get the hostname and ports information. For example:
{code}
2015-11-04 19:47:02,927 [ServiceCache-0] DEBUG
o.a.d.e.c.zk.ZKClusterCoordinator - Cache changed, updating.
2015-11-04 19:47:02,932 [ServiceCache-0] DEBUG
o.a.d.e.c.zk.ZKClusterCoordinator - Active drillbit set changed. Now includes
2 total bits. New active drillbits:
h3.poc.com:31010:31011:31012
h2.poc.com:31010:31011:31012
{code}
We need to know the IP address of each hostname to do further troubleshooting.
Imagine if any drillbit registers itself as "localhost.localdomain" in
zookeeper, we will never know where it comes from. Enabling IP address tracking
can help this case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-1706) date_sub function does not accept string as input in Drill
[
https://issues.apache.org/jira/browse/DRILL-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva closed DRILL-1706.
---
Resolution: Fixed
Fix Version/s: (was: Future)
1.2.0
> date_sub function does not accept string as input in Drill
> --
>
> Key: DRILL-1706
> URL: https://issues.apache.org/jira/browse/DRILL-1706
> Project: Apache Drill
> Issue Type: New Feature
> Components: Functions - Drill, Functions - Hive
>Affects Versions: 0.7.0
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: 1.2.0
>
>
> "date_sub" function does not accept string as input in Drill, however it does
> in Hive.
> This different behavior of the function will make customer re-write their
> query to use "cast as date".
> Minimum reproduce :
> {code}
> 0: jdbc:drill:zk=local> select date_sub('2014-11-12 16:45:22',15) from
> dfs.tmp.`drilltest/test.csv` ;
> Query failed: Failure while running fragment., Invalid format: "2014-11-12
> 16:45:22" is malformed at "14-11-12 16:45:22" [
> 9a6f18da-eb1e-4d91-879a-8d9d528efd59 on 10.250.0.115:31010 ]
> (java.lang.IllegalArgumentException) Invalid format: "2014-11-12 16:45:22"
> is malformed at "14-11-12 16:45:22"
> org.joda.time.format.DateTimeFormatter.parseDateTime():873
> org.apache.drill.exec.test.generated.ProjectorGen23.doSetup():63
> org.apache.drill.exec.test.generated.ProjectorGen23.setup():97
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():427
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.buildSchema():270
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema():80
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.buildSchema():95
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():111
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run():249
> ...():0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Comparing to Hive which is good.
> {code}
> 0: jdbc:hive2://n1a:1/default> select date_sub('2014-11-12 16:45:22',15)
> from passwords limit 1 ;
> +-+
> | _c0 |
> +-+
> | 2014-10-28 |
> +-+
> 1 row selected (6.568 seconds)
> {code}
> Workaround in Drill:
> {code}
> 0: jdbc:drill:zk=local> select date_sub(cast('2014-11-12 16:45:22' as
> date),15) from dfs.tmp.`drilltest/test.csv` ;
> ++
> | EXPR$0 |
> ++
> | 2014-10-28 |
> ++
> 1 row selected (0.082 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (DRILL-1706) date_sub function does not accept string as input in Drill
[
https://issues.apache.org/jira/browse/DRILL-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-1706:
---
Assignee: Arina Ielchiieva
> date_sub function does not accept string as input in Drill
> --
>
> Key: DRILL-1706
> URL: https://issues.apache.org/jira/browse/DRILL-1706
> Project: Apache Drill
> Issue Type: New Feature
> Components: Functions - Drill, Functions - Hive
>Affects Versions: 0.7.0
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: 1.2.0
>
>
> "date_sub" function does not accept string as input in Drill, however it does
> in Hive.
> This different behavior of the function will make customer re-write their
> query to use "cast as date".
> Minimum reproduce :
> {code}
> 0: jdbc:drill:zk=local> select date_sub('2014-11-12 16:45:22',15) from
> dfs.tmp.`drilltest/test.csv` ;
> Query failed: Failure while running fragment., Invalid format: "2014-11-12
> 16:45:22" is malformed at "14-11-12 16:45:22" [
> 9a6f18da-eb1e-4d91-879a-8d9d528efd59 on 10.250.0.115:31010 ]
> (java.lang.IllegalArgumentException) Invalid format: "2014-11-12 16:45:22"
> is malformed at "14-11-12 16:45:22"
> org.joda.time.format.DateTimeFormatter.parseDateTime():873
> org.apache.drill.exec.test.generated.ProjectorGen23.doSetup():63
> org.apache.drill.exec.test.generated.ProjectorGen23.setup():97
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():427
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.buildSchema():270
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema():80
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.buildSchema():95
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():111
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run():249
> ...():0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Comparing to Hive which is good.
> {code}
> 0: jdbc:hive2://n1a:1/default> select date_sub('2014-11-12 16:45:22',15)
> from passwords limit 1 ;
> +-+
> | _c0 |
> +-+
> | 2014-10-28 |
> +-+
> 1 row selected (6.568 seconds)
> {code}
> Workaround in Drill:
> {code}
> 0: jdbc:drill:zk=local> select date_sub(cast('2014-11-12 16:45:22' as
> date),15) from dfs.tmp.`drilltest/test.csv` ;
> ++
> | EXPR$0 |
> ++
> | 2014-10-28 |
> ++
> 1 row selected (0.082 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1706) date_sub function does not accept string as input in Drill
[
https://issues.apache.org/jira/browse/DRILL-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206567#comment-15206567
]
Arina Ielchiieva commented on DRILL-1706:
-
Reproduces in Drill 1.1.0.
Starting from 1.2.0 is not reproducible up to 1.6.0.
> date_sub function does not accept string as input in Drill
> --
>
> Key: DRILL-1706
> URL: https://issues.apache.org/jira/browse/DRILL-1706
> Project: Apache Drill
> Issue Type: New Feature
> Components: Functions - Drill, Functions - Hive
>Affects Versions: 0.7.0
>Reporter: Hao Zhu
>Assignee: Arina Ielchiieva
> Fix For: 1.2.0
>
>
> "date_sub" function does not accept string as input in Drill, however it does
> in Hive.
> This different behavior of the function will make customer re-write their
> query to use "cast as date".
> Minimum reproduce :
> {code}
> 0: jdbc:drill:zk=local> select date_sub('2014-11-12 16:45:22',15) from
> dfs.tmp.`drilltest/test.csv` ;
> Query failed: Failure while running fragment., Invalid format: "2014-11-12
> 16:45:22" is malformed at "14-11-12 16:45:22" [
> 9a6f18da-eb1e-4d91-879a-8d9d528efd59 on 10.250.0.115:31010 ]
> (java.lang.IllegalArgumentException) Invalid format: "2014-11-12 16:45:22"
> is malformed at "14-11-12 16:45:22"
> org.joda.time.format.DateTimeFormatter.parseDateTime():873
> org.apache.drill.exec.test.generated.ProjectorGen23.doSetup():63
> org.apache.drill.exec.test.generated.ProjectorGen23.setup():97
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():427
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.buildSchema():270
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema():80
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.buildSchema():95
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():111
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run():249
> ...():0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Comparing to Hive which is good.
> {code}
> 0: jdbc:hive2://n1a:1/default> select date_sub('2014-11-12 16:45:22',15)
> from passwords limit 1 ;
> +-+
> | _c0 |
> +-+
> | 2014-10-28 |
> +-+
> 1 row selected (6.568 seconds)
> {code}
> Workaround in Drill:
> {code}
> 0: jdbc:drill:zk=local> select date_sub(cast('2014-11-12 16:45:22' as
> date),15) from dfs.tmp.`drilltest/test.csv` ;
> ++
> | EXPR$0 |
> ++
> | 2014-10-28 |
> ++
> 1 row selected (0.082 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe schema command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Summary: Add describe schema command (was: Add describe
database command)
> Add describe schema command
> -
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {noformat}
> DESCRIBE SCHEMA xdf.proc;
> +-++
> |name | location |
> +-++
> | xdf.proc| maprfs://dl.data/processed |
> +-++
> {noformat}
> Current implementation covers only dfs schema.
> For all other "" will be returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe database command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Description:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE SCHEMA xdf.proc;
+-++
|name | location |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
was:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
|name | location |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
> Add describe database command
> ---
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {noformat}
> DESCRIBE SCHEMA xdf.proc;
> +-++
> |name | location |
> +-++
> | xdf.proc| maprfs://dl.data/processed |
> +-++
> {noformat}
> Current implementation covers only dfs schema.
> For all other "" will be returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4514) Add describe database command
[
https://issues.apache.org/jira/browse/DRILL-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4514:
Description:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
|name | location |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
Current implementation covers only dfs schema.
For all other "" will be returned.
was:
Add describe database command which will return directory associated
with a database on the fly.
Syntax:
describe database
describe schema
Output:
{noformat}
DESCRIBE DATABASE xdf.proc;
+-++
| SCHEMA_NAME | LOCATION |
+-++
| xdf.proc| maprfs://dl.data/processed |
+-++
{noformat}
> Add describe database command
> ---
>
> Key: DRILL-4514
> URL: https://issues.apache.org/jira/browse/DRILL-4514
> Project: Apache Drill
> Issue Type: New Feature
>Affects Versions: Future
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>
> Add describe database command which will return directory
> associated with a database on the fly.
> Syntax:
> describe database
> describe schema
> Output:
> {noformat}
> DESCRIBE DATABASE xdf.proc;
> +-++
> |name | location |
> +-++
> | xdf.proc| maprfs://dl.data/processed |
> +-++
> {noformat}
> Current implementation covers only dfs schema.
> For all other "" will be returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4544) Improve error messages for REFRESH TABLE METADATA command
Arina Ielchiieva created DRILL-4544: --- Summary: Improve error messages for REFRESH TABLE METADATA command Key: DRILL-4544 URL: https://issues.apache.org/jira/browse/DRILL-4544 Project: Apache Drill Issue Type: Improvement Components: Metadata Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Priority: Minor Fix For: 1.7.0 Improve the error messages thrown by REFRESH TABLE METADATA command: In the first case below, the error is maprfs.abc doesn't exist. It should throw a Object not found or workspace not found. It is currently throwing a non helpful message; 0: jdbc:drill:> refresh table metadata maprfs.abc.`my_table`; + ok summary + false Error: null + 1 row selected (0.355 seconds) In the second case below, it says refresh table metadata is supported only for single-directory based Parquet tables. But the command works for nested multi-directory Parquet files. 0: jdbc:drill:> refresh table metadata maprfs.vnaranammalpuram.`rfm_sales_vw`; ---+ ok summary ---+ false Table rfm_sales_vw does not support metadata refresh. Support is currently limited to single-directory-based Parquet tables. ---+ 1 row selected (0.418 seconds) 0: jdbc:drill:> -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-2799) Query fails if directory contains .DS_Store
[
https://issues.apache.org/jira/browse/DRILL-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-2799:
---
Assignee: Arina Ielchiieva
> Query fails if directory contains .DS_Store
> ---
>
> Key: DRILL-2799
> URL: https://issues.apache.org/jira/browse/DRILL-2799
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Other
>Reporter: Abhishek Girish
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> On accessing a folder, Mac OS X writes .DS_Store (some metadata) into it. See
> http://en.wikipedia.org/wiki/.DS_Store
> When querying such a folder, Drill throws an error. Drill should ignore this
> file. And there should be a way to configure the same.
> {code:sql}
> > select * from dfs.`/data/json/factbook` limit 1;
> Query failed: DATA_READ ERROR: Error reading JSON. - Invalid UTF-32 character
> 0x42756431(above 10) at char #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Removing the file results in successfully querying the directory.
> Log Snippet:
> {code}
> 2015-04-15 11:14:04,256 [2ad15592-eb25-eb7a-5e7b-1c93e68171c6:frag:0:0] ERROR
> o.a.drill.exec.ops.FragmentContext - Fragment Context received failure --
> Fragment: 0:0
> org.apache.drill.common.exceptions.DrillUserException: DATA_READ ERROR: Error
> reading JSON. - Invalid UTF-32 character 0x42756431(above 10) at char
> #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.DrillUserException$Builder.build(DrillUserException.java:115)
> ~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.handleAndRaise(JSONRecordReader.java:171)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:218)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:170)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2799) Query fails if directory contains .DS_Store
[
https://issues.apache.org/jira/browse/DRILL-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva closed DRILL-2799.
---
Resolution: Fixed
Fix Version/s: (was: Future)
1.2.0
> Query fails if directory contains .DS_Store
> ---
>
> Key: DRILL-2799
> URL: https://issues.apache.org/jira/browse/DRILL-2799
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Other
>Reporter: Abhishek Girish
>Assignee: Arina Ielchiieva
> Fix For: 1.2.0
>
>
> On accessing a folder, Mac OS X writes .DS_Store (some metadata) into it. See
> http://en.wikipedia.org/wiki/.DS_Store
> When querying such a folder, Drill throws an error. Drill should ignore this
> file. And there should be a way to configure the same.
> {code:sql}
> > select * from dfs.`/data/json/factbook` limit 1;
> Query failed: DATA_READ ERROR: Error reading JSON. - Invalid UTF-32 character
> 0x42756431(above 10) at char #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Removing the file results in successfully querying the directory.
> Log Snippet:
> {code}
> 2015-04-15 11:14:04,256 [2ad15592-eb25-eb7a-5e7b-1c93e68171c6:frag:0:0] ERROR
> o.a.drill.exec.ops.FragmentContext - Fragment Context received failure --
> Fragment: 0:0
> org.apache.drill.common.exceptions.DrillUserException: DATA_READ ERROR: Error
> reading JSON. - Invalid UTF-32 character 0x42756431(above 10) at char
> #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.DrillUserException$Builder.build(DrillUserException.java:115)
> ~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.handleAndRaise(JSONRecordReader.java:171)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:218)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:170)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2799) Query fails if directory contains .DS_Store
[
https://issues.apache.org/jira/browse/DRILL-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182805#comment-15182805
]
Arina Ielchiieva commented on DRILL-2799:
-
Is not reproducible starting from Drill version 1.2.0 (till 1.5.0).
Jira will be closed with resolution fixed.
> Query fails if directory contains .DS_Store
> ---
>
> Key: DRILL-2799
> URL: https://issues.apache.org/jira/browse/DRILL-2799
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Other
>Reporter: Abhishek Girish
>Assignee: Arina Ielchiieva
> Fix For: 1.2.0
>
>
> On accessing a folder, Mac OS X writes .DS_Store (some metadata) into it. See
> http://en.wikipedia.org/wiki/.DS_Store
> When querying such a folder, Drill throws an error. Drill should ignore this
> file. And there should be a way to configure the same.
> {code:sql}
> > select * from dfs.`/data/json/factbook` limit 1;
> Query failed: DATA_READ ERROR: Error reading JSON. - Invalid UTF-32 character
> 0x42756431(above 10) at char #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Removing the file results in successfully querying the directory.
> Log Snippet:
> {code}
> 2015-04-15 11:14:04,256 [2ad15592-eb25-eb7a-5e7b-1c93e68171c6:frag:0:0] ERROR
> o.a.drill.exec.ops.FragmentContext - Fragment Context received failure --
> Fragment: 0:0
> org.apache.drill.common.exceptions.DrillUserException: DATA_READ ERROR: Error
> reading JSON. - Invalid UTF-32 character 0x42756431(above 10) at char
> #1, byte #7)
> Filename: /data/json/factbook/.DS_Store
> Record: 1
> [f73266e5-3171-4134-a0a8-671af037ddd9 on abhi6.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.DrillUserException$Builder.build(DrillUserException.java:115)
> ~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.handleAndRaise(JSONRecordReader.java:171)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:218)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:170)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4571) Add link to the Drill log from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223997#comment-15223997 ] Arina Ielchiieva commented on DRILL-4571: - I was going to display all logs available in log directory. Whenever you click on any log are interested in, you'll see last N rows (let's say 10K), also you would be able to download full log. Logs will be available per node, the same is done with profiles (you can see only list of queries run from node you are on). > Add link to the Drill log from the web UI > - > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to the log as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4571) Add link to the Drill log from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224102#comment-15224102 ] Arina Ielchiieva commented on DRILL-4571: - Sure, I will add functionality to display logs from all drillbits. It can look like this: you go to Logs tab, there you can switch between inner tabs with cluster names where drillbits are (like we do between query, physical plan) (didn't implement yet so no screenshot) each tab will show list of logs available for drillbit (log_list.JPG), you click on log and see last n lines (display_log.JPG). > Add link to the Drill log from the web UI > - > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Attachments: display_log.JPG, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to the log as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4571) Add link to the Drill log from the web UI
[ https://issues.apache.org/jira/browse/DRILL-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4571: Attachment: log_list.JPG display_log.JPG > Add link to the Drill log from the web UI > - > > Key: DRILL-4571 > URL: https://issues.apache.org/jira/browse/DRILL-4571 > Project: Apache Drill > Issue Type: Improvement >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Attachments: display_log.JPG, log_list.JPG > > > Now we have link to the profile from the web UI. > It will be handy for the users to have the link to the log as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4559) Re-work DESCRIBE command after moving to Calcite master branch
Arina Ielchiieva created DRILL-4559: --- Summary: Re-work DESCRIBE command after moving to Calcite master branch Key: DRILL-4559 URL: https://issues.apache.org/jira/browse/DRILL-4559 Project: Apache Drill Issue Type: Improvement Components: Metadata Reporter: Arina Ielchiieva Fix For: Future Once we move to Calcite master branch which will include DESCRIBE command functionality ([CALCITE-1168|https://issues.apache.org/jira/browse/CALCITE-1168]) we will need to re-work existing DESCRIBE SCHEMA / TABLE commands to work with Calcite changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4571) Add link to the Drill log from the web UI
Arina Ielchiieva created DRILL-4571: --- Summary: Add link to the Drill log from the web UI Key: DRILL-4571 URL: https://issues.apache.org/jira/browse/DRILL-4571 Project: Apache Drill Issue Type: Improvement Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Now we have link to the profile from the web UI. It will be handy for the users to have the link to the log as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-3474) Add implicit file columns support
[ https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264048#comment-15264048 ] Arina Ielchiieva edited comment on DRILL-3474 at 4/29/16 1:31 PM: -- Implementation will add support to four implicit file columns: filename, suffix, fqn, dirname. They will be available during querying file or directory if called explicitly. was (Author: arina): Implementation will add support to four implicit file columns: filename, suffix, dfqn, dirname. They will be available during querying file or directory if called explicitly. > Add implicit file columns support > - > > Key: DRILL-3474 > URL: https://issues.apache.org/jira/browse/DRILL-3474 > Project: Apache Drill > Issue Type: Bug > Components: Metadata >Affects Versions: 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva > Fix For: Future > > > I could not find another ticket which talks about this ... > The file name should be a column which can be selected or filtered when > querying a directory just like dir0, dir1 are available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3474) Add implicit file columns support
[ https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264048#comment-15264048 ] Arina Ielchiieva commented on DRILL-3474: - Implementation will add support to four implicit file columns: filename, suffix, dfqn, dirname. They will be available during querying file or directory if called explicitly. > Add implicit file columns support > - > > Key: DRILL-3474 > URL: https://issues.apache.org/jira/browse/DRILL-3474 > Project: Apache Drill > Issue Type: Bug > Components: Metadata >Affects Versions: 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva > Fix For: Future > > > I could not find another ticket which talks about this ... > The file name should be a column which can be selected or filtered when > querying a directory just like dir0, dir1 are available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3474) Add implicit file columns support
[ https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-3474: Summary: Add implicit file columns support (was: Filename should be an available column when querying a directory) > Add implicit file columns support > - > > Key: DRILL-3474 > URL: https://issues.apache.org/jira/browse/DRILL-3474 > Project: Apache Drill > Issue Type: Bug > Components: Metadata >Affects Versions: 1.1.0 >Reporter: Jim Scott >Assignee: Arina Ielchiieva > Fix For: Future > > > I could not find another ticket which talks about this ... > The file name should be a column which can be selected or filtered when > querying a directory just like dir0, dir1 are available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4604) Generate warning on Web UI if drillbits version mismatch is detected
[ https://issues.apache.org/jira/browse/DRILL-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4604: Attachment: screenshots_with_different_states.docx > Generate warning on Web UI if drillbits version mismatch is detected > > > Key: DRILL-4604 > URL: https://issues.apache.org/jira/browse/DRILL-4604 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > Attachments: index_page.JPG, index_page_mismatch.JPG, > screenshots_with_different_states.docx > > > Display drillbit version on web UI. If any of drillbits version doesn't match > with current drillbit, generate warning. > Screenshots - TBA. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4604) Generate warning on Web UI if drillbits version mismatch is detected
[ https://issues.apache.org/jira/browse/DRILL-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4604: Description: Display drillbit version on web UI. If any of drillbits version doesn't match with current drillbit, generate warning. Screenshots - screenshots_with_different_states.docx. was: Display drillbit version on web UI. If any of drillbits version doesn't match with current drillbit, generate warning. Screenshots - TBA. > Generate warning on Web UI if drillbits version mismatch is detected > > > Key: DRILL-4604 > URL: https://issues.apache.org/jira/browse/DRILL-4604 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva > Fix For: 1.7.0 > > Attachments: index_page.JPG, index_page_mismatch.JPG, > screenshots_with_different_states.docx > > > Display drillbit version on web UI. If any of drillbits version doesn't match > with current drillbit, generate warning. > Screenshots - screenshots_with_different_states.docx. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4640) Unable to submit physical plan from Web UI on Windows
[
https://issues.apache.org/jira/browse/DRILL-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-4640:
Summary: Unable to submit physical plan from Web UI on Windows (was:
Unable to submit physical plan from Web UI on Window)
> Unable to submit physical plan from Web UI on Windows
> -
>
> Key: DRILL-4640
> URL: https://issues.apache.org/jira/browse/DRILL-4640
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.6.0
>Reporter: Arina Ielchiieva
> Fix For: Future
>
>
> Unable to submit physical plan from Web UI on Windows:
> Steps to reproduce:
> 1. Generate physical plan using query:
> explain plan for SELECT * FROM cp.`employee.json` LIMIT 20
> 2. Copy output in json column:
> {noformat}
> { "head" : { "version" : 1, "generator" : { "type" : "ExplainHandler", "info"
> : "" }, "type" : "APACHE_DRILL_PHYSICAL", "options" : [ ], "queue" : 0,
> "resultMode" : "EXEC" }, "graph" : [ { "pop" : "fs-scan", "@id" : 4,
> "userName" : "User", "files" : [ "classpath:/employee.json" ], "storage" : {
> "type" : "file", "enabled" : true, "connection" : "classpath:///", "config" :
> null, "workspaces" : null, "formats" : { "csv" : { "type" : "text",
> "extensions" : [ "csv" ], "delimiter" : "," }, "tsv" : { "type" : "text",
> "extensions" : [ "tsv" ], "delimiter" : "\t" }, "json" : { "type" : "json",
> "extensions" : [ "json" ] }, "parquet" : { "type" : "parquet" }, "avro" : {
> "type" : "avro" }, "csvh" : { "type" : "text", "extensions" : [ "csvh" ],
> "extractHeader" : true, "delimiter" : "," } } }, "format" : { "type" :
> "json", "extensions" : [ "json" ] }, "columns" : [ "`*`" ], "selectionRoot" :
> "classpath:/employee.json", "cost" : 463.0 }, { "pop" : "limit", "@id" : 3,
> "child" : 4, "first" : 0, "last" : 20, "initialAllocation" : 100,
> "maxAllocation" : 100, "cost" : 20.0 }, { "pop" :
> "selection-vector-remover", "@id" : 2, "child" : 3, "initialAllocation" :
> 100, "maxAllocation" : 100, "cost" : 20.0 }, { "pop" : "project",
> "@id" : 1, "exprs" : [ { "ref" : "`*`", "expr" : "`*`" } ], "child" : 2,
> "initialAllocation" : 100, "maxAllocation" : 100, "cost" : 20.0
> }, { "pop" : "screen", "@id" : 0, "child" : 1, "initialAllocation" : 100,
> "maxAllocation" : 100, "cost" : 20.0 } ] }
> {noformat}
> 3. Submit physical plan from Web UI
> Error:
> {noformat}
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> PatternSyntaxException: Unexpected internal error near index 1 \ ^ [Error Id:
> 4310ee06-2e84-4241-9317-553382948718 on 10.2.2.62:31010]
> {noformat}
> Checked in Chrome and Mozilla Firefox.
> Works fine on Linux.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4640) Unable to submit physical plan from Web UI on Window
Arina Ielchiieva created DRILL-4640:
---
Summary: Unable to submit physical plan from Web UI on Window
Key: DRILL-4640
URL: https://issues.apache.org/jira/browse/DRILL-4640
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.6.0
Reporter: Arina Ielchiieva
Fix For: Future
Unable to submit physical plan from Web UI on Windows:
Steps to reproduce:
1. Generate physical plan using query:
explain plan for SELECT * FROM cp.`employee.json` LIMIT 20
2. Copy output in json column:
{ "head" : { "version" : 1, "generator" : { "type" : "ExplainHandler", "info" :
"" }, "type" : "APACHE_DRILL_PHYSICAL", "options" : [ ], "queue" : 0,
"resultMode" : "EXEC" }, "graph" : [ { "pop" : "fs-scan", "@id" : 4, "userName"
: "User", "files" : [ "classpath:/employee.json" ], "storage" : { "type" :
"file", "enabled" : true, "connection" : "classpath:///", "config" : null,
"workspaces" : null, "formats" : { "csv" : { "type" : "text", "extensions" : [
"csv" ], "delimiter" : "," }, "tsv" : { "type" : "text", "extensions" : [ "tsv"
], "delimiter" : "\t" }, "json" : { "type" : "json", "extensions" : [ "json" ]
}, "parquet" : { "type" : "parquet" }, "avro" : { "type" : "avro" }, "csvh" : {
"type" : "text", "extensions" : [ "csvh" ], "extractHeader" : true, "delimiter"
: "," } } }, "format" : { "type" : "json", "extensions" : [ "json" ] },
"columns" : [ "`*`" ], "selectionRoot" : "classpath:/employee.json", "cost" :
463.0 }, { "pop" : "limit", "@id" : 3, "child" : 4, "first" : 0, "last" : 20,
"initialAllocation" : 100, "maxAllocation" : 100, "cost" : 20.0 },
{ "pop" : "selection-vector-remover", "@id" : 2, "child" : 3,
"initialAllocation" : 100, "maxAllocation" : 100, "cost" : 20.0 },
{ "pop" : "project", "@id" : 1, "exprs" : [ { "ref" : "`*`", "expr" : "`*`" }
], "child" : 2, "initialAllocation" : 100, "maxAllocation" : 100,
"cost" : 20.0 }, { "pop" : "screen", "@id" : 0, "child" : 1,
"initialAllocation" : 100, "maxAllocation" : 100, "cost" : 20.0 } ]
}
3. Submit physical plan from Web UI
Error:
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
PatternSyntaxException: Unexpected internal error near index 1 \ ^ [Error Id:
4310ee06-2e84-4241-9317-553382948718 on 10.2.2.62:31010]
Checked in Chrome and Mozilla Firefox.
Works fine on Linux.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4668) Treat dirN columns as implicit columns
Arina Ielchiieva created DRILL-4668: --- Summary: Treat dirN columns as implicit columns Key: DRILL-4668 URL: https://issues.apache.org/jira/browse/DRILL-4668 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.6.0 Reporter: Arina Ielchiieva Fix For: Future Treat dirN columns as implicit ones. Basically they should not appear in select all statement unless called explicitly. Main reason to do this is that returning dirN columns in select all statement actually changes file output representation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
