[jira] [Updated] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sorabh Hamirwasia updated DRILL-5704:
-
Description: 
Reported by [~agirish]

When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}


# $DRILL_HOME/bin/sqlline -u "jdbc:drill:zk=localhost:5181" 
sqlline> select * from sys.drillbits;
User Error Occurred
org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Failed to 
create schema tree.

  was:
Reported by [~agirish]

When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}



> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by 

[jira] [Commented] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113769#comment-16113769
 ] 

Sorabh Hamirwasia commented on DRILL-5704:
--

With this PR the error message on client side now will be as below, but it can 
vary based on FileSystem implementation of liststatus.

0: jdbc:drill:drillbit=10.10.100.164> SELECT * from sys.drillbits;
Error: RESOURCE ERROR: Failed to create schema tree.

_IOException:  Error getting user info for current user, anonymous_

[Error Id: c438d923-21d5-4465-b8bf-0fb4fc00d026 on qa-node164.qa.lab:31010] 
(state=,code=0)

> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by [~agirish]
> When username is not specified then Drill set's the session user as anonymous 
> if impersonation is enabled. During query execution Drill tries to build 
> schema tree and as part of that it validates if the user has access to the 
> workspace or not by using FileClient Api liststatus which verifies the user 
> from the OS user. Since impersonation is only enabled here without 
> authentication and we don't specify any user in connection string, Drill will 
> use default user which is "anonymous" and pass that to check workspace 
> permission which will fail as node doesn't have any valid user with that name.
> {code:java}
> Caused by: java.io.IOException: Error getting user info for current user, 
> anonymous
>..
>..
> at 
> org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
>  [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> ... 10 common frames omitted
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113768#comment-16113768
 ] 

ASF GitHub Bot commented on DRILL-5704:
---

GitHub user sohami opened a pull request:

https://github.com/apache/drill/pull/895

DRILL-5704: Improve error message on client side when queries fail wi…

…th "Failed to create schema tree." when Impersonation is enabled and 
logins are anonymous

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sohami/drill DRILL-5704

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/895.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #895


commit 7950826cb1ae28c33007aac22834ca8f3b33846a
Author: Sorabh Hamirwasia 
Date:   2017-08-04T01:23:02Z

DRILL-5704: Improve error message on client side when queries fail with 
"Failed to create schema tree." when Impersonation is enabled and logins are 
anonymous




> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by [~agirish]
> When username is not specified then Drill set's the session user as anonymous 
> if impersonation is enabled. During query execution Drill tries to build 
> schema tree and as part of that it validates if the user has access to the 
> workspace or not by using FileClient Api liststatus which verifies the user 
> from the OS user. Since impersonation is only enabled here without 
> authentication and we don't specify any user in connection string, Drill will 
> use default user which is "anonymous" and pass that to check workspace 
> permission which will fail as node doesn't have any valid user with that name.
> {code:java}
> Caused by: java.io.IOException: Error getting user info for current user, 
> anonymous
>..
>..
> at 
> org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
>  [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> ... 10 common frames omitted
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sorabh Hamirwasia updated DRILL-5704:
-
Fix Version/s: 1.12.0

> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by [~agirish]
> When username is not specified then Drill set's the session user as anonymous 
> if impersonation is enabled. During query execution Drill tries to build 
> schema tree and as part of that it validates if the user has access to the 
> workspace or not by using FileClient Api liststatus which verifies the user 
> from the OS user. Since impersonation is only enabled here without 
> authentication and we don't specify any user in connection string, Drill will 
> use default user which is "anonymous" and pass that to check workspace 
> permission which will fail as node doesn't have any valid user with that name.
> {code:java}
> Caused by: java.io.IOException: Error getting user info for current user, 
> anonymous
>..
>..
> at 
> org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
>  [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> ... 10 common frames omitted
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sorabh Hamirwasia updated DRILL-5704:
-
Description: 
Reported by [~agirish]

When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}


  was:
When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}



> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by [~agirish]
> When username is not specified then Drill set's the session user as anonymous 
> if impersonation is enabled. During query execution Drill tries to build 
> schema tree and as part of that it validates if the user has access to the 
> workspace or not by using FileClient Api 

[jira] [Assigned] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sorabh Hamirwasia reassigned DRILL-5704:


Assignee: Sorabh Hamirwasia

> Improve error message on client side when queries fail with "Failed to create 
> schema tree." when Impersonation is enabled and logins are anonymous
> --
>
> Key: DRILL-5704
> URL: https://issues.apache.org/jira/browse/DRILL-5704
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
> Fix For: 1.12.0
>
>
> Reported by [~agirish]
> When username is not specified then Drill set's the session user as anonymous 
> if impersonation is enabled. During query execution Drill tries to build 
> schema tree and as part of that it validates if the user has access to the 
> workspace or not by using FileClient Api liststatus which verifies the user 
> from the OS user. Since impersonation is only enabled here without 
> authentication and we don't specify any user in connection string, Drill will 
> use default user which is "anonymous" and pass that to check workspace 
> permission which will fail as node doesn't have any valid user with that name.
> {code:java}
> Caused by: java.io.IOException: Error getting user info for current user, 
> anonymous
>..
>..
> at 
> org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
>  ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> at 
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
>  [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
> ... 10 common frames omitted
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

Sorabh Hamirwasia created DRILL-5704:


 Summary: Improve error message on client side when queries fail 
with "Failed to create schema tree." when Impersonation is enabled and logins 
are anonymous
 Key: DRILL-5704
 URL: https://issues.apache.org/jira/browse/DRILL-5704
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Sorabh Hamirwasia


When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113730#comment-16113730
 ] 

Julian Hyde edited comment on DRILL-5703 at 8/4/17 12:16 AM:
-

Actually, I'm not sure that you need to rebase. I think you need to create a 
branch that just contains the fix to this one issue, and base your pull request 
on that, not your master branch. If there are multiple commits that are all 
required to fix the issue, squash them before you create the pull request. 
Interactive rebase, e.g. {{git rebase -i origin/master}} is a good way to 
squash commits (and other things like re-ordering commits).


was (Author: julianhyde):
Actually, I'm not sure that you need to rebase. I think you need to create a 
branch that just contains the fix to this one issue, and base your pull request 
on that, not your master branch. If there are multiple commits that are all 
required to fix the issue, squash them before you create the pull request. 
Interactive rebase, e.g. `git rebase -i origin/master` is a good way to squash 
commits (and other things like re-ordering commits).

> Add Syntax Highlighting & Autocompletion to Query Form
> --
>
> Key: DRILL-5703
> URL: https://issues.apache.org/jira/browse/DRILL-5703
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.11.0
>Reporter: Charles Givre
>
> The UI could really benefit from having syntax highlighting and 
> autocompletion in the query window as well as the form to update storage 
> plugins.  This PR adds that capability to the query form using the Ace code 
> editor (https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113730#comment-16113730
 ] 

Julian Hyde commented on DRILL-5703:


Actually, I'm not sure that you need to rebase. I think you need to create a 
branch that just contains the fix to this one issue, and base your pull request 
on that, not your master branch. If there are multiple commits that are all 
required to fix the issue, squash them before you create the pull request. 
Interactive rebase, e.g. `git rebase -i origin/master` is a good way to squash 
commits (and other things like re-ordering commits).

> Add Syntax Highlighting & Autocompletion to Query Form
> --
>
> Key: DRILL-5703
> URL: https://issues.apache.org/jira/browse/DRILL-5703
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.11.0
>Reporter: Charles Givre
>
> The UI could really benefit from having syntax highlighting and 
> autocompletion in the query window as well as the form to update storage 
> plugins.  This PR adds that capability to the query form using the Ace code 
> editor (https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Charles Givre (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113709#comment-16113709
 ] 

Charles Givre commented on DRILL-5703:
--

Hi Julian, 
As a relative n00b to git, can you tell me what to do to fix that?  I thought I 
had rebased it before I submitted the PR, but apparently not.  Regarding the 
autocompletion, it uses the Ace code editor, which autocompletes for SQL, but 
not specifically for Drill.  We could tweak that, but I thought something was 
better than what we currently have.   I'm also going to work up one for the 
storage plugin page.
-- C

> Add Syntax Highlighting & Autocompletion to Query Form
> --
>
> Key: DRILL-5703
> URL: https://issues.apache.org/jira/browse/DRILL-5703
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.11.0
>Reporter: Charles Givre
>
> The UI could really benefit from having syntax highlighting and 
> autocompletion in the query window as well as the form to update storage 
> plugins.  This PR adds that capability to the query form using the Ace code 
> editor (https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5701) drill.connections.rpc.. metric not behaving correctly

[Sorabh Hamirwasia (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113626#comment-16113626
 ] 

Sorabh Hamirwasia commented on DRILL-5701:
--

[~karthikm] - Can you please help to review this PR ?

> drill.connections.rpc.. metric not 
> behaving correctly
> ---
>
> Key: DRILL-5701
> URL: https://issues.apache.org/jira/browse/DRILL-5701
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>
> Reported by [~knguyen]
> When sqlline fails to connect to drillbit such as a handshake failure, the 
> drill.connections.rpc.user. metric subtracts 2 
> counters (one for handshake failure and one when the user exits from 
> sqlline). As a result, the metric can end up with a negative value. For 
> control connections if the handshake fails then again counters related to 
> those are updated incorrectly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5701) drill.connections.rpc.. metric not behaving correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113624#comment-16113624
 ] 

ASF GitHub Bot commented on DRILL-5701:
---

GitHub user sohami opened a pull request:

https://github.com/apache/drill/pull/894

DRILL-5701: drill.connections.rpc.. metric not behaving correctly

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sohami/drill DRILL-5701

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/894.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #894


commit e4d84d1f555a8abb383c72bf947c6787ecea055d
Author: Sorabh Hamirwasia 
Date:   2017-08-02T21:25:21Z

DRILL-5701: 
drill.connections.rpc.. metric not 
behaving correctly




> drill.connections.rpc.. metric not 
> behaving correctly
> ---
>
> Key: DRILL-5701
> URL: https://issues.apache.org/jira/browse/DRILL-5701
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>
> Reported by [~knguyen]
> When sqlline fails to connect to drillbit such as a handshake failure, the 
> drill.connections.rpc.user. metric subtracts 2 
> counters (one for handshake failure and one when the user exits from 
> sqlline). As a result, the metric can end up with a negative value. For 
> control connections if the handshake fails then again counters related to 
> those are updated incorrectly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113614#comment-16113614
 ] 

Julian Hyde commented on DRILL-5703:


That's a lot of commits! Looks like you need to rebase your pull request.

I don't know what kind of autocompletion you are supplying, but Calcite has a 
little-known class called SqlAdvisor that enables auto-completion of 
identifiers. For example, if the cursor is in the middle of the SELECT clause, 
it knows what table aliases and columns are available because it understands 
what is in the FROM clause. It even works if the query is mal-formed.

> Add Syntax Highlighting & Autocompletion to Query Form
> --
>
> Key: DRILL-5703
> URL: https://issues.apache.org/jira/browse/DRILL-5703
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.11.0
>Reporter: Charles Givre
>
> The UI could really benefit from having syntax highlighting and 
> autocompletion in the query window as well as the form to update storage 
> plugins.  This PR adds that capability to the query form using the Ace code 
> editor (https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113439#comment-16113439
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/891
  
@arina! I am planning to keep all the files in the local resources file! I 
have mentioned the reasons above. Yes, onload should be used for IE.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113433#comment-16113433
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/891
  
We are trying to load jquery, datatables libraries here. The jquery library 
is already available in the resources which generic page is using. We are 
including generic in the result page. So there is no need to load that again. 
Also, Colvis has been retired, so it's better to keep it in the resources page. 
So, I am changing this to include all of the files in the resources folder and 
checking for external cdn while loading jquery in the generic page.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113431#comment-16113431
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
@sindhurirayavaram, one more question about browser compatibility:
will this work in Chrome, Mozilla, Safari etc?
For example, quick search shows that there are can be issues with different 
browsers when using `link onerror` [1].

[1] 
https://stackoverflow.com/questions/30171270/link-onerror-do-not-work-in-ie


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5699:

Labels:   (was: ready-to-commit)

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113407#comment-16113407
 ] 

ASF GitHub Bot commented on DRILL-5703:
---

GitHub user cgivre opened a pull request:

https://github.com/apache/drill/pull/893

DRILL-5703 Add Syntax Highlighting & Autocompletion to Query Form

This PR (https://issues.apache.org/jira/browse/DRILL-5703) adds syntax 
highlighting and autocompletion to the query form using the Ace code editor.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cgivre/drill master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/893.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #893


commit a5accbd0d9d2defb6de488680b6c650021ae9931
Author: cgivre 
Date:   2017-07-03T03:13:02Z

Added Crypto Functions

commit 35f8002cf95a9376da3bcf930e9b0de585e6c952
Author: cgivre 
Date:   2017-07-03T03:18:39Z

Removed extraneous code

commit 0d71fb56c33e281788f4a8d8b97534132819d5b4
Author: cgivre 
Date:   2017-07-04T02:39:22Z

Merge branch 'master' of https://github.com/apache/drill

commit 58532a552d48a3a8272c6975406e5e948433666f
Author: cgivre 
Date:   2017-07-04T03:41:49Z

Fixed various errors and coding convention, removed helper class.

commit a33e76028508a72018d1034a7268cd271980e5ff
Author: cgivre 
Date:   2017-07-04T03:58:32Z

Fixed spacing and indents

commit ac34c0ecdb5639a2b9dc0d739d495afca78f
Author: cgivre 
Date:   2017-07-04T04:00:28Z

Fixed class names

commit 28a38c57b7f9619046bbd2715cd983b779b29490
Author: cgivre 
Date:   2017-07-04T15:11:09Z

Added Javadoc, MD2, and fixed style errors

commit d19700250e90560136235f6bb182c8dcfcc5974f
Author: cgivre 
Date:   2017-07-04T19:49:47Z

Removed unused workspace variables

commit 386604e257ab73323de86a39ab33c593b84a040f
Author: cgivre 
Date:   2017-07-14T04:17:58Z

Merge branch 'master' of https://github.com/apache/drill

commit 3e449bbbe176956c489a829383c1b3c62632e432
Author: cgivre 
Date:   2017-07-14T04:54:57Z

Fixed misc errata

commit 5d9a8bb12cee671f2372f21f06ae3ef5954b
Author: cgivre 
Date:   2017-08-03T19:50:05Z

Added code editor for query page

commit 4dfb6254903796b3c122fffc2e5193dcc3558dd2
Author: cgivre 
Date:   2017-08-03T20:09:44Z

Removed extra javascript files for code editor

commit f8bf920dc7ba704413e812ffe650e6d46b985201
Author: cgivre 
Date:   2017-08-03T20:12:46Z

Merge branch 'master' of https://github.com/apache/drill




> Add Syntax Highlighting & Autocompletion to Query Form
> --
>
> Key: DRILL-5703
> URL: https://issues.apache.org/jira/browse/DRILL-5703
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.11.0
>Reporter: Charles Givre
>
> The UI could really benefit from having syntax highlighting and 
> autocompletion in the query window as well as the form to update storage 
> plugins.  This PR adds that capability to the query form using the Ace code 
> editor (https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Charles Givre (JIRA)]

Charles Givre created DRILL-5703:


 Summary: Add Syntax Highlighting & Autocompletion to Query Form
 Key: DRILL-5703
 URL: https://issues.apache.org/jira/browse/DRILL-5703
 Project: Apache Drill
  Issue Type: Improvement
  Components: Web Server
Affects Versions: 1.11.0
Reporter: Charles Givre


The UI could really benefit from having syntax highlighting and autocompletion 
in the query window as well as the form to update storage plugins.  This PR 
adds that capability to the query form using the Ace code editor 
(https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5663) Drillbit fails to start when only keystore path is provided without keystore password.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113349#comment-16113349
 ] 

ASF GitHub Bot commented on DRILL-5663:
---

Github user sindhurirayavaram commented on a diff in the pull request:

https://github.com/apache/drill/pull/874#discussion_r131238139
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
@@ -122,10 +122,10 @@
   String HTTP_SESSION_MEMORY_RESERVATION = 
"drill.exec.http.session.memory.reservation";
   String HTTP_SESSION_MEMORY_MAXIMUM = 
"drill.exec.http.session.memory.maximum";
   String HTTP_SESSION_MAX_IDLE_SECS = 
"drill.exec.http.session_max_idle_secs";
-  String HTTP_KEYSTORE_PATH = "javax.net.ssl.keyStore";
--- End diff --

Using the _javax_ property name for Drill property might be confusing for 
users, since changing that value inside Drill configuration will not reflect on 
the actual System property. So to avoid that confusion this property was 
renamed taking example from 
[Hadoop](https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
 and 
[IBM](https://www.ibm.com/support/knowledgecenter/en/SSZH4A_5.0.6/com.ibm.worklight.help.doc/admin/r_ssl_certificate_keystore_setup.html).
 Though Hadoop provides separate parameter for client/server whereas Drill has 
only server side config, these properties can be renamed to include _.server_ 
if needed. But making this name change will not affect Drill users already 
using the older property since backward compatibility is provided here. It will 
work seamlessly for them. All the below scenarios are tested. 
1) Setting javax.ssl.* properties in drill-override.conf
2) Setting javax.ssl.* properties as system options with -D option.
3) Setting ssl.* properties in drill-override.conf

So if a user provide the property values only as system property, then both 
Drill and JSSE will still be consuming it and work as expected.


> Drillbit fails to start when only keystore path is provided without keystore 
> password.
> --
>
> Key: DRILL-5663
> URL: https://issues.apache.org/jira/browse/DRILL-5663
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Sorabh Hamirwasia
>Assignee: Sindhuri Ramanarayan Rayavaram
>
> When we configure keystore path without keystore password inside 
> drill-override.conf for WebServer, then Drillbit fails to start. We should 
> explicitly check for either both being present or both being absent. If any 
> one of them is only present then throw startup exception for Drill.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5660) Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113341#comment-16113341
 ] 

ASF GitHub Bot commented on DRILL-5660:
---

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/877#discussion_r131164453
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/MetadataVersions.java
 ---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import com.google.common.collect.ImmutableSortedSet;
+import org.apache.drill.common.exceptions.DrillRuntimeException;
+
+import java.util.NavigableSet;
+
+/**
+ * Supported metadata versions.
+ * 
+ * String metadata version consists of the following characters:
+ * optional "v" letter,
+ * major metadata version (any number of digits),
+ * optional "." delimiter (used if minor metadata version is 
specified),
+ * minor metadata version (one digit number)
+ * 
+ * Note: keep them synchronized with {@link 
Metadata.ParquetTableMetadataBase} versions
+ */
+public class MetadataVersions {
+  /**
+   * Version 1: Introduces parquet file metadata caching.
+   * See DRILL-2743
+   */
+  public static final String V1 = "v1";
+  /**
+   * Version 2: Metadata cache file size is reduced.
+   * See DRILL-4053
+   */
+  public static final String V2 = "v2";
+  /**
+   * Version 3: Difference between v3 and v2 : min/max, type_length, 
precision, scale, repetitionLevel, definitionLevel.
+   * Filter pushdown for Parquet is implemented. 
+   * See DRILL-1950
+   */
+  public static final String V3 = "v3";
+  /**
+   * Version 3.1: Absolute paths of files and directories are replaced 
with relative ones. Metadata version value
+   * doesn't contain `v` letter
+   * See DRILL-3867, DRILL-5660
+   */
+  public static final String V3_1 = "3.1";
+
+  /**
+   * Helper method to parse string metadata version into float.
+   *
+   * @param stringVersion text metadata version
+   * @return parsed Float metadata version
+   */
+  public static Float parseStringMetadataVersion(String stringVersion) {
+try {
+  if (stringVersion.contains(".") && 
stringVersion.split("\\.")[1].length() != 1) {
+throw new DrillRuntimeException("Minor metadata version shouldn't 
be greater than 9 or contain more than one digit");
+  }
+  return stringVersion.charAt(0) == 'v' ? 
Float.valueOf(stringVersion.substring(1)) : Float.valueOf(stringVersion);
+} catch (Exception e) {
+  throw new DrillRuntimeException(String.format("Could not parse 
metadata version '%s'", stringVersion), e);
+}
+  }
+
+  /**
+   * All historical versions of the Drill metadata cache files
+   */
+  public static final NavigableSet SUPPORTED_VERSIONS = 
ImmutableSortedSet.of(
--- End diff --

Done


> Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867
> 
>
> Key: DRILL-5660
> URL: https://issues.apache.org/jira/browse/DRILL-5660
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Vitalii Diravka
>  Labels: doc-impacting
> Fix For: 1.12.0
>
>
> Drill recently accepted a PR for the following bug:
> DRILL-3867: Store relative paths in metadata file
> This PR turned out to have a flaw which affects version compatibility.
> The DRILL-3867 PR changed the format of Parquet metadata files to store 
> relative paths. All Drill servers after that PR create files with relative 
> paths. But, the version number of the file is unchanged, so that older 
> Drillbits don't know that they can't understand the file.
> Instead, if an older 

[jira] [Created] (DRILL-5702) Jdbc Driver Class not found

[Holger Kiel (JIRA)]

Holger Kiel created DRILL-5702:
--

 Summary: Jdbc Driver Class not found
 Key: DRILL-5702
 URL: https://issues.apache.org/jira/browse/DRILL-5702
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.11.0
Reporter: Holger Kiel
Priority: Critical


Cannot connect to drill cluster after upgrade to new Jar 
drill-jdbc-all-1.11.0.jar. When replacing Jar file with older release 
drill-jdbc-all-1.10.0.jar, connection works again. Tested with various client 
applications:

{code}
java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
${package.namespace.prefix}org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
 not found
at 
oadd.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
at oadd.org.apache.hadoop.security.Groups.(Groups.java:80)
at oadd.org.apache.hadoop.security.Groups.(Groups.java:74)
at 
oadd.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
at 
oadd.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)
at 
oadd.org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
at 
oadd.org.apache.drill.exec.rpc.security.plain.PlainFactory.createAndLoginUser(PlainFactory.java:63)
at 
oadd.org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:244)
at 
oadd.org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:171)
at 
oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
at 
oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:158)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
oadd.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
{code}

Workaround is using the old driver version.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-3665) Deadlock while executing CTAS that runs out of memory

[Roman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman resolved DRILL-3665.
--
   Resolution: Duplicate
Fix Version/s: (was: Future)
   1.11.0

> Deadlock while executing CTAS that runs out of memory
> -
>
> Key: DRILL-3665
> URL: https://issues.apache.org/jira/browse/DRILL-3665
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Roman
>Priority: Critical
> Fix For: 1.11.0
>
> Attachments: drillbit.log.drill-3665, jstack.txt
>
>
> I had a query running out of memory during CTAS and after that drillbit was 
> rendered unusable:
> {code}
> 0: jdbc:drill:schema=dfs> create table lineitem as select
> . . . . . . . . . . . . > cast(columns[0] as int) l_orderkey,
> . . . . . . . . . . . . > cast(columns[1] as int) l_partkey,
> . . . . . . . . . . . . > cast(columns[2] as int) l_suppkey,
> . . . . . . . . . . . . > cast(columns[3] as int) l_linenumber,
> . . . . . . . . . . . . > cast(columns[4] as double) l_quantity,
> . . . . . . . . . . . . > cast(columns[5] as double) l_extendedprice,
> . . . . . . . . . . . . > cast(columns[6] as double) l_discount,
> . . . . . . . . . . . . > cast(columns[7] as double) l_tax,
> . . . . . . . . . . . . > cast(columns[8] as varchar(200)) l_returnflag,
> . . . . . . . . . . . . > cast(columns[9] as varchar(200)) l_linestatus,
> . . . . . . . . . . . . > cast(columns[10] as date) l_shipdate,
> . . . . . . . . . . . . > cast(columns[11] as date) l_commitdate,
> . . . . . . . . . . . . > cast(columns[12] as date) l_receiptdate,
> . . . . . . . . . . . . > cast(columns[13] as varchar(200)) 
> l_shipinstruct,
> . . . . . . . . . . . . > cast(columns[14] as varchar(200)) l_shipmode,
> . . . . . . . . . . . . > cast(columns[15] as varchar(200)) l_comment
> . . . . . . . . . . . . > from `lineitem.dat`;
> Error: RESOURCE ERROR: One or more nodes ran out of memory while executing 
> the query.
> Fragment 1:10
> [Error Id: 11084315-5388-4500-b165-642a5f595ebf on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> Here is drill's behavior after that:
> 1. Tried to run: "select * from sys.options" in the same sqlline session - 
> hangs.
> 2. Was able to start sqlline and connect to drillbit:
> - If you try running anything on this connection: it hangs.
> - Issue ^C --> you will get result if you are lucky (these queries 
> will appear as: "CANCELLATION_REQUESTED" on WebUI)
>   (I only tried querying sys.memory, sys.options which possibly have 
> a different code path than queries from actual user data)
> - If you are not lucky, you will get this error below:
> {code}
> 0: jdbc:drill:schema=dfs> show files;
> java.lang.RuntimeException: java.sql.SQLException: Unexpected 
> RuntimeException: java.lang.IllegalArgumentException: Buffer has negative 
> reference count.
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> or maybe something like this:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from nation group by n_regionkey;
> Error: CONNECTION ERROR: Exceeded timeout (5000) while waiting send 
> intermediate work fragments to remote nodes. Sent 1 and only heard response 
> back from 0 nodes.
> [Error Id: 6abce8e9-78a1-4b3d-bcec-503930482b40 on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> I'm attaching results of a jstack  and drillbit.log and so far I was not able 
> to reproduce this problem again (working on it).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5663) Drillbit fails to start when only keystore path is provided without keystore password.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113187#comment-16113187
 ] 

ASF GitHub Bot commented on DRILL-5663:
---

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/874
  
Hadoop uses different property names for configuring ssl properties. We can 
provide backward compatibility where we can still use the java properties in 
drill-override.conf.


> Drillbit fails to start when only keystore path is provided without keystore 
> password.
> --
>
> Key: DRILL-5663
> URL: https://issues.apache.org/jira/browse/DRILL-5663
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Sorabh Hamirwasia
>Assignee: Sindhuri Ramanarayan Rayavaram
>
> When we configure keystore path without keystore password inside 
> drill-override.conf for WebServer, then Drillbit fails to start. We should 
> explicitly check for either both being present or both being absent. If any 
> one of them is only present then throw startup exception for Drill.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-3665) Deadlock while executing CTAS that runs out of memory

[Roman (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113185#comment-16113185
 ] 

Roman commented on DRILL-3665:
--

I tried different ways to reproduce this issue on a drill a7e298760f9c9e 
(before DRILL-5599 fix) with different memory setting and different size of 
tables, but I can get the only drillbit down:

{code:xml}
Error: CONNECTION ERROR: Connection /192.168.121.7:52656 <--> 
node1/192.168.121.7:31010 (user client) closed unexpectedly. Drillbit down?


[Error Id: acc425f5-eb8c-4028-84fe-ac649cf36bee ] (state=,code=0)
{code}

But this error is not reproducible after DRILL-5599 fix. 

I used 2 queries:

{code:title=Query 1|borderStyle=solid}
create table lineitema as select
cast(columns[0] as int) l_orderkey,
cast(columns[1] as int) l_partkey,
cast(columns[2] as int) l_suppkey,
cast(columns[3] as int) l_linenumber,
cast(columns[4] as double) l_quantity,
cast(columns[5] as double) l_extendedprice,
cast(columns[6] as double) l_discount,
cast(columns[7] as double) l_tax,
cast(columns[8] as varchar(200)) l_returnflag,
cast(columns[9] as varchar(200)) l_linestatus,
cast(columns[10] as date) l_shipdate,
cast(columns[11] as date) l_commitdate,
cast(columns[12] as date) l_receiptdate,
cast(columns[13] as varchar(200)) l_shipinstruct,
cast(columns[14] as varchar(200)) l_shipmode,
cast(columns[15] as varchar(200)) l_comment
from `lineitembig.dat`;
{code}
{code:title=Query 2|borderStyle=solid}
create table lineitem as select
a.l_orderkey,
a.l_partkey,
a.l_suppkey,
a.l_linenumber,
a.l_quantity,
a.l_extendedprice,
a.l_discount,
a.l_tax,
a.l_returnflag,
a.l_linestatus,
a.l_shipdate,
a.l_commitdate,
a.l_receiptdate,
a.l_shipinstruct,
b.l_shipmode,
b.l_comment
from lineitema a
INNER JOIN
lineitemb b 
ON a.l_orderkey = b.l_orderkey; 
{code}

It seems we can not get the original reproduce on Drill 1.11, but this 
"CONNECTION ERROR" is related to this ticket. So closed as a duplicate of 
DRILL-5599.

> Deadlock while executing CTAS that runs out of memory
> -
>
> Key: DRILL-3665
> URL: https://issues.apache.org/jira/browse/DRILL-3665
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Roman
>Priority: Critical
> Fix For: Future
>
> Attachments: drillbit.log.drill-3665, jstack.txt
>
>
> I had a query running out of memory during CTAS and after that drillbit was 
> rendered unusable:
> {code}
> 0: jdbc:drill:schema=dfs> create table lineitem as select
> . . . . . . . . . . . . > cast(columns[0] as int) l_orderkey,
> . . . . . . . . . . . . > cast(columns[1] as int) l_partkey,
> . . . . . . . . . . . . > cast(columns[2] as int) l_suppkey,
> . . . . . . . . . . . . > cast(columns[3] as int) l_linenumber,
> . . . . . . . . . . . . > cast(columns[4] as double) l_quantity,
> . . . . . . . . . . . . > cast(columns[5] as double) l_extendedprice,
> . . . . . . . . . . . . > cast(columns[6] as double) l_discount,
> . . . . . . . . . . . . > cast(columns[7] as double) l_tax,
> . . . . . . . . . . . . > cast(columns[8] as varchar(200)) l_returnflag,
> . . . . . . . . . . . . > cast(columns[9] as varchar(200)) l_linestatus,
> . . . . . . . . . . . . > cast(columns[10] as date) l_shipdate,
> . . . . . . . . . . . . > cast(columns[11] as date) l_commitdate,
> . . . . . . . . . . . . > cast(columns[12] as date) l_receiptdate,
> . . . . . . . . . . . . > cast(columns[13] as varchar(200)) 
> l_shipinstruct,
> . . . . . . . . . . . . > cast(columns[14] as varchar(200)) l_shipmode,
> . . . . . . . . . . . . > cast(columns[15] as varchar(200)) l_comment
> . . . . . . . . . . . . > from `lineitem.dat`;
> Error: RESOURCE ERROR: One or more nodes ran out of memory while executing 
> the query.
> Fragment 1:10
> [Error Id: 11084315-5388-4500-b165-642a5f595ebf on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> Here is drill's behavior after that:
> 1. Tried to run: "select * from sys.options" in the same sqlline session - 
> hangs.
> 2. Was able to start sqlline and connect to drillbit:
> - If you try running anything on this connection: it hangs.
> - Issue ^C --> you will get result if you are lucky (these queries 
> will appear as: "CANCELLATION_REQUESTED" on WebUI)
>   (I only tried querying sys.memory, sys.options which possibly have 
> a different code path than queries from actual user data)
> - If you are not lucky, you will get this error below:
> {code}
> 0: jdbc:drill:schema=dfs> show files;
> java.lang.RuntimeException: 

[jira] [Assigned] (DRILL-3665) Deadlock while executing CTAS that runs out of memory

[Roman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman reassigned DRILL-3665:


Assignee: Roman  (was: Deneche A. Hakim)

> Deadlock while executing CTAS that runs out of memory
> -
>
> Key: DRILL-3665
> URL: https://issues.apache.org/jira/browse/DRILL-3665
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Roman
>Priority: Critical
> Fix For: Future
>
> Attachments: drillbit.log.drill-3665, jstack.txt
>
>
> I had a query running out of memory during CTAS and after that drillbit was 
> rendered unusable:
> {code}
> 0: jdbc:drill:schema=dfs> create table lineitem as select
> . . . . . . . . . . . . > cast(columns[0] as int) l_orderkey,
> . . . . . . . . . . . . > cast(columns[1] as int) l_partkey,
> . . . . . . . . . . . . > cast(columns[2] as int) l_suppkey,
> . . . . . . . . . . . . > cast(columns[3] as int) l_linenumber,
> . . . . . . . . . . . . > cast(columns[4] as double) l_quantity,
> . . . . . . . . . . . . > cast(columns[5] as double) l_extendedprice,
> . . . . . . . . . . . . > cast(columns[6] as double) l_discount,
> . . . . . . . . . . . . > cast(columns[7] as double) l_tax,
> . . . . . . . . . . . . > cast(columns[8] as varchar(200)) l_returnflag,
> . . . . . . . . . . . . > cast(columns[9] as varchar(200)) l_linestatus,
> . . . . . . . . . . . . > cast(columns[10] as date) l_shipdate,
> . . . . . . . . . . . . > cast(columns[11] as date) l_commitdate,
> . . . . . . . . . . . . > cast(columns[12] as date) l_receiptdate,
> . . . . . . . . . . . . > cast(columns[13] as varchar(200)) 
> l_shipinstruct,
> . . . . . . . . . . . . > cast(columns[14] as varchar(200)) l_shipmode,
> . . . . . . . . . . . . > cast(columns[15] as varchar(200)) l_comment
> . . . . . . . . . . . . > from `lineitem.dat`;
> Error: RESOURCE ERROR: One or more nodes ran out of memory while executing 
> the query.
> Fragment 1:10
> [Error Id: 11084315-5388-4500-b165-642a5f595ebf on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> Here is drill's behavior after that:
> 1. Tried to run: "select * from sys.options" in the same sqlline session - 
> hangs.
> 2. Was able to start sqlline and connect to drillbit:
> - If you try running anything on this connection: it hangs.
> - Issue ^C --> you will get result if you are lucky (these queries 
> will appear as: "CANCELLATION_REQUESTED" on WebUI)
>   (I only tried querying sys.memory, sys.options which possibly have 
> a different code path than queries from actual user data)
> - If you are not lucky, you will get this error below:
> {code}
> 0: jdbc:drill:schema=dfs> show files;
> java.lang.RuntimeException: java.sql.SQLException: Unexpected 
> RuntimeException: java.lang.IllegalArgumentException: Buffer has negative 
> reference count.
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> or maybe something like this:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from nation group by n_regionkey;
> Error: CONNECTION ERROR: Exceeded timeout (5000) while waiting send 
> intermediate work fragments to remote nodes. Sent 1 and only heard response 
> back from 0 nodes.
> [Error Id: 6abce8e9-78a1-4b3d-bcec-503930482b40 on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> I'm attaching results of a jstack  and drillbit.log and so far I was not able 
> to reproduce this problem again (working on it).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-1162) 25 way join ended up with OOM

[Volodymyr Vysotskyi (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113039#comment-16113039
 ] 

Volodymyr Vysotskyi commented on DRILL-1162:


Thanks for the explanation of background about DRILL-2236. I have no doubt that 
this fix was required before Drill started using {{HepPlanner}}. 
Considering only the case when tables are joining on PK, I agree with you that 
DRILL-2236 would set smaller input for the build side since 
{{LoptOptimizeJoinRule}} considers not only rows count but also columns count, 
if the rows count is the same 
([swapInputs()|https://github.com/mapr/incubator-calcite/blob/DrillCalcite1.4.0-mapr-1.4.0/core/src/main/java/org/apache/calcite/rel/rules/LoptOptimizeJoinRule.java#L1857]
 method). So in the case when left and right inputs have the same rows count, 
after applying this rule input with the join will be on the probe side, since 
join operator has usually larger columns count. But real rows count is not 
greater than estimated for joining on PK, so we have benefit when the fix for 
DRILL-2236 swaps join inputs after applying {{LoptOptimizeJoinRule}}.

Unfortunately, for the case of non-PK join condition and when estimated rows 
count is the same, this swap makes build side larger.

Disabling the hashjoin_swap would help only for the queries like in this Jira 
(join inputs have the same rows count). 
For example, with these tables:
{code:sql}
create table p as select * from `lineitem1.parquet` limit 60040;
create table o as select * from `lineitem1.parquet` limit 50030;
create table n as select * from `lineitem1.parquet` limit 40020;
create table m as select * from `lineitem1.parquet` limit 30010;
create table l as select * from `lineitem1.parquet` limit 2;
create table k as select * from `lineitem1.parquet` limit 1;
{code}
query 
{code:sql}
select count(*) from k
inner join l on k.l_partkey=l.l_partkey
inner join m on m.l_partkey=l.l_partkey
inner join n on m.l_partkey=n.l_partkey
inner join o on n.l_partkey=o.l_partkey
inner join p on p.l_partkey=o.l_partkey;
{code}
also fails with OOM when hashjoin_swap is disabled or enabled.

I agree with you that the real problem is the join row count estimation.
Hive, for example, estimates join row count considering both PK join and non-PK 
join cases: 
[getRowCount()|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java#L65].
 It determines the case and calculates row count depending on it. 
I realize that we could not use the similar code since Drill does not have 
table and column statistics. In DRILL-1328 table statistics will be 
implemented, but it may cause inconsistent behaviour, since this statistics may 
be not created for all tables, or for a single table from the query with a 
join, or a file with statistics may be corrupted. 

Hash join spilling to the disk should also help to omit this OOM even with such 
plan.

> 25 way join ended up with OOM
> -
>
> Key: DRILL-1162
> URL: https://issues.apache.org/jira/browse/DRILL-1162
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow, Query Planning & Optimization
>Reporter: Rahul Challapalli
>Assignee: Volodymyr Vysotskyi
>Priority: Critical
> Fix For: Future
>
> Attachments: error.log, oom_error.log
>
>
> git.commit.id.abbrev=e5c2da0
> The below query results in 0 results being returned 
> {code:sql}
> select count(*) from `lineitem1.parquet` a 
> inner join `part.parquet` j on a.l_partkey = j.p_partkey 
> inner join `orders.parquet` k on a.l_orderkey = k.o_orderkey 
> inner join `supplier.parquet` l on a.l_suppkey = l.s_suppkey 
> inner join `partsupp.parquet` m on j.p_partkey = m.ps_partkey and l.s_suppkey 
> = m.ps_suppkey 
> inner join `customer.parquet` n on k.o_custkey = n.c_custkey 
> inner join `lineitem2.parquet` b on a.l_orderkey = b.l_orderkey 
> inner join `lineitem2.parquet` c on a.l_partkey = c.l_partkey 
> inner join `lineitem2.parquet` d on a.l_suppkey = d.l_suppkey 
> inner join `lineitem2.parquet` e on a.l_extendedprice = e.l_extendedprice 
> inner join `lineitem2.parquet` f on a.l_comment = f.l_comment 
> inner join `lineitem2.parquet` g on a.l_shipdate = g.l_shipdate 
> inner join `lineitem2.parquet` h on a.l_commitdate = h.l_commitdate 
> inner join `lineitem2.parquet` i on a.l_receiptdate = i.l_receiptdate 
> inner join `lineitem2.parquet` o on a.l_receiptdate = o.l_receiptdate 
> inner join `lineitem2.parquet` p on a.l_receiptdate = p.l_receiptdate 
> inner join `lineitem2.parquet` q on a.l_receiptdate = q.l_receiptdate 
> inner join `lineitem2.parquet` r on a.l_receiptdate = r.l_receiptdate 
> inner join `lineitem2.parquet` s on a.l_receiptdate = s.l_receiptdate 
> inner join `lineitem2.parquet` t on 

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112875#comment-16112875
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
Regarding build failure, it will fail only if we did not download these 
files before. If we did, it will take them from the cache:
`[INFO] --- download-maven-plugin:1.3.0:wget 
(add-datatables-css-as-resource) @ drill-java-exec ---
[INFO] Got from cache: 
.m2\repository\.cache\download-maven-plugin\dataTables.colVis.css_050ee51b8fd84b79ae0240ce840501c6
`
Basically we need internet only when we build the project for the first 
time, the next time having internet access is not necessary,  So I guess we are 
fine here.

+1, LGTM.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112856#comment-16112856
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
Actually you are right, there are benefits of using CDN first [1].

[1] 
https://stackoverflow.com/questions/1014203/best-way-to-use-googles-hosted-jquery-but-fall-back-to-my-hosted-library-on-go


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-5699:
---

Assignee: Sindhuri Ramanarayan Rayavaram  (was: Arina Ielchiieva)

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-5699:
---

Assignee: Arina Ielchiieva  (was: Sindhuri Ramanarayan Rayavaram)

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5699:

Reviewer: Arina Ielchiieva

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5699:

External issue ID:   (was: DRILL-5054)

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5699:

Fix Version/s: 1.12.0

> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
> Fix For: 1.12.0
>
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112824#comment-16112824
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
> Even if we have local files it's better to check the browser's cache 
first. If that fails, we can load them locally.
But why? javascript and css files have stable versions. If we download them 
during build time, they won't change after that.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112810#comment-16112810
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/891
  
@arina-ielchiieva Thanks for the comments! Yes drillbit fails if it can't 
download them during build time. These are pretty large fails. Is it good to 
add them in the source tree? 
To answer your second question! Even if we have local files it's better to 
check the browser's cache first. If that fails, we can load them locally.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112801#comment-16112801
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
I have checked and build will fail: `Failed to execute goal 
com.googlecode.maven-download-plugin:download-maven-plugin:1.3.0:wget 
(add-jquery-as-resource) on project drill-java-exec: IO Error: Could not get 
content -> [Help 1]`, so I guess we just can refer to static resources.


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5699) Drill Web UI Page Source Has Links To External Sites

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112712#comment-16112712
 ] 

ASF GitHub Bot commented on DRILL-5699:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
@sindhurirayavaram, thanks for the PR! I have two questions though:
1. Will Drill build fail if we could not download these the javascript and 
css files during build time?
2. If the answer to the first question is no than why we try to download 
the javascript and css files from the internet if we have already loaded them 
locally when loading the result page?


> Drill Web UI Page Source Has Links To External Sites
> 
>
> Key: DRILL-5699
> URL: https://issues.apache.org/jira/browse/DRILL-5699
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Reporter: Sindhuri Ramanarayan Rayavaram
>Assignee: Sindhuri Ramanarayan Rayavaram
>Priority: Minor
>
> Drill uses external CDN for javascript and css files in the result page. When 
> there is no internet connection this page fails to load. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5660) Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112542#comment-16112542
 ] 

ASF GitHub Bot commented on DRILL-5660:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/877
  
@vdiravka 
limiting minor version to 9 is breaking the whole concept of major and 
minor versions usage. Since when we reach minor max we'll need to bump up the 
major though structure changes did not occur.
If using float is not working for major and minor version (I believe you 
having trouble comparing 1.1 and 1.10) then let's leave our `MetadataVersion` 
class, just get rid of `v` in version name.



> Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867
> 
>
> Key: DRILL-5660
> URL: https://issues.apache.org/jira/browse/DRILL-5660
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Vitalii Diravka
>  Labels: doc-impacting
> Fix For: 1.12.0
>
>
> Drill recently accepted a PR for the following bug:
> DRILL-3867: Store relative paths in metadata file
> This PR turned out to have a flaw which affects version compatibility.
> The DRILL-3867 PR changed the format of Parquet metadata files to store 
> relative paths. All Drill servers after that PR create files with relative 
> paths. But, the version number of the file is unchanged, so that older 
> Drillbits don't know that they can't understand the file.
> Instead, if an older Drill tries to read the file, queries fail left and 
> right. Drill will resolve the paths, but does so relative to the user's HDFS 
> home directory, which is wrong.
> What should have happened is that we should have bumped the parquet metadata 
> file version number so that older Drillbits can’t read the file. This ticket 
> requests that we do that.
> Now, one could argue that the lack of version number change is fine. Once a 
> user upgrades Drill, they won't use an old Drillbit. But, things are not that 
> simple:
> * A developer tests a branch based on a pre-DRILL-3867 build on a cluster in 
> which metadata files have been created by a post-DRILL-3867 build. (This has 
> already occurred multiple times in our shop.)
> * A user tries to upgrade to Drill 1.11, finds an issue, and needs to roll 
> back to Drill 1.10. Doing so will cause queries to fail due to 
> seemingly-corrupt metadata files.
> * A user tries to do a rolling upgrade: running 1.11 on some servers, 1.10 on 
> others. Once a 1.11 server is installed, the metadata is updated ("corrupted" 
> from the perspective of 1.10) and queries fail.
> Standard practice in this scenario is to:
> * Bump the file version number when the file format changes, and
> * Software refuses to read files with a version newer than the software was 
> designed for.
> Of course, it is highly desirable that newer servers read old files, but that 
> is not the issue here.
> *Main technical points of working of parquet metadata caching for now.*
> Only process of reading the parquet metadata is changed (the process of 
> writing isn't changed):
> +1. Metadata files are valid:+
> Metadata objects are created by deserialization of parquet metadata files in 
> the process of creating ParquetGroupScan physical operator. 
> All supported versions are stored in the "MetadataVersion.Constants" class 
> and in the Jackson annotations for Metadata.ParquetTableMetadataBase class.
> +2. Metadata files version isn't supported (created by newer Drill version). 
> Drill table has at least one metadata file of unsupported version:+
> JsonMappingException is obtained and swallowed without creating metadata 
> object. Error message is logged. The state is stored in MetadataContext, 
> therefore further there will be no attempt to deserialize metadata file again 
> in context of performing current query. The physical plan will be created 
> without using parquet metadata caching. Warning message is logged for every 
> further check "is metadata corrupted".
> +3. Drill table has at least one corrupted metadata file, which can't be 
> deserialized:+
> JsonParseException is obtained. Then the same behaviour as for the 
> unsupported version files.
> +4. The metadata file was removed by other process:+
> FileNotFound is obtained. Then the same behaviour as for the unsupported 
> version files.
> The new versions of metadata should be added in such manner:
> 1. Increasing of the metadata major version if metadata structure is changed.
> 2. Increasing of the metadata minor version if only metadata content is 
> changed, but metadata structure is the same.
> For the first case a new metadata structure (class) should be created 
> (possible an improvement of 

[jira] [Commented] (DRILL-5660) Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112540#comment-16112540
 ] 

ASF GitHub Bot commented on DRILL-5660:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/877#discussion_r131101557
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/MetadataVersions.java
 ---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import com.google.common.collect.ImmutableSortedSet;
+import org.apache.drill.common.exceptions.DrillRuntimeException;
+
+import java.util.NavigableSet;
+
+/**
+ * Supported metadata versions.
+ * 
+ * String metadata version consists of the following characters:
+ * optional "v" letter,
+ * major metadata version (any number of digits),
+ * optional "." delimiter (used if minor metadata version is 
specified),
+ * minor metadata version (one digit number)
+ * 
+ * Note: keep them synchronized with {@link 
Metadata.ParquetTableMetadataBase} versions
+ */
+public class MetadataVersions {
+  /**
+   * Version 1: Introduces parquet file metadata caching.
+   * See DRILL-2743
+   */
+  public static final String V1 = "v1";
+  /**
+   * Version 2: Metadata cache file size is reduced.
+   * See DRILL-4053
+   */
+  public static final String V2 = "v2";
+  /**
+   * Version 3: Difference between v3 and v2 : min/max, type_length, 
precision, scale, repetitionLevel, definitionLevel.
+   * Filter pushdown for Parquet is implemented. 
+   * See DRILL-1950
+   */
+  public static final String V3 = "v3";
+  /**
+   * Version 3.1: Absolute paths of files and directories are replaced 
with relative ones. Metadata version value
+   * doesn't contain `v` letter
+   * See DRILL-3867, DRILL-5660
+   */
+  public static final String V3_1 = "3.1";
+
+  /**
+   * Helper method to parse string metadata version into float.
+   *
+   * @param stringVersion text metadata version
+   * @return parsed Float metadata version
+   */
+  public static Float parseStringMetadataVersion(String stringVersion) {
+try {
+  if (stringVersion.contains(".") && 
stringVersion.split("\\.")[1].length() != 1) {
+throw new DrillRuntimeException("Minor metadata version shouldn't 
be greater than 9 or contain more than one digit");
+  }
+  return stringVersion.charAt(0) == 'v' ? 
Float.valueOf(stringVersion.substring(1)) : Float.valueOf(stringVersion);
+} catch (Exception e) {
+  throw new DrillRuntimeException(String.format("Could not parse 
metadata version '%s'", stringVersion), e);
+}
+  }
+
+  /**
+   * All historical versions of the Drill metadata cache files
+   */
+  public static final NavigableSet SUPPORTED_VERSIONS = 
ImmutableSortedSet.of(
--- End diff --

You can use `SortedSet`, more abstract implementation.


> Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867
> 
>
> Key: DRILL-5660
> URL: https://issues.apache.org/jira/browse/DRILL-5660
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Vitalii Diravka
>  Labels: doc-impacting
> Fix For: 1.12.0
>
>
> Drill recently accepted a PR for the following bug:
> DRILL-3867: Store relative paths in metadata file
> This PR turned out to have a flaw which affects version compatibility.
> The DRILL-3867 PR changed the format of Parquet metadata files to store 
> relative paths. All Drill servers after that PR create files with relative 
> paths. But, the version number of the file is unchanged, so that older 
> Drillbits don't know