[jira] [Updated] (DRILL-5324) Provide simplified column reader/writer for use in tests

[Paul Rogers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Rogers updated DRILL-5324:
---
Labels: ready-to-commit  (was: )

> Provide simplified column reader/writer for use in tests
> 
>
> Key: DRILL-5324
> URL: https://issues.apache.org/jira/browse/DRILL-5324
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> In support of DRILL-5323, we wish to provide a very easy way to work with row 
> sets. See the comment section for examples of the target API.
> Drill provides over 100 different value vectors, any of which may be required 
> to perform a specific unit test. Creating these vectors, populating them, and 
> retrieving values, is very tedious. The work is so complex that it acts to 
> discourage developers from writing such tests.
> To simplify the task, we wish to provide a simplified row set reader and 
> writer. To do that, we need to generate the corresponding column reader and 
> writer for each value vector. This ticket focuses on the column-level readers 
> and writers, and the required code generation.
> Drill already provides vector readers and writers derived from 
> {{FieldReader}}. However, these readers do not provide a uniform get/set 
> interface that is type independent on the application side. Instead, 
> application code must be aware of the type of the vector, something we seek 
> to avoid for test code.
> The reader and writer classes are designed to be used in many contexts, not 
> just for testing. As a result, their implementation makes no assumptions 
> about the broader row reader and writer, other than that a row index and the 
> required value vector are both available. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5089) Skip initializing all enabled storage plugins for every query

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968517#comment-15968517
 ] 

Julian Hyde commented on DRILL-5089:


How are you able to call {{SimpleCalciteSchema.from}}? Isn't 
{{SimpleCalciteSchema}} package-private?

In CALCITE-1748 you ask to override a method that returns {{CalciteSchema}} but 
in CALCITE-911 we agreed that Drill wouldn't create your own CalciteSchema 
sub-classes. What has changed?

Can you take a look at CALCITE-1742, and tell me how it relates to your 
problem? I'd rather fix Calcite's schema cache for Drill's and Phoenix's needs 
rather than let people drill holes in our APIs.

> Skip initializing all enabled storage plugins for every query
> -
>
> Key: DRILL-5089
> URL: https://issues.apache.org/jira/browse/DRILL-5089
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Query Planning & Optimization
>Affects Versions: 1.9.0
>Reporter: Abhishek Girish
>Assignee: Chunhui Shi
>Priority: Critical
>
> In a query's lifecycle, at attempt is made to initialize each enabled storage 
> plugin, while building the schema tree. This is done regardless of the actual 
> plugins involved within a query. 
> Sometimes, when one or more of the enabled storage plugins have issues - 
> either due to misconfiguration or the underlying datasource being slow or 
> being down, the overall query time taken increases drastically. Most likely 
> due the attempt being made to register schemas from a faulty plugin.
> For example, when a jdbc plugin is configured with SQL Server, and at one 
> point the underlying SQL Server db goes down, any Drill query starting to 
> execute at that point and beyond begin to slow down drastically. 
> We must skip registering unrelated schemas (& workspaces) for a query. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5324) Provide simplified column reader/writer for use in tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968422#comment-15968422
 ] 

ASF GitHub Bot commented on DRILL-5324:
---

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/783#discussion_r111495333
  
--- Diff: 
exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/ScalarWriter.java
 ---
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.vector.accessor;
+
+import java.math.BigDecimal;
+
+import org.joda.time.Period;
+
--- End diff --

Adding a comment here saying that Writers use an implicit index would make 
it easier for readers to understand why there is no index parameter and why the 
ScalarWriter interface can be inherited-from by both the ColumnWriter and the 
ArrayWriter


> Provide simplified column reader/writer for use in tests
> 
>
> Key: DRILL-5324
> URL: https://issues.apache.org/jira/browse/DRILL-5324
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> In support of DRILL-5323, we wish to provide a very easy way to work with row 
> sets. See the comment section for examples of the target API.
> Drill provides over 100 different value vectors, any of which may be required 
> to perform a specific unit test. Creating these vectors, populating them, and 
> retrieving values, is very tedious. The work is so complex that it acts to 
> discourage developers from writing such tests.
> To simplify the task, we wish to provide a simplified row set reader and 
> writer. To do that, we need to generate the corresponding column reader and 
> writer for each value vector. This ticket focuses on the column-level readers 
> and writers, and the required code generation.
> Drill already provides vector readers and writers derived from 
> {{FieldReader}}. However, these readers do not provide a uniform get/set 
> interface that is type independent on the application side. Instead, 
> application code must be aware of the type of the vector, something we seek 
> to avoid for test code.
> The reader and writer classes are designed to be used in many contexts, not 
> just for testing. As a result, their implementation makes no assumptions 
> about the broader row reader and writer, other than that a row index and the 
> required value vector are both available. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5324) Provide simplified column reader/writer for use in tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968421#comment-15968421
 ] 

ASF GitHub Bot commented on DRILL-5324:
---

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/783#discussion_r111289506
  
--- Diff: exec/vector/src/main/codegen/templates/ColumnAccessors.java ---
@@ -0,0 +1,333 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+<@pp.dropOutputFile />
+<@pp.changeOutputFile 
name="/org/apache/drill/exec/vector/accessor/ColumnAccessors.java" />
+<#include "/@includes/license.ftl" />
+<#macro getType label>
+@Override
+public ValueType valueType() {
+  <#if label == "Int">
+  return ValueType.INTEGER;
+  <#else>
+  return ValueType.${label?upper_case};
+  
+}
+
+<#macro bindReader prefix drillType>
+  <#if drillType = "Decimal9" || drillType == "Decimal18">
+private MaterializedField field;
+  
+private ${prefix}${drillType}Vector.Accessor accessor;
+
+@Override
+public void bind(RowIndex vectorIndex, ValueVector vector) {
+  bind(vectorIndex);
+  <#if drillType = "Decimal9" || drillType == "Decimal18">
+  field = vector.getField();
+  
+  accessor = ((${prefix}${drillType}Vector) vector).getAccessor();
+}
+
+  <#if drillType = "Decimal9" || drillType == "Decimal18">
+@Override
+public void bind(RowIndex vectorIndex, MaterializedField field, 
VectorAccessor va) {
+  bind(vectorIndex, field, va);
+  this.field = field;
+}
+
+ 
+   private ${prefix}${drillType}Vector.Accessor accessor() {
+  if (vectorAccessor == null) {
+return accessor;
+  } else {
+return ((${prefix}${drillType}Vector) 
vectorAccessor.vector()).getAccessor();
+  }
+}
+
+<#macro get drillType accessorType label isArray>
+@Override
+public ${accessorType} get${label}(<#if isArray>int index) {
+  <#if isArray>
+<#assign index=", index"/>
+<#assign getObject="getSingleObject">
+  <#else>
+<#assign index=""/>
+<#assign getObject="getObject">
+  
+  <#if drillType == "VarChar">
+  return new String(accessor().get(vectorIndex.index()${index}), 
Charsets.UTF_8);
+  <#elseif drillType == "Var16Char">
+  return new String(accessor().get(vectorIndex.index()${index}), 
Charsets.UTF_16);
+  <#elseif drillType == "VarBinary">
+  return accessor().get(vectorIndex.index()${index});
+  <#elseif drillType == "Decimal9" || drillType == "Decimal18">
+  return DecimalUtility.getBigDecimalFromPrimitiveTypes(
+accessor().get(vectorIndex.index()${index}),
+field.getScale(),
+field.getPrecision());
+  <#elseif accessorType == "Decimal18">
+  return 
DecimalUtilities.getBigDecimalFromPrimitiveTypes(accessor().${getObject}(vectorIndex.index()${index});
--- End diff --

As discusses offline, this seems to be deadcode as there is no 
DecimalUtilities class in the Drill source base.


> Provide simplified column reader/writer for use in tests
> 
>
> Key: DRILL-5324
> URL: https://issues.apache.org/jira/browse/DRILL-5324
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> In support of DRILL-5323, we wish to provide a very easy way to work with row 
> sets. See the comment section for examples of the target API.
> Drill provides over 100 different value vectors, any of which may be required 
> to perform a specific unit test. Creating these vectors, populating them, and 
> retrieving values, is very tedious. 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968269#comment-15968269
 ] 

Sudheesh Katkam commented on DRILL-5433:


Can you share relevant portion of Hive config? AFAIK there is no requirement to 
run as user "drill".

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
> ... 21 more
> Caused by: javax.security.sasl.SaslException: Server 

[jira] [Reopened] (DRILL-5104) Foreman sets external sort memory allocation even for a physical plan

[Rahul Challapalli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Challapalli reopened DRILL-5104:
--

> Foreman sets external sort memory allocation even for a physical plan
> -
>
> Key: DRILL-5104
> URL: https://issues.apache.org/jira/browse/DRILL-5104
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
> Attachments: 2711af55-e1f7-bf4e-3dcf-b9f66f69fc33.sys.drill, 
> physical.json
>
>
> Consider the (disabled) unit test 
> {{TestSimpleExternalSort.outOfMemoryExternalSort}} which uses the physical 
> plan {{xsort/oom_sort_test.json}} that contains a setting for the amount of 
> memory to allocate:
> {code}
>{
> ...
> pop:"external-sort",
> ...
> initialAllocation: 100,
> maxAllocation: 3000
> },
> {code}
> When run, the amount of memory is set to 715827882. The reason is that code 
> was added to {{Foreman}} to compute the memory to allocate to the external 
> sort:
> {code}
>   private void runPhysicalPlan(final PhysicalPlan plan) throws 
> ExecutionSetupException {
> validatePlan(plan);
> MemoryAllocationUtilities.setupSortMemoryAllocations(plan, queryContext);
> {code}
> The problem is that a physical plan should execute as provided to enable 
> detailed testing.
> To solve this problem, move the sort memory setup to the path taken by SQL 
> queries, but not via physical plans.
> This change is necessary to re-enable the previously-disabled external sort 
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968262#comment-15968262
 ] 

Parag Darji commented on DRILL-5433:


Yes all hadoop services are running fine (hdfs yarn hive hbase Kafka storm and 
more).
But none of them are running as user "drill".
Drill is just not working with kerberos (or the way we are passing the 
parameters in the sqlline command).

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968250#comment-15968250
 ] 

Sudheesh Katkam commented on DRILL-5433:


That's seems correct to me.

Do you have other services that are configured with Kerberos and working 
correctly? Any Hadoop ecosystem components?

I am not an expert in Kerberos, and so I am not sure what exactly the 
problem/solution is. I googled the error: *Server not found in Kerberos 
database (7) - UNKNOWN_SERVER* and each results provides a different solution, 
but the overall theme is around Kerberos configuration ([reverse name 
resolution|https://serverfault.com/questions/376762/nfs-with-kerberos-mount-failing-due-to-server-not-found-in-kerberos-database],
 [DNS-related Error 
Messages|https://technet.microsoft.com/en-us/library/bb463167.aspx], [duplicate 
SPNs|https://sourceforge.net/p/spnego/discussion/1003769/thread/67c0183c/], 
[others|http://thejavamonkey.blogspot.com/2008/04/clientserver-hello-world-in-kerberos.html],
 etc.) You may want to look into those options, and the debug should help you 
more. If you do figure things out, please post your findings here.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968166#comment-15968166
 ] 

Parag Darji commented on DRILL-5433:


Permission seems to be correct. Please correct me if I'm wrong, if there was 
permission issue drillbit won't start and/or kinit for dirll id would have 
error'ed.
I tried to move the drill.keytab file from /opt/drill into "drill" user's home 
dir and still failing.
{code}
drill@:/home/drill/.keytab> ls -ld /home/drill/.keytab
drwx-- 2 drill drill 4096 Apr  6 16:28 /home/drill/.keytab
drill@:/home/drill/.keytab> ls -ld /home/drill/.keytab/drill.keytab
-rwx-- 1 drill drill 262 Apr  6 16:28 /home/drill/.keytab/drill.keytab
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968136#comment-15968136
 ] 

Parag Darji edited comment on DRILL-5433 at 4/13/17 7:49 PM:
-

I will look into scenario 1 permission.

Sorry I forgot to do the klist for test user.

{code}
host1:/var/log/ambari-server # su - test
test@:/home/test> klist
Ticket cache: FILE:/tmp/krb5cc_5007
Default principal: test/lab...@lab.com

Valid starting ExpiresService principal
04/13/17 15:00:25  04/14/17 15:00:25  krbtgt/lab@lab.com
renew until 04/13/17 15:00:25
test@:/home/test> clear
test@:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>> Look up native default credential cache
>>>KinitOptions cache name is /tmp/krb5cc_5007
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is krbtgt/lab@lab.com
>>>DEBUG  key type: 18
>>>DEBUG  auth time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  start time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  end time: Fri Apr 14 15:00:25 EDT 2017
>>>DEBUG  renew_till time: Thu Apr 13 15:00:25 EDT 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is 
>>>X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/lab@lab.com
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  start time: null
>>>DEBUG  end time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
Can't set level for java.util.logging.ConsoleHandler
Search Subject for Kerberos V5 INIT cred (<>, 
sun.security.jgss.krb5.Krb5InitCredential)
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3, number of 
>>> retries =3, #bytes=659
>>> KDCCommunication: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3,Attempt 
>>> =1, #bytes=659
>>> KrbKdcReq send: #bytes read=161
>>> KdcAccessibility: remove altbthdlhdpsb01.cscdev.com
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 cTime is Thu Aug 19 05:00:10 EDT 1982 39859561
 sTime is Thu Apr 13 15:46:13 EDT 2017 1492112773000
 suSec is 302285
 error code is 7
 error Message is Server not found in Kerberos database
 crealm is LAB.COM
 cname is test/labhdp
 realm is LAB.COM
 sname is drill/ladhdp
 msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at 
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
at 
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at 
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:231)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.evaluateChallenge(AuthenticationOutcomeListener.java:228)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.initiate(AuthenticationOutcomeListener.java:89)
at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968136#comment-15968136
 ] 

Parag Darji edited comment on DRILL-5433 at 4/13/17 7:48 PM:
-

I will look into scenario 1 permission.

Sorry I forgot to do the klist for test user.

{code}
host1:/var/log/ambari-server # su - test
test@:/home/test> klist
Ticket cache: FILE:/tmp/krb5cc_5007
Default principal: test/lab...@lab.com

Valid starting ExpiresService principal
04/13/17 15:00:25  04/14/17 15:00:25  krbtgt/lab@lab.com
renew until 04/13/17 15:00:25
test@:/home/test> clear
test@:/home/test> sqlline -u 
"jdbc:drill:drillbit=altbthdlhdpsb01.cscdev.com;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>> Look up native default credential cache
>>>KinitOptions cache name is /tmp/krb5cc_5007
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is krbtgt/lab@lab.com
>>>DEBUG  key type: 18
>>>DEBUG  auth time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  start time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  end time: Fri Apr 14 15:00:25 EDT 2017
>>>DEBUG  renew_till time: Thu Apr 13 15:00:25 EDT 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is 
>>>X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/lab@lab.com
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  start time: null
>>>DEBUG  end time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
Can't set level for java.util.logging.ConsoleHandler
Search Subject for Kerberos V5 INIT cred (<>, 
sun.security.jgss.krb5.Krb5InitCredential)
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3, number of 
>>> retries =3, #bytes=659
>>> KDCCommunication: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3,Attempt 
>>> =1, #bytes=659
>>> KrbKdcReq send: #bytes read=161
>>> KdcAccessibility: remove altbthdlhdpsb01.cscdev.com
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 cTime is Thu Aug 19 05:00:10 EDT 1982 39859561
 sTime is Thu Apr 13 15:46:13 EDT 2017 1492112773000
 suSec is 302285
 error code is 7
 error Message is Server not found in Kerberos database
 crealm is LAB.COM
 cname is test/labhdp
 realm is LAB.COM
 sname is drill/ladhdp
 msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at 
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
at 
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at 
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:231)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.evaluateChallenge(AuthenticationOutcomeListener.java:228)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.initiate(AuthenticationOutcomeListener.java:89)
   

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968136#comment-15968136
 ] 

Parag Darji commented on DRILL-5433:


I will look into scenario 1 permission.

Sorry I forgot to do the klist for test user.

{code}
altbthdlhdpsb01:/var/log/ambari-server # su - test
test@altbthdlhdpsb01:/home/test> klist
Ticket cache: FILE:/tmp/krb5cc_5007
Default principal: test/lab...@lab.com

Valid starting ExpiresService principal
04/13/17 15:00:25  04/14/17 15:00:25  krbtgt/lab@lab.com
renew until 04/13/17 15:00:25
test@altbthdlhdpsb01:/home/test> clear
test@altbthdlhdpsb01:/home/test> sqlline -u 
"jdbc:drill:drillbit=altbthdlhdpsb01.cscdev.com;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>> Look up native default credential cache
>>>KinitOptions cache name is /tmp/krb5cc_5007
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is krbtgt/lab@lab.com
>>>DEBUG  key type: 18
>>>DEBUG  auth time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  start time: Thu Apr 13 15:00:25 EDT 2017
>>>DEBUG  end time: Fri Apr 14 15:00:25 EDT 2017
>>>DEBUG  renew_till time: Thu Apr 13 15:00:25 EDT 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is test/lab...@lab.com
>>>DEBUG  server principal is 
>>>X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/lab@lab.com
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  start time: null
>>>DEBUG  end time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
Can't set level for java.util.logging.ConsoleHandler
Search Subject for Kerberos V5 INIT cred (<>, 
sun.security.jgss.krb5.Krb5InitCredential)
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for test/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Fri Apr 14 15:00:25 EDT 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3, number of 
>>> retries =3, #bytes=659
>>> KDCCommunication: kdc=altbthdlhdpsb01.cscdev.com UDP:88, timeout=3,Attempt 
>>> =1, #bytes=659
>>> KrbKdcReq send: #bytes read=161
>>> KdcAccessibility: remove altbthdlhdpsb01.cscdev.com
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 cTime is Thu Aug 19 05:00:10 EDT 1982 39859561
 sTime is Thu Apr 13 15:46:13 EDT 2017 1492112773000
 suSec is 302285
 error code is 7
 error Message is Server not found in Kerberos database
 crealm is LAB.COM
 cname is test/labhdp
 realm is LAB.COM
 sname is drill/ladhdp
 msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at 
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
at 
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at 
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:231)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.evaluateChallenge(AuthenticationOutcomeListener.java:228)
at 

[jira] [Issue Comment Deleted] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Parag Darji updated DRILL-5433:
---
Comment: was deleted

(was: I will check on scenario 1.
Sorry I forgot to kinit for option 2:
{code}
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Error: Failure in connecting to Drill: org.apache.drill.exec.rpc.RpcException: 
CONNECTION : java.nio.channels.UnresolvedAddressException (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.exec.rpc.user.UserClient$2.connectionFailed(UserClient.java:181)
at 
org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler.connectionFailed(QueryResultHandler.java:389)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:249)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:212)
at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)
at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
at 
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:235)
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1089)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:50)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:510)
at 
io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:909)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:203)
at io.netty.bootstrap.Bootstrap$2.run(Bootstrap.java:165)
at 
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:47)
at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968084#comment-15968084
 ] 

Parag Darji edited comment on DRILL-5433 at 4/13/17 7:35 PM:
-

I will check on scenario 1.
Sorry I forgot to kinit for option 2:
{code}
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Error: Failure in connecting to Drill: org.apache.drill.exec.rpc.RpcException: 
CONNECTION : java.nio.channels.UnresolvedAddressException (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.exec.rpc.user.UserClient$2.connectionFailed(UserClient.java:181)
at 
org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler.connectionFailed(QueryResultHandler.java:389)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:249)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:212)
at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)
at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
at 
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:235)
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1089)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:50)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:510)
at 
io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:909)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:203)
at io.netty.bootstrap.Bootstrap$2.run(Bootstrap.java:165)
at 
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968084#comment-15968084
 ] 

Parag Darji commented on DRILL-5433:


I will check on scenario 1.
Sorry I forgot to kinit for option 2:
{code}
test@altbthdlhdpsb01:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Error: Failure in connecting to Drill: org.apache.drill.exec.rpc.RpcException: 
CONNECTION : java.nio.channels.UnresolvedAddressException (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.exec.rpc.RpcException: CONNECTION : 
java.nio.channels.UnresolvedAddressException
at 
org.apache.drill.exec.rpc.user.UserClient$2.connectionFailed(UserClient.java:181)
at 
org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler.connectionFailed(QueryResultHandler.java:389)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:249)
at 
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:212)
at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)
at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
at 
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:235)
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1089)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:50)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:543)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:528)
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:510)
at 
io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:909)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:203)
at io.netty.bootstrap.Bootstrap$2.run(Bootstrap.java:165)
at 
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:47)
at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 6:42 PM:
-

I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the keytab file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}


was (Author: sudheeshkatkam):
I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 6:26 PM:
-

I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}


was (Author: sudheeshkatkam):
I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess would be that the permissions 
on the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam commented on DRILL-5433:


I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess would be that the permissions 
on the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by 

[jira] [Created] (DRILL-5436) Need a way to input password which contains space when calling sqlline

[Hao Zhu (JIRA)]

Hao Zhu created DRILL-5436:
--

 Summary: Need a way to input password which contains space when 
calling sqlline
 Key: DRILL-5436
 URL: https://issues.apache.org/jira/browse/DRILL-5436
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - CLI
Affects Versions: 1.10.0
Reporter: Hao Zhu


create a user named "spaceuser" with password "hello world".
All below failed:
{code}
sqlline -u jdbc:drill:zk=xxx -n spaceuser -p 'hello world'
sqlline -u jdbc:drill:zk=xxx -n spaceuser -p "hello world"
sqlline -u jdbc:drill:zk=xxx -n spaceuser -p 'hello\ world'
sqlline -u jdbc:drill:zk=xxx -n spaceuser -p "hello\ world"
{code}

Need a way to input password which contains space when calling sqlline



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967950#comment-15967950
 ] 

Parag Darji commented on DRILL-5433:


I've tried both ways, seems to be the same error:
{code}
drill@host1:/home/drill> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>>> KdcAccessibility: reset
>>> KdcAccessibility: reset
>>> KeyTabInputStream, readName(): LAB.COM
>>> KeyTabInputStream, readName(): drill
>>> KeyTabInputStream, readName(): labhdp
>>> KeyTab: load() entry length: 71; type: 18
>>> KeyTabInputStream, readName(): LAB.COM
>>> KeyTabInputStream, readName(): drill
>>> KeyTabInputStream, readName(): labhdp
>>> KeyTab: load() entry length: 55; type: 17
>>> KeyTabInputStream, readName(): LAB.COM
>>> KeyTabInputStream, readName(): drill
>>> KeyTabInputStream, readName(): labhdp
>>> KeyTab: load() entry length: 63; type: 16
>>> KeyTabInputStream, readName(): LAB.COM
>>> KeyTabInputStream, readName(): drill
>>> KeyTabInputStream, readName(): labhdp
>>> KeyTab: load() entry length: 55; type: 23
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
Error: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed unexpectedly. [Caused 
by java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: 
Failed to login. [Caused by javax.security.auth.login.LoginException: Unable to 
obtain password from user
]] (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed unexpectedly. [Caused 
by java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: 
Failed to login. [Caused by javax.security.auth.login.LoginException: Unable to 
obtain password from user
]]
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed unexpectedly. [Caused 
by java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: 
Failed to login. [Caused by javax.security.auth.login.LoginException: Unable to 
obtain password from user
]]
at 
org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
at 
org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
at 
org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
... 18 more
Caused by: javax.security.sasl.SaslException: Authentication failed 
unexpectedly. [Caused by java.util.concurrent.ExecutionException: 
javax.security.sasl.SaslException: Failed to login. [Caused by 
javax.security.auth.login.LoginException: Unable to obtain password from user
]]
at 
org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:207)
at 
org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 5:22 PM:
-

Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

In either case, service principal (for drill) is specified using {{principal=}}.


was (Author: sudheeshkatkam):
Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 5:20 PM:
-

Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}


was (Author: sudheeshkatkam):
Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam commented on DRILL-5433:


Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967883#comment-15967883
 ] 

Parag Darji commented on DRILL-5433:


Do mean log in as user "test" and run below commnad?
{code}
sqlline --maxWidth=1 -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=test/lad...@lab.com;keytab:/home/test/test.keytab"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967865#comment-15967865
 ] 

Sudheesh Katkam commented on DRILL-5433:


I am not sure if what you are testing occurs in a typical Kerberos deployment 
i.e. a service trying to getting a ticket for itself ({{... 
drill/lab...@lab.com for drill/lad...@lab.com ...}}).

Usually sqlline script is invoked as another user (e.g. {{b...@lab.com}}). Can 
you try this?
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

This uses keytab instead of ticket as credential during login.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> 

[jira] [Updated] (DRILL-5424) Fix IOBE for reverse function

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5424:

Labels: ready-to-commit  (was: )

> Fix IOBE for reverse function
> -
>
> Key: DRILL-5424
> URL: https://issues.apache.org/jira/browse/DRILL-5424
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.9.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Volodymyr Vysotskyi
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> Query with reverse function fails:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test.json`;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 259, length: 1 
> (expected: range(0, 256))
> {code}
> for table with several long varchars.
> {noformat}
> cat /tmp/test.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}
> The same query works for the table with less row number:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test2.json`;
> +---+
> |EXPR$0 |
> +---+
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> +---+
> {code}
> {noformat}
> cat /tmp/test2.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5424) Fix IOBE for reverse function

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5424:

Fix Version/s: 1.11.0

> Fix IOBE for reverse function
> -
>
> Key: DRILL-5424
> URL: https://issues.apache.org/jira/browse/DRILL-5424
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.9.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Volodymyr Vysotskyi
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> Query with reverse function fails:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test.json`;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 259, length: 1 
> (expected: range(0, 256))
> {code}
> for table with several long varchars.
> {noformat}
> cat /tmp/test.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}
> The same query works for the table with less row number:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test2.json`;
> +---+
> |EXPR$0 |
> +---+
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> +---+
> {code}
> {noformat}
> cat /tmp/test2.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5424) Fix IOBE for reverse function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967711#comment-15967711
 ] 

ASF GitHub Bot commented on DRILL-5424:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/815
  
+1


> Fix IOBE for reverse function
> -
>
> Key: DRILL-5424
> URL: https://issues.apache.org/jira/browse/DRILL-5424
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.9.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Volodymyr Vysotskyi
>Priority: Minor
>
> Query with reverse function fails:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test.json`;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 259, length: 1 
> (expected: range(0, 256))
> {code}
> for table with several long varchars.
> {noformat}
> cat /tmp/test.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}
> The same query works for the table with less row number:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test2.json`;
> +---+
> |EXPR$0 |
> +---+
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> +---+
> {code}
> {noformat}
> cat /tmp/test2.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967664#comment-15967664
 ] 

Parag Darji edited comment on DRILL-5433 at 4/13/17 2:42 PM:
-

Here is the output after enabling extended logging:
{code}
drill@:/var/log> klist
Ticket cache: FILE:/tmp/krb5cc_5005
Default principal: drill/lab...@lab.com

Valid starting ExpiresService principal
04/12/17 16:41:00  04/13/17 16:41:00  krbtgt/lab@lab.com
renew until 04/12/17 16:41:00

drill@:/home/drill> sqlline --maxWidth=1 -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>> Look up native default credential cache
>>>KinitOptions cache name is /tmp/krb5cc_5005
>>>DEBUG   client principal is drill/lab...@lab.com
>>>DEBUG  server principal is krbtgt/lab@lab.com
>>>DEBUG  key type: 18
>>>DEBUG  auth time: Wed Apr 12 16:41:00 EDT 2017
>>>DEBUG  start time: Wed Apr 12 16:41:00 EDT 2017
>>>DEBUG  end time: Thu Apr 13 16:41:00 EDT 2017
>>>DEBUG  renew_till time: Wed Apr 12 16:41:00 EDT 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is drill/lab...@lab.com
>>>DEBUG  server principal is 
>>>X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/lab@lab.com
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  start time: null
>>>DEBUG  end time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
Can't set level for java.util.logging.ConsoleHandler
Search Subject for Kerberos V5 INIT cred (<>, 
sun.security.jgss.krb5.Krb5InitCredential)
Found ticket for drill/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Thu Apr 13 16:41:00 EDT 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for drill/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Thu Apr 13 16:41:00 EDT 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=host1.fqdn UDP:88, timeout=3, number of retries =3, 
>>> #bytes=661
>>> KDCCommunication: kdc=host1.fqdn UDP:88, timeout=3,Attempt =1, #bytes=661
>>> KrbKdcReq send: #bytes read=162
>>> KdcAccessibility: remove host1.fqdn
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 cTime is Fri Jun 26 07:25:12 EDT 2015 1435317912000
 sTime is Thu Apr 13 10:29:20 EDT 2017 149209376
 suSec is 935095
 error code is 7
 error Message is Server not found in Kerberos database
 crealm is LAB.COM
 cname is drill/labhdp
 realm is LAB.COM
 sname is drill/ladhdp
 msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at 
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
at 
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at 
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:231)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.evaluateChallenge(AuthenticationOutcomeListener.java:228)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.initiate(AuthenticationOutcomeListener.java:89)
at 
org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:231)
at 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Parag Darji (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967664#comment-15967664
 ] 

Parag Darji commented on DRILL-5433:


Here is the output after enabling extended logging:
{code}
drill@:/var/log> klist
Ticket cache: FILE:/tmp/krb5cc_5005
Default principal: drill/lab...@lab.com

Valid starting ExpiresService principal
04/12/17 16:41:00  04/13/17 16:41:00  krbtgt/lab@lab.com
renew until 04/12/17 16:41:00

drill@:/home/drill> sqlline --maxWidth=1 -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
Bad level value for property: java.util.logging.ConsoleHandler.level
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>> Look up native default credential cache
>>>KinitOptions cache name is /tmp/krb5cc_5005
>>>DEBUG   client principal is drill/lab...@lab.com
>>>DEBUG  server principal is krbtgt/lab@lab.com
>>>DEBUG  key type: 18
>>>DEBUG  auth time: Wed Apr 12 16:41:00 EDT 2017
>>>DEBUG  start time: Wed Apr 12 16:41:00 EDT 2017
>>>DEBUG  end time: Thu Apr 13 16:41:00 EDT 2017
>>>DEBUG  renew_till time: Wed Apr 12 16:41:00 EDT 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is drill/lab...@lab.com
>>>DEBUG  server principal is 
>>>X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/lab@lab.com
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  start time: null
>>>DEBUG  end time: Wed Dec 31 19:00:00 EST 1969
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
Can't set level for java.util.logging.ConsoleHandler
Search Subject for Kerberos V5 INIT cred (<>, 
sun.security.jgss.krb5.Krb5InitCredential)
Found ticket for drill/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Thu Apr 13 16:41:00 EDT 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for drill/lab...@lab.com to go to krbtgt/lab@lab.com expiring 
on Thu Apr 13 16:41:00 EDT 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=host1.fqdn UDP:88, timeout=3, number of retries =3, 
>>> #bytes=661
>>> KDCCommunication: kdc=host1.fqdn UDP:88, timeout=3,Attempt =1, #bytes=661
>>> KrbKdcReq send: #bytes read=162
>>> KdcAccessibility: remove host1.fqdn
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 cTime is Fri Jun 26 07:25:12 EDT 2015 1435317912000
 sTime is Thu Apr 13 10:29:20 EDT 2017 149209376
 suSec is 935095
 error code is 7
 error Message is Server not found in Kerberos database
 crealm is LAB.COM
 cname is drill/labhdp
 realm is LAB.COM
 sname is drill/ladhdp
 msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at 
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
at 
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at 
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:231)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener$1.run(AuthenticationOutcomeListener.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.evaluateChallenge(AuthenticationOutcomeListener.java:228)
at 
org.apache.drill.exec.rpc.security.AuthenticationOutcomeListener.initiate(AuthenticationOutcomeListener.java:89)
at 
org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:231)
at 
org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
at 

[jira] [Assigned] (DRILL-5435) Using Limit causes Memory Leaked Error since 1.10

[Zelaine Fong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zelaine Fong reassigned DRILL-5435:
---

Assignee: Parth Chandra

> Using Limit causes Memory Leaked Error since 1.10
> -
>
> Key: DRILL-5435
> URL: https://issues.apache.org/jira/browse/DRILL-5435
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.10.0
>Reporter: F Méthot
>Assignee: Parth Chandra
>
> Here is the details I can provide:
>   We migrated our production system from Drill 1.9 to 1.10 just 5 days ago. 
> (220 nodes cluster)
> Our log show there was some 900+ queries ran without problem in first 4 days. 
>  (similar queries, that never use the `limit` clause)
> Yesterday we started doing simple adhoc select * ... limit 10 queries (like 
> we often do, that was our first use of limit with 1.10)
> and we got a `Memory was leaked` exception below.
> Also, once we get the error, Most of all subsequent user queries fails with 
> Channel Close Exception. We need to restart Drill to bring it back to normal.
> A day later, I used a similar select * limit 10 queries, and the same thing 
> happen, had to restart Drill.
> In the exception, it was refering to a file (1_0_0.parquet)
> I moved that file to smaller test cluster (12 nodes) and got the error on the 
> first attempt. but I am no longer able to reproduce the issue on that file. 
> Between the 12 and 220 nodes cluster, a different Column name and Row Group 
> Start was listed in the error.
> The parquet file was generated by Drill 1.10.
> I tried the same file with a local drill-embedded 1.9 and 1.10 and had no 
> issue.
> Here is the error (manually typed), if you think of anything obvious, let us 
> know.
> AsyncPageReader - User Error Occured: Exception Occurred while reading from 
> disk (can not read class o.a.parquet.format.PageHeader: java.io.IOException: 
> input stream is closed.)
> File:/1_0_0.parquet
> Column: StringColXYZ
> Row Group Start: 115215476
> [Error Id: ]
>   at UserException.java:544)
>   at 
> o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.handleAndThrowException(AsynvPageReader.java:199)
>   at 
> o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.access(AsynvPageReader.java:81)
>   at 
> o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:483)
>   at 
> o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:392)
>   at 
> o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:392)
> ...
> Caused by: java.io.IOException: can not read class 
> org.apache.parquet.format.PageHeader: java.io.IOException: Input Stream is 
> closed.
>at o.a.parquet.format.Util.read(Util.java:216)
>at o.a.parquet.format.Util.readPageHeader(Util.java:65)
>at 
> o.a.drill.exec.store.parquet.columnreaders.AsyncPageReader(AsyncPageReaderTask:430)
> Caused by: parquet.org.apache.thrift.transport.TTransportException: Input 
> stream is closed
>at ...read(TIOStreamTransport.java:129)
>at TTransport.readAll(TTransport.java:84)
>at TCompactProtocol.readByte(TCompactProtocol.java:474)
>at TCompactProtocol.readFieldBegin(TCompactProtocol.java:481)
>at InterningProtocol.readFieldBegin(InterningProtocol.java:158)
>at o.a.parquet.format.PageHeader.read(PageHeader.java:828)
>at o.a.parquet.format.Util.read(Util.java:213)
> Fragment 0:0
> [Error id: ...]
> o.a.drill.common.exception.UserException: SYSTEM ERROR: 
> IllegalStateException: Memory was leaked by query. Memory leaked: (524288)
> Allocator(op:0:0:4:ParquetRowGroupScan) 100/524288/39919616/100
>   at o.a.d.common.exceptions.UserException (UserException.java:544)
>   at 
> o.a.d.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)
>   at o.a.d.exec.work.fragment.FragmentExecutor.cleanup( 
> FragmentExecutor.java:160)
>   at o.a.d.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
> ...
> Caused by: IllegalStateException: Memory was leaked by query. Memory leaked: 
> (524288)
>   at o.a.d.exec.memory.BaseAllocator.close(BaseAllocator.java:502)
>   at o.a.d.exec.ops.OperatorContextImpl(OperatorContextImpl.java:149)
>   at o.a.d.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:422)
>   at o.a.d.exec.ops.FragmentContext.close(FragmentContext.java:411)
>   at 
> o.a.d.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:318)
>   at 
> o.a.d.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:155)
> This fixed the problem:
> alter  set `store.parquet.reader.pagereader.async`=false;



--
This message was sent by Atlassian JIRA

[jira] [Created] (DRILL-5435) Using Limit causes Memory Leaked Error since 1.10

[JIRA]

F Méthot created DRILL-5435:
---

 Summary: Using Limit causes Memory Leaked Error since 1.10
 Key: DRILL-5435
 URL: https://issues.apache.org/jira/browse/DRILL-5435
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Affects Versions: 1.10.0
Reporter: F Méthot


Here is the details I can provide:

  We migrated our production system from Drill 1.9 to 1.10 just 5 days ago. 
(220 nodes cluster)

Our log show there was some 900+ queries ran without problem in first 4 days.  
(similar queries, that never use the `limit` clause)

Yesterday we started doing simple adhoc select * ... limit 10 queries (like we 
often do, that was our first use of limit with 1.10)
and we got a `Memory was leaked` exception below.

Also, once we get the error, Most of all subsequent user queries fails with 
Channel Close Exception. We need to restart Drill to bring it back to normal.

A day later, I used a similar select * limit 10 queries, and the same thing 
happen, had to restart Drill.

In the exception, it was refering to a file (1_0_0.parquet)
I moved that file to smaller test cluster (12 nodes) and got the error on the 
first attempt. but I am no longer able to reproduce the issue on that file. 
Between the 12 and 220 nodes cluster, a different Column name and Row Group 
Start was listed in the error.
The parquet file was generated by Drill 1.10.

I tried the same file with a local drill-embedded 1.9 and 1.10 and had no issue.


Here is the error (manually typed), if you think of anything obvious, let us 
know.


AsyncPageReader - User Error Occured: Exception Occurred while reading from 
disk (can not read class o.a.parquet.format.PageHeader: java.io.IOException: 
input stream is closed.)

File:/1_0_0.parquet
Column: StringColXYZ
Row Group Start: 115215476

[Error Id: ]
  at UserException.java:544)
  at 
o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.handleAndThrowException(AsynvPageReader.java:199)
  at 
o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.access(AsynvPageReader.java:81)
  at 
o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:483)
  at 
o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:392)
  at 
o.a.d.exec.store.parquet.columnreaders.AsyncPageReader.AsyncPageReaderTask.call(AsyncPageReader.java:392)
...
Caused by: java.io.IOException: can not read class 
org.apache.parquet.format.PageHeader: java.io.IOException: Input Stream is 
closed.
   at o.a.parquet.format.Util.read(Util.java:216)
   at o.a.parquet.format.Util.readPageHeader(Util.java:65)
   at 
o.a.drill.exec.store.parquet.columnreaders.AsyncPageReader(AsyncPageReaderTask:430)
Caused by: parquet.org.apache.thrift.transport.TTransportException: Input 
stream is closed
   at ...read(TIOStreamTransport.java:129)
   at TTransport.readAll(TTransport.java:84)
   at TCompactProtocol.readByte(TCompactProtocol.java:474)
   at TCompactProtocol.readFieldBegin(TCompactProtocol.java:481)
   at InterningProtocol.readFieldBegin(InterningProtocol.java:158)
   at o.a.parquet.format.PageHeader.read(PageHeader.java:828)
   at o.a.parquet.format.Util.read(Util.java:213)


Fragment 0:0
[Error id: ...]
o.a.drill.common.exception.UserException: SYSTEM ERROR: IllegalStateException: 
Memory was leaked by query. Memory leaked: (524288)
Allocator(op:0:0:4:ParquetRowGroupScan) 100/524288/39919616/100
  at o.a.d.common.exceptions.UserException (UserException.java:544)
  at 
o.a.d.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)
  at o.a.d.exec.work.fragment.FragmentExecutor.cleanup( 
FragmentExecutor.java:160)
  at o.a.d.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
...
Caused by: IllegalStateException: Memory was leaked by query. Memory leaked: 
(524288)
  at o.a.d.exec.memory.BaseAllocator.close(BaseAllocator.java:502)
  at o.a.d.exec.ops.OperatorContextImpl(OperatorContextImpl.java:149)
  at o.a.d.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:422)
  at o.a.d.exec.ops.FragmentContext.close(FragmentContext.java:411)
  at 
o.a.d.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:318)
  at 
o.a.d.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:155)




This fixed the problem:
alter  set `store.parquet.reader.pagereader.async`=false;



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5424) Fix IOBE for reverse function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967587#comment-15967587
 ] 

ASF GitHub Bot commented on DRILL-5424:
---

Github user vvysotskyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/815#discussion_r111389438
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/expr/fn/impl/TestStringFunctions.java
 ---
@@ -273,4 +273,13 @@ public void testSplit() throws Exception {
 .run();
   }
 
+  @Test
+  public void testReverse() throws Exception {
+testBuilder()
+  .sqlQuery("select reverse(reverse(n_comment)) words from 
cp.`tpch/nation.parquet`")
+  .unOrdered()
+  .sqlBaselineQuery("select n_comment words from 
cp.`tpch/nation.parquet`")
+  .build()
+  .run();
+  }
--- End diff --

Thanks, added these both tests.


> Fix IOBE for reverse function
> -
>
> Key: DRILL-5424
> URL: https://issues.apache.org/jira/browse/DRILL-5424
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.9.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Volodymyr Vysotskyi
>Priority: Minor
>
> Query with reverse function fails:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test.json`;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 259, length: 1 
> (expected: range(0, 256))
> {code}
> for table with several long varchars.
> {noformat}
> cat /tmp/test.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}
> The same query works for the table with less row number:
> {code:sql}
> 0: jdbc:drill:zk=local> select reverse(a) from dfs.`/tmp/test2.json`;
> +---+
> |EXPR$0 |
> +---+
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> | zyxwvutsrqponmlkjihgfedcbazyxwvutsrqponmlkjihgfedcba  |
> +---+
> {code}
> {noformat}
> cat /tmp/test2.json
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {"a": "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (DRILL-5434) IllegalStateException: Memory was leaked by query.

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-5434:
-

 Summary: IllegalStateException: Memory was leaked by query.
 Key: DRILL-5434
 URL: https://issues.apache.org/jira/browse/DRILL-5434
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.11.0
Reporter: Khurram Faraaz


Issue a long running COUNT query.
While the query is being executed, stop the foreman drillbit, ./drillbit.sh stop
A memory leak is reported in the drillbit.log

Apache Drill 1.11.0 
git.commit.id.abbrev=06e1522

Stack trace from drillbit.log
{noformat}
2017-04-13 06:14:36,828 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
2710e8b2-d4dc-1bee-016e-b69fd4966916: SELECT COUNT(*) FROM `twoKeyJsn.json`
2017-04-13 06:14:36,929 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,929 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,929 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,929 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,930 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,930 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,930 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,932 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2017-04-13 06:14:36,934 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using 
1 threads. Time: 2ms total, 2.102992ms avg, 2ms max.
2017-04-13 06:14:36,934 [2710e8b2-d4dc-1bee-016e-b69fd4966916:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using 
1 threads. Earliest start: 0.555000 μs, Latest start: 0.555000 μs, Average 
start: 0.555000 μs .
2017-04-13 06:14:36,949 [2710e8b2-d4dc-1bee-016e-b69fd4966916:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2710e8b2-d4dc-1bee-016e-b69fd4966916:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2017-04-13 06:14:36,949 [2710e8b2-d4dc-1bee-016e-b69fd4966916:frag:0:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2710e8b2-d4dc-1bee-016e-b69fd4966916:0:0: 
State to report: RUNNING
Thu Apr 13 06:14:40 UTC 2017 Terminating drillbit pid 5107
2017-04-13 06:14:40,756 [Drillbit-ShutdownHook#0] INFO  
o.apache.drill.exec.server.Drillbit - Received shutdown request.
2017-04-13 06:14:47,819 [pool-169-thread-2] INFO  
o.a.drill.exec.rpc.data.DataServer - closed eventLoopGroup 
io.netty.channel.nio.NioEventLoopGroup@4f3ee67c in 1024 ms
2017-04-13 06:14:47,819 [pool-169-thread-2] INFO  
o.a.drill.exec.service.ServiceEngine - closed dataPool in 1024 ms
2017-04-13 06:14:49,806 [Drillbit-ShutdownHook#0] WARN  
o.apache.drill.exec.work.WorkManager - Closing WorkManager but there are 1 
running fragments.
2017-04-13 06:14:49,807 [2710e8b2-d4dc-1bee-016e-b69fd4966916:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2710e8b2-d4dc-1bee-016e-b69fd4966916:0:0: 
State change requested RUNNING --> FAILED
2017-04-13 06:14:49,807 [Drillbit-ShutdownHook#0] INFO  
o.a.drill.exec.compile.CodeCompiler - Stats: code gen count: 6964, cache miss 
count: 335, hit rate: 95%
2017-04-13 06:14:49,807 [2710e8b2-d4dc-1bee-016e-b69fd4966916:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2710e8b2-d4dc-1bee-016e-b69fd4966916:0:0: 
State change requested FAILED --> FINISHED
2017-04-13 06:14:49,809 [2710e8b2-d4dc-1bee-016e-b69fd4966916:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException

Fragment 0:0

[Error Id: 94817261-98a9-4153-8b3a-2d9c95d80cc1 on centos-01.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
NullPointerException

Fragment 0:0

[Error Id: 94817261-98a9-4153-8b3a-2d9c95d80cc1 on centos-01.qa.lab:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
 ~[drill-common-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)