[jira] [Updated] (DRILL-5229) Upgrade kudu client to org.apache.kudu:kudu-client:1.2.0

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5229:
---
Labels: ready-to-commit  (was: )

> Upgrade kudu client to org.apache.kudu:kudu-client:1.2.0 
> -
>
> Key: DRILL-5229
> URL: https://issues.apache.org/jira/browse/DRILL-5229
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Other
>Affects Versions: 1.8.0
>Reporter: Rahul Raj
>Assignee: Sudheesh Katkam
>  Labels: ready-to-commit
>
> Getting an error -" out-of-order key" for a query select v,count(k) from
> kudu.test group by v where k is the primary key. This happens only when the
> aggregation is done on primary key. Should drill move to the latest kudu
> client to investigate this further?
> Current drill kudu connector uses org.kududb:kudu-client:0.6.0 from
> cloudera repository, where the latest released library
> org.apache.kudu:kudu-client:1.2.0 is hosted on maven central. There are a
> few breaking changes with the new library:
>1. TIMESTAMP renamed to UNIXTIME_MICROS
>2. In KuduRecordReader#setup -
>KuduScannerBuilder#lowerBoundPartitionKeyRaw renamed to lowerBoundRaw
>andKuduScannerBuilder#exclusiveUpperBoundPartitionKeyRaw renamed
>exclusiveUpperBoundRaw. Both methods are deprecated.
>3. In KuduRecordWriterImpl#updateSchema - client.createTable(name,
>kuduSchema) requires CreateTableOperatios as the third argument



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5229) Upgrade kudu client to org.apache.kudu:kudu-client:1.2.0

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5229:
---
Fix Version/s: (was: 2.0.0)

> Upgrade kudu client to org.apache.kudu:kudu-client:1.2.0 
> -
>
> Key: DRILL-5229
> URL: https://issues.apache.org/jira/browse/DRILL-5229
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Other
>Affects Versions: 1.8.0
>Reporter: Rahul Raj
>
> Getting an error -" out-of-order key" for a query select v,count(k) from
> kudu.test group by v where k is the primary key. This happens only when the
> aggregation is done on primary key. Should drill move to the latest kudu
> client to investigate this further?
> Current drill kudu connector uses org.kududb:kudu-client:0.6.0 from
> cloudera repository, where the latest released library
> org.apache.kudu:kudu-client:1.2.0 is hosted on maven central. There are a
> few breaking changes with the new library:
>1. TIMESTAMP renamed to UNIXTIME_MICROS
>2. In KuduRecordReader#setup -
>KuduScannerBuilder#lowerBoundPartitionKeyRaw renamed to lowerBoundRaw
>andKuduScannerBuilder#exclusiveUpperBoundPartitionKeyRaw renamed
>exclusiveUpperBoundRaw. Both methods are deprecated.
>3. In KuduRecordWriterImpl#updateSchema - client.createTable(name,
>kuduSchema) requires CreateTableOperatios as the third argument



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5533) Fix flag assignment in FunctionInitializer.checkInit() method

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5533:
---
Labels: ready-to-commit  (was: )

> Fix flag assignment in FunctionInitializer.checkInit() method
> -
>
> Key: DRILL-5533
> URL: https://issues.apache.org/jira/browse/DRILL-5533
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
>  Labels: ready-to-commit
>
> FunctionInitializer.checkInit() method uses DCL to ensure that function body 
> is loaded only once. But flag parameter is never updated and all threads are 
> entering synchronized block.
> Also FunctionInitializer.getImports() always returns empty list.
> https://github.com/apache/drill/blob/3e8b01d5b0d3013e3811913f0fd6028b22c1ac3f/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionInitializer.java
> Changes:
> 1. Fix DCL in FunctionInitializer.checkInit() method (update flag parameter  
> when function body is loaded).
> 2. Fix ImportGrabber.getImports() method to return list with imports.
> 3. Add unit tests for FunctionInitializer.
> 4. Minor refactoring (rename methods, add javadoc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5379) Set Hdfs Block Size based on Parquet Block Size

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5379:
--

Assignee: Sudheesh Katkam

> Set Hdfs Block Size based on Parquet Block Size
> ---
>
> Key: DRILL-5379
> URL: https://issues.apache.org/jira/browse/DRILL-5379
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - Parquet
>Affects Versions: 1.9.0
>Reporter: F Méthot
>Assignee: Sudheesh Katkam
>  Labels: ready-to-commit
> Fix For: Future
>
>
> It seems there a way to force Drill to store CTAS generated parquet file as a 
> single block when using HDFS. Java HDFS API allows to do that, files could be 
> created with the Parquet block-size set in a session or system config.
> Since it is ideal  to have single parquet file per hdfs block.
> Here is the HDFS API that allow to do that:
> http://archive.cloudera.com/cdh4/cdh/4/hadoop/api/org/apache/hadoop/fs/FileSystem.html#create(org.apache.hadoop.fs.Path,%20boolean,%20int,%20short,%20long)
> http://archive.cloudera.com/cdh4/cdh/4/hadoop/api/org/apache/hadoop/fs/FileSystem.html#create(org.apache.hadoop.fs.Path,%20boolean,%20int,%20short,%20long)
> Drill uses the hadoop ParquetFileWriter 
> (https://github.com/Parquet/parquet-mr/blob/master/parquet-hadoop/src/main/java/parquet/hadoop/ParquetFileWriter.java).
> This is where the file creation occurs so it might be tricky.
> However, ParquetRecordWriter.java 
> (https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordWriter.java)
>  in Drill creates the ParquetFileWriter with an hadoop configuration object.
> something to explore: Could the block size be set as a property within the 
> Configuration object before passing it to ParquetFileWriter constructor?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5356) Refactor Parquet Record Reader

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5356:
---
Labels: ready-to-commit  (was: )

> Refactor Parquet Record Reader
> --
>
> Key: DRILL-5356
> URL: https://issues.apache.org/jira/browse/DRILL-5356
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0, 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> The Parquet record reader class is a key part of Drill that has evolved over 
> time to become somewhat hard to follow.
> A number of us are working on Parquet-related tasks and find we have to spend 
> an uncomfortable amount of time trying to understand the code. In particular, 
> this writer needs to figure out how to convince the reader to provide 
> higher-density record batches.
> Rather than continue to decypher the complex code multiple times, this ticket 
> requests to refactor the code to make it functionally identical, but 
> structurally cleaner. The result will be faster time to value when working 
> with this code.
> This is a lower-priority change and will be coordinated with others working 
> on this code base. This ticket is only for the record reader class itself; it 
> does not include the various readers and writers that Parquet uses since 
> another project is actively modifying those classes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5504) Vector validator to diagnose offset vector issues

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5504:
---
Labels: ready-to-commit  (was: )

> Vector validator to diagnose offset vector issues
> -
>
> Key: DRILL-5504
> URL: https://issues.apache.org/jira/browse/DRILL-5504
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> DRILL-5470 describes a case in which an offset vector appears to have become 
> corrupted, yielding a bogus field-length value that is orders of magnitude 
> larger than the vector that contains the data.
> Debugging such cases is slow and tedious. To help, we propose to create a 
> "vector validator" that spins through vectors looking for problems.
> Then, to allow the validator to be used in the field, extend the "iterator 
> validator batch iterator" to optionally allow vector validation on each batch.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5512) Standardize error handling in ScanBatch

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5512:
---
Fix Version/s: (was: 1.10.0)

> Standardize error handling in ScanBatch
> ---
>
> Key: DRILL-5512
> URL: https://issues.apache.org/jira/browse/DRILL-5512
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>  Labels: ready-to-commit
>
> ScanBatch is the Drill operator executor that handles most readers. Like most 
> Drill operators, it uses an ad-hoc set of error detection and reporting 
> methods that evolved over Drill development.
> This ticket asks to standardize on error handling as outlined in DRILL-5083. 
> This basically means reporting all errors as a {{UserException}} rather than 
> using the {{IterOutcome.STOP}} return status or using the 
> {{FragmentContext.fail()}} method.
> This work requires the new error codes introduced in DRILL-5511, and is a 
> step toward making readers aware of vector size limits.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5481) Allow Drill to persist profiles in-memory only with a max capacity

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5481:
--

Assignee: Sudheesh Katkam  (was: Kunal Khatua)

> Allow Drill to persist profiles in-memory only with a max capacity
> --
>
> Key: DRILL-5481
> URL: https://issues.apache.org/jira/browse/DRILL-5481
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Kunal Khatua
>Assignee: Sudheesh Katkam
>  Labels: ready-to-commit
>
> To allow for fast persistence of profiles on a temporary basis (i.e. till the 
> life of the Drillbit), an existing test class 
> {{org.apache.drill.exec.testing.store.NoWriteLocalStore.java}} was refactored 
> to {{org.apache.drill.exec.store.sys.store.EphemeralPersistentStore}} and 
> given the ability to maintain a max capacity.
> This should allow query profiles to be available for as long as the Drillbit 
> process' lifespan.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5481) Allow Drill to persist profiles in-memory only with a max capacity

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5481:
--

Assignee: Kunal Khatua  (was: Sudheesh Katkam)

> Allow Drill to persist profiles in-memory only with a max capacity
> --
>
> Key: DRILL-5481
> URL: https://issues.apache.org/jira/browse/DRILL-5481
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
>
> To allow for fast persistence of profiles on a temporary basis (i.e. till the 
> life of the Drillbit), an existing test class 
> {{org.apache.drill.exec.testing.store.NoWriteLocalStore.java}} was refactored 
> to {{org.apache.drill.exec.store.sys.store.EphemeralPersistentStore}} and 
> given the ability to maintain a max capacity.
> This should allow query profiles to be available for as long as the Drillbit 
> process' lifespan.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5496) Must restart drillbits whenever a secure Hive metastore is restarted

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5496:
---
Labels: ready-to-commit  (was: )

> Must restart drillbits whenever a secure Hive metastore is restarted
> 
>
> Key: DRILL-5496
> URL: https://issues.apache.org/jira/browse/DRILL-5496
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> DRILL-4964: "Drill fails to connect to hive metastore after hive metastore is 
> restarted unless drillbits are restarted also" attempted to fix a bug in 
> Drill in which Drill hangs if Hive is restarted. Now, we see that all 
> subsequent "show schemas" queries fail.
> Steps to repro:
> 1. Build a secure cluster (we used MapR)
> 2. Install Hive and Drill services
> 3. Configure drill impersonation and authentication
> 4. Restart hivemeta service
> 5. Connect to drill and execute query involving hive storage, issue occurs
> 6. Restart the drill-bits services and execute the query, issue is no longer 
> hit
> The problem occurs in the same place as the earlier fix, but might represent 
> a slightly different use case: in this case the connection is secure.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5481) Allow Drill to persist profiles in-memory only with a max capacity

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5481:
---
Labels: ready-to-commit  (was: )

> Allow Drill to persist profiles in-memory only with a max capacity
> --
>
> Key: DRILL-5481
> URL: https://issues.apache.org/jira/browse/DRILL-5481
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
>
> To allow for fast persistence of profiles on a temporary basis (i.e. till the 
> life of the Drillbit), an existing test class 
> {{org.apache.drill.exec.testing.store.NoWriteLocalStore.java}} was refactored 
> to {{org.apache.drill.exec.store.sys.store.EphemeralPersistentStore}} and 
> given the ability to maintain a max capacity.
> This should allow query profiles to be available for as long as the Drillbit 
> process' lifespan.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5516) Limit memory usage for Hbase reader

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5516:
---
Labels: ready-to-commit  (was: )

> Limit memory usage for Hbase reader
> ---
>
> Key: DRILL-5516
> URL: https://issues.apache.org/jira/browse/DRILL-5516
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - HBase
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> If early limit 0 optimization is set to true (alter session set 
> `planner.enable_limit0_optimization` = true), when executing limit 0 queries 
> Drill will return data type from available metadata if possible.
> When Drill can not determine data types from metadata (or if early limit 0 
> optimization is set to false), Drill will read first batch of data and 
> determine schema.
> Hbase reader determines max batch size using magic number (4000) which can 
> lead to OOM when row size is large. The overall vector/batch size issue will 
> be reconsidered in future releases.This is temporary fix to avoid OOM.
> To limit memory usage for Hbase reader we are adding max allowed allocated 
> memory contant which will default to 64 mb. Thus batch size will be limited 
> to 4000 (as before if memory limit does not exceed) or to number of records 
> that are within max allowed memory limit. If first row in batch is larger 
> than allowed default, it will be written in batch but batch will contain only 
> this row.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-4335) Apache Drill should support network encryption

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-4335:
---
Labels: ready-to-commit security  (was: security)

> Apache Drill should support network encryption
> --
>
> Key: DRILL-4335
> URL: https://issues.apache.org/jira/browse/DRILL-4335
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Keys Botzum
>Assignee: Sorabh Hamirwasia
>  Labels: ready-to-commit, security
> Attachments: ApacheDrillEncryptionUsingSASLDesign.pdf
>
>
> This is clearly related to Drill-291 but wanted to make explicit that this 
> needs to include network level encryption and not just authentication. This 
> is particularly important for the client connection to Drill which will often 
> be sending passwords in the clear until there is encryption.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5431) Support SSL

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5431:
--

Assignee: Sudheesh Katkam

> Support SSL
> ---
>
> Key: DRILL-5431
> URL: https://issues.apache.org/jira/browse/DRILL-5431
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Client - Java, Client - ODBC
>Reporter: Sudheesh Katkam
>Assignee: Sudheesh Katkam
>
> Support SSL between Drillbit and JDBC/ODBC drivers. Drill already supports 
> HTTPS for web traffic.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (DRILL-5322) Provide an OperatorFixture for sub-operator unit testing setup

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam resolved DRILL-5322.

Resolution: Fixed

Fixed as part of DRILL-5318.

> Provide an OperatorFixture for sub-operator unit testing setup
> --
>
> Key: DRILL-5322
> URL: https://issues.apache.org/jira/browse/DRILL-5322
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> We recently created various "fixture" classes to assist with system-level 
> testing: {{LogFixture}}, {{ClusterFixture}} and {{ClientFixture}}. Each 
> handles the tedious work of setting up the conditions to run certain kinds of 
> tests.
> In the same way, we need an {{OperatorFixture}} to set up the low-level bits 
> and pieces needed for operator-level, and sub-operator-level unit testing.
> The {{DrillConfig}} is used by both the system-level and operator-level 
> fixtures. So, pull the config-setup tasks our of (cluster) {{FixtureBuilder}} 
> (should rename) and into a new {{ConfigBuilder}}. Leave the existing methods 
> in {{FixtureBuilder}}, but modify them to be wrappers around the new config 
> builder.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (DRILL-5320) Refactor OptionManager for unit testing

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam resolved DRILL-5320.

Resolution: Fixed

See DRILL-5319.

> Refactor OptionManager for unit testing
> ---
>
> Key: DRILL-5320
> URL: https://issues.apache.org/jira/browse/DRILL-5320
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> The {{OptionManager}} interface serves two purposes:
> * Create and modify options
> * Access option values
> The implementations  of this class are integrated with the rest of Drill, 
> making it difficult to use the classes in isolation in unit testing. Further, 
> since operators are given the full interface, the operator has the ability to 
> modify options, and so each unit test should either verify that no 
> modification is, in fact, done, or must track down modifications and test 
> them.
> For operator and sub-operator unit tests we need a simpler interface. As it 
> turns out, most low-level uses of {{OptionManager}} are all read-only. This 
> allows a simple refactoring to enhance unit testability: create a new 
> super-interface {{OptionSet}}, which provides only the read-only methods.
> Then, refactor low-level classes (code generation, compilers, and so on) to 
> use the restricted {{OptionSet}} interface.
> Finally, for unit tests, create a trivial, map-based implementation that can 
> be populated as needed for each specific test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (DRILL-5321) Refactor FragmentContext for unit testing

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam resolved DRILL-5321.

Resolution: Fixed

See DRILL-5319.

> Refactor FragmentContext for unit testing
> -
>
> Key: DRILL-5321
> URL: https://issues.apache.org/jira/browse/DRILL-5321
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.11.0
>
>
> Each operator has visibility to the {{FragmentContext}} class. 
> {{FragmentContext}} provides access to all of Drill internals: the Drillbit 
> context, the network interfaces, RPC messages and so on.
> Further, all the code generation mechanisms require a {{FragmentContext}} 
> object.
> This structure creates a large barrier to unit testing. To test, say, a 
> particular bit of generated code, we must have the entire Drillbit running so 
> we can obtain a {{FragmentContext}}. Clearly, this is less than ideal.
> Upon inspection, it turns out that the {{FragmentContext}} is mostly needed, 
> by many operators, to generate code. Of the many methods in 
> {{FragmentContext}}, code generation uses only six.
> The solution is to create a new super-interface, {{CodeGenContext}}, which 
> holds those six methods. The {{CodeGenContext}} can be easily re-implemented 
> for unit testing.
> Then, modify all the code-generation classes that currently take 
> {{FragmentContext}} to take {{CodeGenContext}} instead.
> Since {{FragmentContext}} derives from {{CodeGenContext}}, existing operator 
> code "just works."



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5323) Provide test tools to create, populate and compare row sets

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5323:
---
Labels: ready-to-commit  (was: )

> Provide test tools to create, populate and compare row sets
> ---
>
> Key: DRILL-5323
> URL: https://issues.apache.org/jira/browse/DRILL-5323
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> Operators work with individual row sets. A row set is a collection of records 
> stored as column vectors. (Drill uses various terms for this concept. A 
> record batch is a row set with an operator implementation wrapped around it. 
> A vector container is a row set, but with much functionality left as an 
> exercise for the developer. And so on.)
> To simplify tests, we need a {{TestRowSet}} concept that wraps a 
> {{VectorContainer}} and provides easy ways to:
> * Define a schema for the row set.
> * Create a set of vectors that implement the schema.
> * Populate the row set with test data via code.
> * Add an SV2 to the row set.
> * Pass the row set to operator components (such as generated code blocks.)
> * Compare the results of the operation with an expected result set.
> * Dispose of the underling direct memory when work is done.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5405) Add missing operator types

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5405:
---
Labels:   (was: ready-to-commit)

> Add missing operator types
> --
>
> Key: DRILL-5405
> URL: https://issues.apache.org/jira/browse/DRILL-5405
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.11.0
>
> Attachments: maprdb_sub_scan.JPG, unknown_operator.JPG
>
>
> Add missing operator types: FLATTEN, MONGO_SUB_SCAN, MAPRDB_SUB_SCAN so they 
> won't be displayed on Web UI as UNKNOWN_OPERATOR.
> Example:
> before the fix -> unknown_operator.JPG
> after the fix -> maprdb_sub_scan.JPG



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam resolved DRILL-5433.

   Resolution: Not A Problem
Fix Version/s: (was: 1.10.0)

Closing as "not a problem" because the issues are system configuration related.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
> ... 21 more
> Caused by: javax.security.sasl.SaslException: Server requires authentication 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971812#comment-15971812
 ] 

Sudheesh Katkam commented on DRILL-5433:


Notice the error: *"unsupported key type found the default TGT: 18"* and 
[google results|https://goo.gl/4Dt3gx] show various reasons related to 
configuration.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
> ... 21 more

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971784#comment-15971784
 ] 

Sudheesh Katkam commented on DRILL-5433:


I am not sure how Kerberos is affected by a change in Java version. [Google 
search|https://goo.gl/75GIvU] suggests renewing ticket ({{kinit -R}}). If this 
does not work, re-enable debug logging, and check the error?

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971691#comment-15971691
 ] 

Sudheesh Katkam commented on DRILL-5433:


This comment is incorrect:
{quote}
Since you already have JDK 1.7, check JAVA_HOME env variable, and set it 
correctly, and restart Drill.
{quote}
[Install JDK 7|http://openjdk.java.net/install/] and then set JAVA_HOME; looks 
like the system is running JRE.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971552#comment-15971552
 ] 

Sudheesh Katkam commented on DRILL-5433:


>From the trace, it looks like Kerberos authentication succeeded :) You should 
>disable debug options now.

Drill uses JDK and Janino to compile code generated at runtime (different 
compilers for different code sizes). So the other error is saying JDK is 
missing. For now you could try changing the [compiler 
policy|https://drill.apache.org/docs/configuration-options-introduction/#system-options]
 for the session:
{code}
> set `exec.java_compiler` = 'JANINO';
...
> show databases;
...
{code}

Since you already have JDK 1.7, check JAVA_HOME env variable, and set it 
correctly, and restart Drill.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971386#comment-15971386
 ] 

Sudheesh Katkam commented on DRILL-5433:


You can try a different Kerberos config for Drill using {{_host}} in the 
service principal name, which is very similar to the Hive config you provided 
(documented as " single principal per node" in the 
[documentation|http://drill.apache.org/docs/configuring-kerberos-authentication/]).
 Change drill-override.conf to:
{code}
drill.exec: {
  cluster-id: "drillbits1",
  zk.connect: "host1.fqdn:2181,host2.fqdn:2181,host3.fqdn:2181",
  security: {
  user.auth.enabled: true,
  user.auth.impl: "pam",
  user.auth.pam_profiles: [ "sudo", "login" ],
  packages += "org.apache.drill.exec.rpc.user.security",
  auth.mechanisms: ["KERBEROS","PLAIN"],
  auth.principal: "drill/_h...@lab.com",
  auth.keytab: "/opt/drill/.keytab/drill.keytab"
}
}
{code}
Notice the change in {{auth.principal}}. For this, please ensure that 
principals for each drillbit exist in Kerberos (with _host replaced by the FQDN 
of that drillbit).

After restarting the drillbits, you can connect as the "test" user (after 
obtaining the TGT) using the following commands:
{code}
> sqlline -u "jdbc:drill:zk=host1.fqdn;auth=kerberos"
...
> sqlline -u "jdbc:drill:drillbit=host1.fqdn;auth=kerberos"
...
{code}
Notice that Drill service principal is not be specified in the connection 
parameters (the value is deduced: the service name by default is "drill", and 
hostname in the first case is "host1.fqdn", and in the second case, zookeeper 
provides the hostnames).

I am not sure about the hive storage plugin config; this seems like a different 
issue. And AFAIK Drill was not tested on Hortonworks Kerberos environment.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968269#comment-15968269
 ] 

Sudheesh Katkam commented on DRILL-5433:


Can you share relevant portion of Hive config? AFAIK there is no requirement to 
run as user "drill".

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
> ... 21 more
> Caused by: javax.security.sasl.SaslException: Server 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968250#comment-15968250
 ] 

Sudheesh Katkam commented on DRILL-5433:


That's seems correct to me.

Do you have other services that are configured with Kerberos and working 
correctly? Any Hadoop ecosystem components?

I am not an expert in Kerberos, and so I am not sure what exactly the 
problem/solution is. I googled the error: *Server not found in Kerberos 
database (7) - UNKNOWN_SERVER* and each results provides a different solution, 
but the overall theme is around Kerberos configuration ([reverse name 
resolution|https://serverfault.com/questions/376762/nfs-with-kerberos-mount-failing-due-to-server-not-found-in-kerberos-database],
 [DNS-related Error 
Messages|https://technet.microsoft.com/en-us/library/bb463167.aspx], [duplicate 
SPNs|https://sourceforge.net/p/spnego/discussion/1003769/thread/67c0183c/], 
[others|http://thejavamonkey.blogspot.com/2008/04/clientserver-hello-world-in-kerberos.html],
 etc.) You may want to look into those options, and the debug should help you 
more. If you do figure things out, please post your findings here.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 6:42 PM:
-

I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the keytab file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}


was (Author: sudheeshkatkam):
I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 6:26 PM:
-

I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess is that the permissions on 
the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}


was (Author: sudheeshkatkam):
I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess would be that the permissions 
on the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968007#comment-15968007
 ] 

Sudheesh Katkam commented on DRILL-5433:


I am not exactly sure why the first scenario isn't working. Kerberos error 
messages are usually cryptic. You may want to do further research on the error 
(*Unable to obtain password from user*). My guess would be that the permissions 
on the file are incorrect; if so, [correct 
that|https://help.ubuntu.com/community/Kerberos#Security].

Regarding the second scenario, as I mentioned, the user is expected to have a 
TGT, i.e. after logging in as "test", you should run {{kinit}} to get a TGT.
{code}
host1:/var/log # su - test
test@host1:/home/test> kinit
...
test@host1:/home/test> sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
...
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 5:22 PM:
-

Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

In either case, service principal (for drill) is specified using {{principal=}}.


was (Author: sudheeshkatkam):
Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/13/17 5:20 PM:
-

Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}. In this case, the user is the service itself.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}


was (Author: sudheeshkatkam):
Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967915#comment-15967915
 ] 

Sudheesh Katkam commented on DRILL-5433:


Sorry, my comment wasn't clear.

+ To login as the service itself ({{drill/lad...@lab.com}}), can you try this? 
This command uses keytab credentials. If keytab for the user principal is 
specified using {{keytab=}}, then the user principal needs to be specified 
using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

+ To login as the "test" user (maybe {{t...@lab.com}}), do what was initially 
done. This command uses ticket credentials. In this case (typical use case), 
the assumption is that there is a TGT (e.g. user had invoked "kinit"). The TGT 
contains the user principal, so the command need not explicitly specify user 
principal using {{-n}}.
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967865#comment-15967865
 ] 

Sudheesh Katkam commented on DRILL-5433:


I am not sure if what you are testing occurs in a typical Kerberos deployment 
i.e. a service trying to getting a ticket for itself ({{... 
drill/lab...@lab.com for drill/lad...@lab.com ...}}).

Usually sqlline script is invoked as another user (e.g. {{b...@lab.com}}). Can 
you try this?
{code}
sqlline -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com;keytab=/opt/drill/.keytab/drill.keytab"
 -n "drill/lad...@lab.com"
{code}

This uses keytab instead of ticket as credential during login.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966640#comment-15966640
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/12/17 9:22 PM:
-

The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

There will be more info printed to the console. You could also look into 
Kerberos logs; logging configuration is in */etc/krb5.conf* file.

(Also make sure you don't comment any sensitive info here.)


was (Author: sudheeshkatkam):
The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

The console will now print more information. You could also look into Kerberos 
logs; logging configuration is in */etc/krb5.conf* file.

(Also make sure you don't comment any sensitive info here.)

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966640#comment-15966640
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/12/17 9:21 PM:
-

The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

The console will now print more information. You could also look into Kerberos 
logs; logging configuration is in */etc/krb5.conf* file.

(Also make sure you don't comment any sensitive info here.)


was (Author: sudheeshkatkam):
The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. And there is more info 
to the console was well. You could also look into Kerberos logs; logging 
configuration is in */etc/krb5.conf* file.

(Also make sure you don't comment any sensitive info here.)

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966640#comment-15966640
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/12/17 9:18 PM:
-

The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. And there is more info 
to the console was well. You could also look into Kerberos logs; logging 
configuration is in */etc/krb5.conf* file.

(Also make sure you don't comment any sensitive info here.)


was (Author: sudheeshkatkam):
The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. And there is more info 
to the console was well. You could also look into Kerberos logs; logging 
configuration is in */etc/krb5.conf* file.

(Also make sure you don't commit with any sensitive info here.)

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at 

[jira] [Comment Edited] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966640#comment-15966640
 ] 

Sudheesh Katkam edited comment on DRILL-5433 at 4/12/17 9:18 PM:
-

The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. And there is more info 
to the console was well. You could also look into Kerberos logs; logging 
configuration is in */etc/krb5.conf* file.

(Also make sure you don't commit with any sensitive info here.)


was (Author: sudheeshkatkam):
The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. You could also look 
into Kerberos logs; logging configuration is in */etc/krb5.conf* file.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966640#comment-15966640
 ] 

Sudheesh Katkam commented on DRILL-5433:


The error says: *Server not found in Kerberos database (7) - UNKNOWN_SERVER* 
This needs to be investigated.

Some pointers for more logging:
+ Create a file named */tmp/java.logging.txt* with contents:
{code}
javax.security.sasl.level=FINEST
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
{code}

+ Edit $DRILL_HOME/conf/drill-env.sh: 
{code}
export SQLLINE_JAVA_OPTS = "$SQLLINE_JAVA_OPTS -Dsun.security.jgss.debug=true 
-Dsun.security.krb5.debug=true 
-Djava.util.logging.config.file=/tmp/java.logging.txt"
{code}

Maybe the */tmp/java.logging.txt* will have information. You could also look 
into Kerberos logs; logging configuration is in */etc/krb5.conf* file.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966604#comment-15966604
 ] 

Sudheesh Katkam commented on DRILL-5433:


What's the error why you try this?
{code}
sqlline --maxWidth=1 -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

I would guess the error is a slight different message.

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> {code}
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> 

[jira] [Updated] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5433:
---
Description: 
I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
I'm getting below error while running "drill-conf" or sqlline as user "drill" 
which is configured in the "drill-override.conf" file. 

{code}
drill@host:/opt/drill/bin>  drill-conf
Error: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed: Server requires 
authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
javax.security.sasl.SaslException: Server requires authentication using 
[kerberos, plain]. Insufficient credentials?] (state=,code=0)
java.sql.SQLException: Failure in connecting to Drill: 
org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed: Server requires 
authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
javax.security.sasl.SaslException: Server requires authentication using 
[kerberos, plain]. Insufficient credentials?]
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
javax.security.sasl.SaslException: Authentication failed: Server requires 
authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
javax.security.sasl.SaslException: Server requires authentication using 
[kerberos, plain]. Insufficient credentials?]
at 
org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
at 
org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
at 
org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
... 18 more
Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
requires authentication using [kerberos, plain]. Insufficient credentials? 
[Caused by javax.security.sasl.SaslException: Server requires authentication 
using [kerberos, plain]. Insufficient credentials?]
at 
org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
at 
org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
at 
com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
at 
org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
... 21 more
Caused by: javax.security.sasl.SaslException: Server requires authentication 
using [kerberos, plain]. Insufficient credentials?
at 
org.apache.drill.exec.rpc.user.UserClient.getAuthenticatorFactory(UserClient.java:285)
at 
org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:216)
... 22 more
apache drill 1.10.0
"this isn't your grandfather's sql"
{code}
Same error when running below command:
{code}
sqlline --maxWidth=1 -u 
"jdbc:drill:drillbit=host1.fqdn;auth=kerberos;principal=drill/lad...@lab.com"
{code}

"Drill" user has has valid keytab/ticket.
The Drill UI is working fine with local authentication.

drill-override.conf file:
{code}
drill.exec: {
  cluster-id: "drillbits1",
  zk.connect: "host1.fqdn:2181,host2.fqdn:2181,host3.fqdn:2181",

  security: {
  user.auth.enabled: true,
  user.auth.impl: "pam",
  user.auth.pam_profiles: [ "sudo", "login" ],
  packages += "org.apache.drill.exec.rpc.user.security",
  

[jira] [Commented] (DRILL-5433) Authentication failed: Server requires authentication using [kerberos, plain]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966568#comment-15966568
 ] 

Sudheesh Katkam commented on DRILL-5433:


Can you check the error in drillbit.log file (by default in $DRILL_HOME/log 
directory)?

> Authentication failed: Server requires authentication using [kerberos, plain]
> -
>
> Key: DRILL-5433
> URL: https://issues.apache.org/jira/browse/DRILL-5433
> Project: Apache Drill
>  Issue Type: Task
>  Components: Functions - Drill
>Affects Versions: 1.10.0
> Environment: OS: Redhat Linux 6.7, HDP 2.5.3, Kerberos enabled, 
> Hardware: VmWare
>Reporter: Parag Darji
>Priority: Minor
>  Labels: newbie, security
> Fix For: 1.10.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I've setup Apace drill 1.10.0 on RHEL 6.7, HDP 2.5.3, kerberos enabled
> I'm getting below error while running "drill-conf" or sqlline as user "drill" 
> which is configured in the "drill-override.conf" file. 
> drill@host:/opt/drill/bin>  drill-conf
> Error: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?] (state=,code=0)
> java.sql.SQLException: Failure in connecting to Drill: 
> org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:166)
> at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
> at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at 
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
> at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
> at sqlline.Commands.connect(Commands.java:1083)
> at sqlline.Commands.connect(Commands.java:1015)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
> at sqlline.SqlLine.dispatch(SqlLine.java:742)
> at sqlline.SqlLine.initArgs(SqlLine.java:528)
> at sqlline.SqlLine.begin(SqlLine.java:596)
> at sqlline.SqlLine.start(SqlLine.java:375)
> at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: 
> javax.security.sasl.SaslException: Authentication failed: Server requires 
> authentication using [kerberos, plain]. Insufficient credentials? [Caused by 
> javax.security.sasl.SaslException: Server requires authentication using 
> [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:157)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
> at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
> at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:157)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Authentication failed: Server 
> requires authentication using [kerberos, plain]. Insufficient credentials? 
> [Caused by javax.security.sasl.SaslException: Server requires authentication 
> using [kerberos, plain]. Insufficient credentials?]
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:204)
> at 
> org.apache.drill.exec.rpc.user.UserClient$3.mapException(UserClient.java:197)
> at 
> com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
> at 
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:155)
> ... 21 more
> Caused by: javax.security.sasl.SaslException: Server requires authentication 
> 

[jira] [Created] (DRILL-5431) Support SSL

[Sudheesh Katkam (JIRA)]

Sudheesh Katkam created DRILL-5431:
--

 Summary: Support SSL
 Key: DRILL-5431
 URL: https://issues.apache.org/jira/browse/DRILL-5431
 Project: Apache Drill
  Issue Type: New Feature
  Components: Client - Java, Client - ODBC
Reporter: Sudheesh Katkam


Support SSL between Drillbit and JDBC/ODBC drivers. Drill already supports 
HTTPS for web traffic.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (DRILL-4280) Kerberos Authentication

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963218#comment-15963218
 ] 

Sudheesh Katkam edited comment on DRILL-4280 at 4/10/17 5:20 PM:
-

Hi [~djc0604], Kerberos SPNEGO support was not included as part of resolving 
this ticket, and so Drill cannot be enabled only with Kerberos if web server is 
required. PLAIN also needs to be enabled. Accordingly, I opened DRILL-5425.


was (Author: sudheeshkatkam):
Hi [~djc0604], Kerberos SPNEGO support was not included as part of this ticket, 
and so Drill cannot be enabled only with Kerberos if web server is required. 
PLAIN also needs to be enabled. Accordingly, I opened DRILL-5425.

> Kerberos Authentication
> ---
>
> Key: DRILL-4280
> URL: https://issues.apache.org/jira/browse/DRILL-4280
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Keys Botzum
>Assignee: Sudheesh Katkam
>  Labels: security
> Fix For: 1.10.0
>
>
> Drill should support Kerberos based authentication from clients. This means 
> that both the ODBC and JDBC drivers as well as the web/REST interfaces should 
> support inbound Kerberos. For Web this would most likely be SPNEGO while for 
> ODBC and JDBC this will be more generic Kerberos.
> Since Hive and much of Hadoop supports Kerberos there is a potential for a 
> lot of reuse of ideas if not implementation.
> Note that this is related to but not the same as 
> https://issues.apache.org/jira/browse/DRILL-3584 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (DRILL-4280) Kerberos Authentication

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963218#comment-15963218
 ] 

Sudheesh Katkam edited comment on DRILL-4280 at 4/10/17 5:20 PM:
-

Hi [~djc0604], Kerberos SPNEGO support was not included as part of the patch, 
and so Drill cannot be enabled only with Kerberos if web server is required. 
PLAIN also needs to be enabled. Accordingly, I opened DRILL-5425.


was (Author: sudheeshkatkam):
Hi [~djc0604], Kerberos SPNEGO support was not included as part of resolving 
this ticket, and so Drill cannot be enabled only with Kerberos if web server is 
required. PLAIN also needs to be enabled. Accordingly, I opened DRILL-5425.

> Kerberos Authentication
> ---
>
> Key: DRILL-4280
> URL: https://issues.apache.org/jira/browse/DRILL-4280
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Keys Botzum
>Assignee: Sudheesh Katkam
>  Labels: security
> Fix For: 1.10.0
>
>
> Drill should support Kerberos based authentication from clients. This means 
> that both the ODBC and JDBC drivers as well as the web/REST interfaces should 
> support inbound Kerberos. For Web this would most likely be SPNEGO while for 
> ODBC and JDBC this will be more generic Kerberos.
> Since Hive and much of Hadoop supports Kerberos there is a potential for a 
> lot of reuse of ideas if not implementation.
> Note that this is related to but not the same as 
> https://issues.apache.org/jira/browse/DRILL-3584 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-4280) Kerberos Authentication

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963218#comment-15963218
 ] 

Sudheesh Katkam commented on DRILL-4280:


Hi [~djc0604], Kerberos SPNEGO support was not included as part of this ticket, 
and so Drill cannot be enabled only with Kerberos if web server is required. 
PLAIN also needs to be enabled. Accordingly, I opened DRILL-5425.

> Kerberos Authentication
> ---
>
> Key: DRILL-4280
> URL: https://issues.apache.org/jira/browse/DRILL-4280
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Keys Botzum
>Assignee: Sudheesh Katkam
>  Labels: security
> Fix For: 1.10.0
>
>
> Drill should support Kerberos based authentication from clients. This means 
> that both the ODBC and JDBC drivers as well as the web/REST interfaces should 
> support inbound Kerberos. For Web this would most likely be SPNEGO while for 
> ODBC and JDBC this will be more generic Kerberos.
> Since Hive and much of Hadoop supports Kerberos there is a potential for a 
> lot of reuse of ideas if not implementation.
> Note that this is related to but not the same as 
> https://issues.apache.org/jira/browse/DRILL-3584 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO

[Sudheesh Katkam (JIRA)]

Sudheesh Katkam created DRILL-5425:
--

 Summary: Support HTTP Kerberos auth using SPNEGO
 Key: DRILL-5425
 URL: https://issues.apache.org/jira/browse/DRILL-5425
 Project: Apache Drill
  Issue Type: New Feature
  Components: Web Server
Reporter: Sudheesh Katkam


DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests to 
add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for HTTP 
connections.

This requires creating "direct" web sessions; currently web sessions are 
sessions over Java client sessions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5387) TestBitBitKerberos and TestUserBitKerberos cause sporadic unit test failures

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5387:
--

Assignee: Arina Ielchiieva  (was: Sudheesh Katkam)

> TestBitBitKerberos and TestUserBitKerberos cause sporadic unit test failures
> 
>
> Key: DRILL-5387
> URL: https://issues.apache.org/jira/browse/DRILL-5387
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Sudheesh Katkam
>Assignee: Arina Ielchiieva
>
> TestOptionsAuthEnabled and TestInboundImpersonation sporadically fail. There 
> is a [Java 
> trick|https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/hadoop/security/UgiTestUtil.java#L29]
>  to reset some static state in TestUserBitKerberos and TestBitBitKerberos to 
> ensure the JVM is reusable for other tests as done in the [Hadoop auth 
> tests|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithMiniKdc.java#L53],
>  but this trick does not seem to work always. So disable these tests. In the 
> future, maybe the tests can be run separately through surefire but not as 
> part of the default build?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5315) Small Comment Typo in drillClient.hpp

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5315:
--

Assignee: Sudheesh Katkam

> Small Comment Typo in drillClient.hpp
> -
>
> Key: DRILL-5315
> URL: https://issues.apache.org/jira/browse/DRILL-5315
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - C++
>Affects Versions: 1.7.0
>Reporter: Rob Wu
>Assignee: Sudheesh Katkam
>Priority: Trivial
>  Labels: ready-to-commit
>
> Small typo of the word "properties" at:
> https://github.com/apache/drill/blob/master/contrib/native/client/src/include/drill/drillClient.hpp#L1280



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5355) Misc. code cleanup

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5355:
---
Labels: ready-to-commit  (was: )

> Misc. code cleanup 
> ---
>
> Key: DRILL-5355
> URL: https://issues.apache.org/jira/browse/DRILL-5355
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> Recent work in Drill has identified a number of cases where code can be 
> cleaned up: adding missing annotations, etc. These changes don't fit as part 
> of a separate ticket and so are rolled up into a this "general hygiene" 
> ticket.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (DRILL-5387) TestBitBitKerberos and TestUserBitKerberos cause sporadic unit test failures

[Sudheesh Katkam (JIRA)]

Sudheesh Katkam created DRILL-5387:
--

 Summary: TestBitBitKerberos and TestUserBitKerberos cause sporadic 
unit test failures
 Key: DRILL-5387
 URL: https://issues.apache.org/jira/browse/DRILL-5387
 Project: Apache Drill
  Issue Type: Bug
Reporter: Sudheesh Katkam
Assignee: Sudheesh Katkam


TestOptionsAuthEnabled and TestInboundImpersonation sporadically fail. There is 
a [Java 
trick|https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/hadoop/security/UgiTestUtil.java#L29]
 to reset some static state in TestUserBitKerberos and TestBitBitKerberos to 
ensure the JVM is reusable for other tests as done in the [Hadoop auth 
tests|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithMiniKdc.java#L53],
 but this trick does not seem to work always. So disable these tests. In the 
future, maybe the tests can be run separately through surefire but not as part 
of the default build?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-3510:
---
Labels: doc-impacting ready-to-commit  (was: doc-impacting)

> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL 
> identifiers 
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as 
> what MySQL does. However, this is different from ANSI SQL specification, 
> where double quote is used as identifier quotes.  
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user. 
> Drill should follow the same way, so that Drill users do not have to rewrite 
> their existing queries, if their queries use double quotes. 
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5368) Memory leak in C++ server metadata handler

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5368:
---
Labels: ready-to-commit  (was: )

> Memory leak in C++ server metadata handler
> --
>
> Key: DRILL-5368
> URL: https://issues.apache.org/jira/browse/DRILL-5368
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Affects Versions: 1.10.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>  Labels: ready-to-commit
>
> When receiving server metadata response, a protobuf ServerMetaResp object is 
> dynamically allocated but never freed.
> Since for this handler, there's no need to keep the instance attached to the 
> handler (content is copied over by the MetaData class), a reference is enough 
> and allocation can be done on the stack.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5369) Missing initialization for ServerMetaContext

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5369:
---
Labels: ready-to-commit  (was: )

> Missing initialization for ServerMetaContext
> 
>
> Key: DRILL-5369
> URL: https://issues.apache.org/jira/browse/DRILL-5369
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
>
> {{ServerMetaContext}} is not initialized properly which might cause some 
> unexpected issues (like getMetadata() returning before receiving the answer) 
> in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5311) C++ connector connect doesn't check handshake result for timeout

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934095#comment-15934095
 ] 

Sudheesh Katkam commented on DRILL-5311:


The [framework|https://github.com/mapr/drill-test-framework] does not support 
ODBC/ C++ connector.

> C++ connector connect doesn't check handshake result for timeout
> 
>
> Key: DRILL-5311
> URL: https://issues.apache.org/jira/browse/DRILL-5311
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>Assignee: Sudheesh Katkam
>  Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> The C++ connector connect methods returns okay as soon as the tcp connection 
> is succesfully established between client and server, and the handshake 
> message is sent. However it doesn't wait for handshake to have completed.
> The consequence is that if handshake failed, the error is deferred to the 
> first query, which might be unexpected by the application.
> I believe that validateHanshake method in drillClientImpl should wait for the 
> handshake to complete, as it seems a bit more saner...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (DRILL-5311) C++ connector connect doesn't wait for handshake to complete

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam reassigned DRILL-5311:
--

Assignee: Sudheesh Katkam

> C++ connector connect doesn't wait for handshake to complete
> 
>
> Key: DRILL-5311
> URL: https://issues.apache.org/jira/browse/DRILL-5311
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>Assignee: Sudheesh Katkam
>  Labels: ready-to-commit
>
> The C++ connector connect methods returns okay as soon as the tcp connection 
> is succesfully established between client and server, and the handshake 
> message is sent. However it doesn't wait for handshake to have completed.
> The consequence is that if handshake failed, the error is deferred to the 
> first query, which might be unexpected by the application.
> I believe that validateHanshake method in drillClientImpl should wait for the 
> handshake to complete, as it seems a bit more saner...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5311) C++ connector connect doesn't wait for handshake to complete

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5311:
---
Labels: ready-to-commit  (was: )

> C++ connector connect doesn't wait for handshake to complete
> 
>
> Key: DRILL-5311
> URL: https://issues.apache.org/jira/browse/DRILL-5311
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>  Labels: ready-to-commit
>
> The C++ connector connect methods returns okay as soon as the tcp connection 
> is succesfully established between client and server, and the handshake 
> message is sent. However it doesn't wait for handshake to have completed.
> The consequence is that if handshake failed, the error is deferred to the 
> first query, which might be unexpected by the application.
> I believe that validateHanshake method in drillClientImpl should wait for the 
> handshake to complete, as it seems a bit more saner...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5313) C++ client build failure on linux

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5313:
---
Labels: ready-to-commit  (was: )

> C++ client build failure on linux
> -
>
> Key: DRILL-5313
> URL: https://issues.apache.org/jira/browse/DRILL-5313
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Affects Versions: 1.10
>Reporter: Sorabh Hamirwasia
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
>
> We are seeing below errors while building Drill C++ client on linux platform:
> [root@qa-node161 build]# make
> [  6%] Built target y2038
> [ 38%] Built target protomsgs
> [ 41%] Building CXX object 
> src/clientlib/CMakeFiles/drillClient.dir/drillClientImpl.cpp.o
> /root/drill/drill/contrib/native/client/src/clientlib/drillClientImpl.cpp: In 
> function ‘void Drill::updateLikeFilter(exec::user::LikeFilter&, const 
> std::string&)’:
> /root/drill/drill/contrib/native/client/src/clientlib/drillClientImpl.cpp:782:
>  error: ‘s_searchEscapeString’ is not a member of ‘Drill::meta::DrillMetadata’
> make[2]: *** [src/clientlib/CMakeFiles/drillClient.dir/drillClientImpl.cpp.o] 
> Error 1
> make[1]: *** [src/clientlib/CMakeFiles/drillClient.dir/all] Error 2
> make: *** [all] Error 2
> It looks to be related to one of Laurent's pull request below:
> https://github.com/apache/drill/pull/712



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5311) C++ connector connect doesn't wait for handshake to complete

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893674#comment-15893674
 ] 

Sudheesh Katkam commented on DRILL-5311:


I am surprised about this one, I thought DrillClientImpl#recvHandshake is 
blocking, specifically 
[here|https://github.com/apache/drill/blob/master/contrib/native/client/src/clientlib/drillClientImpl.cpp#L273]
 (as the log says).

> C++ connector connect doesn't wait for handshake to complete
> 
>
> Key: DRILL-5311
> URL: https://issues.apache.org/jira/browse/DRILL-5311
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>
> The C++ connector connect methods returns okay as soon as the tcp connection 
> is succesfully established between client and server, and the handshake 
> message is sent. However it doesn't wait for handshake to have completed.
> The consequence is that if handshake failed, the error is deferred to the 
> first query, which might be unexpected by the application.
> I believe that validateHanshake method in drillClientImpl should wait for the 
> handshake to complete, as it seems a bit more saner...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5290) Provide an option to build operator table once for built-in static functions and reuse it across queries.

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893658#comment-15893658
 ] 

Sudheesh Katkam commented on DRILL-5290:


Fixed in 
[2b5a6f0|https://github.com/apache/drill/commit/2b5a6f0b523859297f86298ef384cd146e425f72]

> Provide an option to build operator table once for built-in static functions 
> and reuse it across queries.
> -
>
> Key: DRILL-5290
> URL: https://issues.apache.org/jira/browse/DRILL-5290
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.9.0
>Reporter: Padma Penumarthy
>Assignee: Padma Penumarthy
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.10.0
>
>
> Currently, DrillOperatorTable which contains standard SQL operators and 
> functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets 
> built for each query as part of creating QueryContext. This is an expensive 
> operation ( ~30 msec to build) and allocates  ~2M on heap for each query. For 
> high throughput, low latency operational queries, we quickly run out of heap 
> memory, causing JVM hangs. Build operator table once during startup for 
> static built-in functions and save in DrillbitContext, so we can reuse it 
> across queries.
> Provide a system/session option to not use dynamic UDFs so we can use the 
> operator table saved in DrillbitContext and avoid building each time.
> *Please note, changes are adding new option exec.udf.use_dynamic which needs 
> to be documented.*



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893657#comment-15893657
 ] 

Sudheesh Katkam commented on DRILL-5287:


Fixed in 
[7ebb985|https://github.com/apache/drill/commit/7ebb985edc823692673a42276b4e2a80fd1f256c]

> Provide option to skip updates of ephemeral state changes in Zookeeper
> --
>
> Key: DRILL-5287
> URL: https://issues.apache.org/jira/browse/DRILL-5287
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Padma Penumarthy
>Assignee: Padma Penumarthy
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.10.0
>
>
> We put transient profiles in zookeeper and update state as query progresses 
> and changes states. It is observed that this adds latency of ~45msec for each 
> update in the query execution path. This gets even worse when high number of 
> concurrent queries are in progress. For concurrency=100, the average query 
> response time even for short queries  is 8 sec vs 0.2 sec with these updates 
> disabled. For short lived queries in a high-throughput scenario, it is of no 
> value to update state changes in zookeeper. We need an option to disable 
> these updates for short running operational queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5221) cancel message is delayed until queryid or data is received

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893656#comment-15893656
 ] 

Sudheesh Katkam commented on DRILL-5221:


Fixed in 
[20a374c|https://github.com/apache/drill/commit/20a374c5bfabb49f4c8b3144a5a8529d17ee03fd]

> cancel message is delayed until queryid or data is received
> ---
>
> Key: DRILL-5221
> URL: https://issues.apache.org/jira/browse/DRILL-5221
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - C++
>Affects Versions: 1.9.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> When user is calling the cancel method of the C++ client, the client wait for 
> a message from the server to reply back with a cancellation message.
> In case of queries taking a long time to return batch results, it means 
> cancellation won't be effective until the next batch is received, instead of 
> cancelling right away the query (assuming the query id has already been 
> received, which is generally the case).
> It seems this was foreseen by [~vkorukanti] in his initial patch 
> (https://github.com/vkorukanti/drill/commit/e0ef6349aac48de5828b6d725c2cf013905d18eb)
>  but was omitted when I backported it post metadata changes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5167) C++ connector does not set escape string for metadata search pattern

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893655#comment-15893655
 ] 

Sudheesh Katkam commented on DRILL-5167:


Fixed in 
[c81f588|https://github.com/apache/drill/commit/c81f58884e2cef279a7d60ea611b6a422a2eb56e]

> C++ connector does not set escape string for metadata search pattern
> 
>
> Key: DRILL-5167
> URL: https://issues.apache.org/jira/browse/DRILL-5167
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> C++ connector does not set the escape string for search pattern when doing 
> metadata operation (getCatalogs/getSchema/getTables/getColumns). It is 
> assumed to be '\\' as returned by DrillMetadata::getSearchEscapeString(), but 
> because this is not sent over the wire, the server will actually consider 
> that there's no escape character, and might return different or no result 
> compared to what has been requested.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5301) Add server metadata API

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893654#comment-15893654
 ] 

Sudheesh Katkam commented on DRILL-5301:


Fixed in 
[d2e0f41|https://github.com/apache/drill/commit/d2e0f415c9cbaf609805708586832b6771883f53]
 and 
[d3238b1|https://github.com/apache/drill/commit/d3238b1b2270533285025d69b35906506212f492]

> Add server metadata API
> ---
>
> Key: DRILL-5301
> URL: https://issues.apache.org/jira/browse/DRILL-5301
> Project: Apache Drill
>  Issue Type: Improvement
>  Components:  Server, Client - C++, Client - Java, Client - JDBC, 
> Client - ODBC
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> JDBC and ODBC clients exposes lots of metadata regarding server version and 
> support of various parts of the SQL standard.
> Currently the returned information is hardcoded in both clients/drivers which 
> means that the infomation returned is support as of the client version, not 
> the server version.
> Instead, a new method should be provided to the clients to query the actual 
> server support. Support on the client or the server should be optional (for 
> example a client should not use this API if the server doesn't support it and 
> fallback to default values).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-4730) Update JDBC DatabaseMetaData implementation to use new Metadata APIs

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893652#comment-15893652
 ] 

Sudheesh Katkam commented on DRILL-4730:


Fixed in 
[17f888d|https://github.com/apache/drill/commit/17f888d9058be2be8953cb1ea5b37297b7d2fef3]

> Update JDBC DatabaseMetaData implementation to use new Metadata APIs
> 
>
> Key: DRILL-4730
> URL: https://issues.apache.org/jira/browse/DRILL-4730
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Client - JDBC
>Reporter: Venki Korukanti
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> DRILL-4728 is going to add support for new metadata APIs. Replace the 
> INFORMATION_SCHEMA queries used to get the metadata with the new APIs 
> provided in Java client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5221) cancel message is delayed until queryid or data is received

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5221:
---
Labels: ready-to-commit  (was: )

> cancel message is delayed until queryid or data is received
> ---
>
> Key: DRILL-5221
> URL: https://issues.apache.org/jira/browse/DRILL-5221
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - C++
>Affects Versions: 1.9.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
>
> When user is calling the cancel method of the C++ client, the client wait for 
> a message from the server to reply back with a cancellation message.
> In case of queries taking a long time to return batch results, it means 
> cancellation won't be effective until the next batch is received, instead of 
> cancelling right away the query (assuming the query id has already been 
> received, which is generally the case).
> It seems this was foreseen by [~vkorukanti] in his initial patch 
> (https://github.com/vkorukanti/drill/commit/e0ef6349aac48de5828b6d725c2cf013905d18eb)
>  but was omitted when I backported it post metadata changes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5167) C++ connector does not set escape string for metadata search pattern

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5167:
---
Labels: ready-to-commit  (was: )

> C++ connector does not set escape string for metadata search pattern
> 
>
> Key: DRILL-5167
> URL: https://issues.apache.org/jira/browse/DRILL-5167
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>  Labels: ready-to-commit
>
> C++ connector does not set the escape string for search pattern when doing 
> metadata operation (getCatalogs/getSchema/getTables/getColumns). It is 
> assumed to be '\\' as returned by DrillMetadata::getSearchEscapeString(), but 
> because this is not sent over the wire, the server will actually consider 
> that there's no escape character, and might return different or no result 
> compared to what has been requested.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5301) Add server metadata API

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5301:
---
Labels: ready-to-commit  (was: )

> Add server metadata API
> ---
>
> Key: DRILL-5301
> URL: https://issues.apache.org/jira/browse/DRILL-5301
> Project: Apache Drill
>  Issue Type: Improvement
>  Components:  Server, Client - C++, Client - Java, Client - JDBC, 
> Client - ODBC
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
>
> JDBC and ODBC clients exposes lots of metadata regarding server version and 
> support of various parts of the SQL standard.
> Currently the returned information is hardcoded in both clients/drivers which 
> means that the infomation returned is support as of the client version, not 
> the server version.
> Instead, a new method should be provided to the clients to query the actual 
> server support. Support on the client or the server should be optional (for 
> example a client should not use this API if the server doesn't support it and 
> fallback to default values).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-4730) Update JDBC DatabaseMetaData implementation to use new Metadata APIs

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-4730:
---
Labels: ready-to-commit  (was: )

> Update JDBC DatabaseMetaData implementation to use new Metadata APIs
> 
>
> Key: DRILL-4730
> URL: https://issues.apache.org/jira/browse/DRILL-4730
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Client - JDBC
>Reporter: Venki Korukanti
>Assignee: Laurent Goujon
>  Labels: ready-to-commit
>
> DRILL-4728 is going to add support for new metadata APIs. Replace the 
> INFORMATION_SCHEMA queries used to get the metadata with the new APIs 
> provided in Java client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (DRILL-5088) Error when reading DBRef column

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam resolved DRILL-5088.

   Resolution: Fixed
Fix Version/s: 1.10.0

Fixed in 
[b892b99|https://github.com/apache/drill/commit/b892b997dfa0259550942f076b0afd89b27c9fdf]

> Error when reading DBRef column
> ---
>
> Key: DRILL-5088
> URL: https://issues.apache.org/jira/browse/DRILL-5088
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Data Types
> Environment: drill 1.9.0
> mongo 3.2
>Reporter: Guillaume Champion
>Assignee: Chunhui Shi
> Fix For: 1.10.0
>
>
> In a mongo database with DBRef, when a DBRef is inserted in the first line of 
> a mongo's collection drill query failed :
> {code}
> 0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
> Error: SYSTEM ERROR: CodecConfigurationException: Can't find a codec for 
> class com.mongodb.DBRef.
> {code}
> Simple example to reproduce:
> In mongo instance
> {code}
> db.contact2.drop();
> db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8938"), "account" 
> : DBRef("contact", ObjectId("999cbf116b69060001fd8611")) });
> {code}
> In drill :
> {code}
> 0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
> Error: SYSTEM ERROR: CodecConfigurationException: Can't find a codec for 
> class com.mongodb.DBRef.
> [Error Id: 2944d766-e483-4453-a706-3d481397b186 on Analytics-Biznet:31010] 
> (state=,code=0)
> {code}
> If the first line doesn't contain de DBRef, drill will querying correctly :
> In a mongo instance :
> {code}
> db.contact2.drop();
> db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8939") });
> db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8938"), "account" 
> : DBRef("contact", ObjectId("999cbf116b69060001fd8611")) });
> {code}
> In drill :
> {code}
> 0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
> +--+---+
> | _id  |account   
>  |
> +--+---+
> | {"$oid":"582081d96b69060001fd8939"}  | {"$id":{}}   
>  |
> | {"$oid":"582081d96b69060001fd8938"}  | 
> {"$ref":"contact","$id":{"$oid":"999cbf116b69060001fd8611"}}  |
> +--+---+
> 2 rows selected (0,563 seconds)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5114) Rationalize use of Logback logging in unit tests

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5114:
---
Labels:   (was: ready-to-commit)

> Rationalize use of Logback logging in unit tests
> 
>
> Key: DRILL-5114
> URL: https://issues.apache.org/jira/browse/DRILL-5114
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>
> Drill uses Logback as its logger. The logger is used in several to display 
> some test output. Test output is sent to stdout, rather than a log file. 
> Since Drill also uses Logback, that same configuration sends much Drill 
> logging output to stdout as well, cluttering test output.
> Logback requires that that one Logback config file (either logback.xml or 
> hogback-test.xml) exist on the class path. Tests store the config file in the 
> src/test/resources folder of each sub-project.
> These files set the default logging level to debug. While this setting is 
> fine when working with individual tests, the output is overwhelming for bulk 
> test runs.
> The first requested change is to set the default logging level to error.
> The existing config files are usually called "logback.xml." Change the name 
> of test files to "logback-test.xml" to make clear that they are, in fact, 
> test configs.
> The {{exec/java-exec/src/test/resources/logback.xml}} config file is a full 
> version of Drill's production config file. Replace this with a config 
> suitable for testing (that is, the same as other modules.)
> The java-exec project includes a production-like config file in its non-test 
> sources: {{exec/java-exec/src/main/resources/logback.xml}}. Remove this as it 
> is not needed. (Instead, rely on the one shipped in the distribution 
> subsystem, which is the one copied to the Drill distribution.)
> Since Logback complains bitterly (via many log messages) when it cannot find 
> a configuration file (and each sub-module must have its own test 
> configuration), add missing logging configuration files:
> * exec/memory/base/src/test/resources/logback-test.xml
> * logical/src/test/resources/logback-test.xml



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5190) Display planning and queued time for a query in its profile page

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5190:
---
Labels: ready-to-commit  (was: )

> Display planning and queued time for a query in its profile page
> 
>
> Key: DRILL-5190
> URL: https://issues.apache.org/jira/browse/DRILL-5190
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
>
> Currently, the Web UI does not display the time spent in planning for a query 
> in its profile page. The estimate needs to be done by seeing how late did the 
> earliest major fragment start. 
> As an additional enhancement, we can also track for the time spent by a query 
> in waiting in the queue and the actual execution time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5259) Allow listing a user-defined number of profiles

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5259:
---
Labels: ready-to-commit  (was: )

> Allow listing a user-defined number of profiles 
> 
>
> Key: DRILL-5259
> URL: https://issues.apache.org/jira/browse/DRILL-5259
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Trivial
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> Currently, the web UI only lists the last 100 profiles. 
> This count is currently hard coded. The proposed change would be to create an 
> option in drill-override.conf to provide a flexible default value, and also 
> an option within the UI (via optional parameter in the path). 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5242) The UI breaks when trying to render profiles having unknown metrics

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5242:
---
Labels: ready-to-commit  (was: )

> The UI breaks when trying to render profiles having unknown metrics
> ---
>
> Key: DRILL-5242
> URL: https://issues.apache.org/jira/browse/DRILL-5242
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> When profiles are generated using a fork of Drill that has introduced new 
> metrics, the server of the parent branch will fail in rendering the operator 
> metrics correctly. 
> The workaround should be to simply skip unknown metrics. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-4280) Kerberos Authentication

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870754#comment-15870754
 ] 

Sudheesh Katkam commented on DRILL-4280:


I am aiming for this to be part of the next release, which maybe soon, given 
the poll on the dev list recently. So comments sooner rather than later would 
be helpful (I am gonna be on-and-off on vacation after next week).

BTW thank you for your initial comments.

> Kerberos Authentication
> ---
>
> Key: DRILL-4280
> URL: https://issues.apache.org/jira/browse/DRILL-4280
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Keys Botzum
>Assignee: Sudheesh Katkam
>  Labels: security
>
> Drill should support Kerberos based authentication from clients. This means 
> that both the ODBC and JDBC drivers as well as the web/REST interfaces should 
> support inbound Kerberos. For Web this would most likely be SPNEGO while for 
> ODBC and JDBC this will be more generic Kerberos.
> Since Hive and much of Hadoop supports Kerberos there is a potential for a 
> lot of reuse of ideas if not implementation.
> Note that this is related to but not the same as 
> https://issues.apache.org/jira/browse/DRILL-3584 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5269) SYSTEM ERROR: JsonMappingException: No suitable constructor found for type [simple type, class org.apache.drill.exec.store.direct.DirectSubScan]

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870197#comment-15870197
 ] 

Sudheesh Katkam commented on DRILL-5269:


Can you share the query that caused the error?

> SYSTEM ERROR: JsonMappingException: No suitable constructor found for type 
> [simple type, class org.apache.drill.exec.store.direct.DirectSubScan]
> 
>
> Key: DRILL-5269
> URL: https://issues.apache.org/jira/browse/DRILL-5269
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Anas
>Priority: Critical
>
> I am a query that has nested joins. The query fails with the following 
> exception.
> {code}
> Message: SYSTEM ERROR: JsonMappingException: No suitable constructor found 
> for type [simple type, class 
> org.apache.drill.exec.store.direct.DirectSubScan]: can not instantiate from 
> JSON object (missing default constructor or creator, or perhaps need to 
> add/enable type information?)
>  at [Source: {
>   "pop" : "broadcast-sender",
>   "@id" : 0,
>   "receiver-major-fragment" : 1,
>   "child" : {
> "pop" : "project",
> "@id" : 1,
> "exprs" : [ {
>   "ref" : "`$f1`",
>   "expr" : "greater_than(`__measure__10`, 0) "
> } ],
> "child" : {
>   "pop" : "nested-loop-join",
>   "@id" : 2,
>   "left" : {
> "pop" : "selection-vector-remover",
> "@id" : 4,
> "child" : {
>   "pop" : "filter",
>   "@id" : 6,
>   "child" : {
> "pop" : "project",
> "@id" : 8,
> "exprs" : [ {
>   "ref" : "`$f0`",
>   "expr" : "`count`"
> } ],
> "child" : {
>   "pop" : "DirectSubScan",
>   "@id" : 10,
>   "initialAllocation" : 100,
>   "maxAllocation" : 100,
>   "reader" : [ {
> "count" : 9977
>   } ],
>   "cost" : 0.0
> },
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 20.0
>   },
>   "expr" : "greater_than(`$f0`, 0) ",
>   "initialAllocation" : 100,
>   "maxAllocation" : 100,
>   "cost" : 10.0
> },
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 10.0
>   },
>   "right" : {
> "pop" : "streaming-aggregate",
> "@id" : 3,
> "child" : {
>   "pop" : "hash-aggregate",
>   "@id" : 5,
>   "child" : {
> "pop" : "project",
> "@id" : 7,
> "exprs" : [ {
>   "ref" : "`$f0`",
>   "expr" : "cast( ( ( if (booleanOr(equal(`Age_Groups`, '45-54') 
> , equal(`Age_Groups`, '55+') )  ) then ('45+' )  else (`Age_Groups` )  end  ) 
>  ) as VARCHAR(65536) )"
> } ],
> "child" : {
>   "pop" : "parquet-row-group-scan",
>   "@id" : 9,
>   "userName" : "anas",
>   "storage" : {
> "type" : "file",
> "enabled" : true,
> "connection" : "file:///",
> "config" : null,
> "workspaces" : {
>   "root" : {
> "location" : "/",
> "writable" : false,
> "defaultInputFormat" : null
>   },
>   "tmp" : {
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : null
>   }
> },
> "formats" : {
>   "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
>   },
>   "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
>   },
>   "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
>   },
>   "parquet" : {
> "type" : "parquet"
>   },
>   "json" : {
> "type" : "json",
> "extensions" : [ "json" ]
>   },
>   "avro" : {
> "type" : "avro"
>   },
>   "sequencefile" : {
> "type" : "sequencefile",
> "extensions" : [ "seq" ]
>  

[jira] [Comment Edited] (DRILL-3584) Drill Kerberos HDFS Support / Documentation

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862111#comment-15862111
 ] 

Sudheesh Katkam edited comment on DRILL-3584 at 2/13/17 9:22 PM:
-

I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add appropriate 
config to drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable 'secure impersonation' in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.


was (Author: sudheeshkatkam):
I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add appropriate 
config to drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> ---
>
> Key: DRILL-3584
> URL: https://issues.apache.org/jira/browse/DRILL-3584
> Project: Apache Drill
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Hari Sekhon
>Priority: Critical
>  Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters 
> and it doesn't appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure 
> Hadoop 

[jira] [Comment Edited] (DRILL-3584) Drill Kerberos HDFS Support / Documentation

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862111#comment-15862111
 ] 

Sudheesh Katkam edited comment on DRILL-3584 at 2/13/17 9:22 PM:
-

I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add appropriate 
config drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.


was (Author: sudheeshkatkam):
I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add the following 
to drill-override.conf
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> ---
>
> Key: DRILL-3584
> URL: https://issues.apache.org/jira/browse/DRILL-3584
> Project: Apache Drill
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Hari Sekhon
>Priority: Critical
>  Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters 
> and it doesn't appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure 
> Hadoop clusters, specifically HDFS + 

[jira] [Comment Edited] (DRILL-3584) Drill Kerberos HDFS Support / Documentation

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862111#comment-15862111
 ] 

Sudheesh Katkam edited comment on DRILL-3584 at 2/13/17 9:22 PM:
-

I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add appropriate 
config to drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.


was (Author: sudheeshkatkam):
I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add appropriate 
config drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> ---
>
> Key: DRILL-3584
> URL: https://issues.apache.org/jira/browse/DRILL-3584
> Project: Apache Drill
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Hari Sekhon
>Priority: Critical
>  Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters 
> and it doesn't appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure 
> Hadoop clusters, 

[jira] [Commented] (DRILL-3584) Drill Kerberos HDFS Support / Documentation

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862111#comment-15862111
 ] 

Sudheesh Katkam commented on DRILL-3584:


I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for 
Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) 
authentication, but I have not tested them since I do not have the 
infrastructure to setup KDC or HDFS (and not possible to write a unit test for 
this, MiniDFS and Drill use the same auth library which has static variables 
that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the 
drillbit can be started with a Kerberos principal and keytab. Add the following 
to drill-override.conf
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details 
here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
 Ensure "drill" as a [user 
identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
 exists in HDFS (Drill uses HDFS for [Dynamic 
UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will 
login to KDC, and when accessing HDFS, Drill uses the configured Kerberos 
credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as 
"drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details 
here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will 
allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For 
example, if "drill" is authorized to impersonate "bob", then while accessing 
the HDFS, access rights are checked for "bob" and authentication credentials of 
"drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> ---
>
> Key: DRILL-3584
> URL: https://issues.apache.org/jira/browse/DRILL-3584
> Project: Apache Drill
>  Issue Type: New Feature
>Affects Versions: 1.1.0
>Reporter: Hari Sekhon
>Priority: Critical
>  Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters 
> and it doesn't appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure 
> Hadoop clusters, specifically HDFS + Kerberos (plus obviously secure 
> kerberized Hive/HCatalog etc.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-3214) Config option to cast empty string to null does not cast empty string to null

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860250#comment-15860250
 ] 

Sudheesh Katkam commented on DRILL-3214:


Hi, would the "alter system" suggestion help in your case?

As in try: 
{code}
> alter system set `drill.exec.functions.cast_empty_string_to_null` = true;
{code}

and then the query.

> Config option to cast empty string to null does not cast empty string to null
> -
>
> Key: DRILL-3214
> URL: https://issues.apache.org/jira/browse/DRILL-3214
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.0.0
> Environment: faec150598840c40827e6493992d81209aa936da
>Reporter: Khurram Faraaz
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.1.0
>
>
> Config option drill.exec.functions.cast_empty_string_to_null does not seem to 
> be working as designed.
> Disable casting of empty strings to null. 
> {code}
> 0: jdbc:drill:schema=dfs.tmp> alter session set 
> `drill.exec.functions.cast_empty_string_to_null` = false;
> +---+--+
> |  ok   | summary  |
> +---+--+
> | true  | drill.exec.functions.cast_empty_string_to_null updated.  |
> +---+--+
> 1 row selected (0.078 seconds)
> {code}
> In this query we see empty strings are retained in query output in columns[1].
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT columns[0], columns[1], columns[2] from 
> `threeColsDouble.csv`;
> +--+-+-+
> |  EXPR$0  | EXPR$1  | EXPR$2  |
> +--+-+-+
> | 156  | 234 | 1   |
> | 2653543  | 434 | 0   |
> | 367345   | 567567  | 23  |
> | 34554| 1234| 45  |
> | 4345 | 567678  | 19876   |
> | 34556| 0   | 1109|
> | 5456 | -1  | 1098|
> | 6567 | | 34534   |
> | 7678 | 1   | 6   |
> | 8798 | 456 | 243 |
> | 265354   | 234 | 123 |
> | 367345   | | 234 |
> | 34554| 1   | 2   |
> | 4345 | 0   | 10  |
> | 34556| -1  | 19  |
> | 5456 | 23423   | 345 |
> | 6567 | 0   | 2348|
> | 7678 | 1   | 2   |
> | 8798 | | 45  |
> | 099  | 19  | 17  |
> +--+-+-+
> 20 rows selected (0.13 seconds)
> {code}
> Casting empty strings to integer leads to NumberFormatException
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT columns[0], cast(columns[1] as int), 
> columns[2] from `threeColsDouble.csv`;
> Error: SYSTEM ERROR: java.lang.NumberFormatException: 
> Fragment 0:0
> [Error Id: b08f4247-263a-460d-b37b-91a70375f7ba on centos-03.qa.lab:31010] 
> (state=,code=0)
> {code}
> Enable casting empty string to null.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> alter session set 
> `drill.exec.functions.cast_empty_string_to_null` = true;
> +---+--+
> |  ok   | summary  |
> +---+--+
> | true  | drill.exec.functions.cast_empty_string_to_null updated.  |
> +---+--+
> 1 row selected (0.077 seconds)
> {code}
> Run query
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT columns[0], cast(columns[1] as int), 
> columns[2] from `threeColsDouble.csv`;
> Error: SYSTEM ERROR: java.lang.NumberFormatException: 
> Fragment 0:0
> [Error Id: de633399-15f9-4a79-a21f-262bd5551207 on centos-03.qa.lab:31010] 
> (state=,code=0)
> {code}
> Note from the output of below query that the empty strings are not casted to 
> null, although drill.exec.functions.cast_empty_string_to_null was set to true.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT columns[0], columns[1], columns[2] from 
> `threeColsDouble.csv`;
> +--+-+-+
> |  EXPR$0  | EXPR$1  | EXPR$2  |
> +--+-+-+
> | 156  | 234 | 1   |
> | 2653543  | 434 | 0   |
> | 367345   | 567567  | 23  |
> | 34554| 1234| 45  |
> | 4345 | 567678  | 19876   |
> | 34556| 0   | 1109|
> | 5456 | -1  | 1098|
> | 6567 | | 34534   |
> | 7678 | 1   | 6   |
> | 8798 | 456 | 243 |
> | 265354   | 234 | 123 |
> | 367345   | | 234 |
> | 34554| 1   | 2   |
> | 4345 | 0   | 10  |
> | 34556| -1  | 19  |
> | 5456 | 23423   | 345 |
> | 6567 | 0   | 2348|
> | 7678 | 1   | 2   

[jira] [Commented] (DRILL-4039) Query fails when non-ascii characters are used in string literals

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860235#comment-15860235
 ] 

Sudheesh Katkam commented on DRILL-4039:


[~llystar] I noticed the patch does not have your name (rather "From: root 
"). Could you update the patch to add your name?

> Query fails when non-ascii characters are used in string literals
> -
>
> Key: DRILL-4039
> URL: https://issues.apache.org/jira/browse/DRILL-4039
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - JDBC
>Affects Versions: 1.1.0
> Environment: Linux lnxx64r6 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 
> 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Sergio Lob
> Attachments: DRILL-4039.patch.txt
>
>
> The following query against DRILL returns this error:
> SYSTEM ERROR: CalciteException: Failed to encode  'НАСТРОЕние' in character 
> set 'ISO-8859-1'
>  cc39118a-cde6-4a6e-a1d6-4b6b7e847b8a on maprd
> Query is:
>     SELECT
>    T1.`F01INT`,
>    T1.`F02UCHAR_10`,
>    T1.`F03UVARCHAR_10`
>     FROM
>    DPRV64R6_TRDUNI01T T1
>     WHERE
>    (T1.`F03UVARCHAR_10` =  'НАСТРОЕние')
>     ORDER BY
>    T1.`F01INT`;
> This issue looks similar to jira HIVE-12207.
> Is there a fix or workaround for this?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5219) Remove DrillUserProperties filtering in C++ driver

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5219:
---
Labels: ready-to-commit  (was: )

> Remove DrillUserProperties filtering in C++ driver
> --
>
> Key: DRILL-5219
> URL: https://issues.apache.org/jira/browse/DRILL-5219
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>  Labels: ready-to-commit
>
> Unlike the Java client, the C++ connector filter out unknown Drill user 
> properties:
> https://github.com/apache/drill/blob/master/contrib/native/client/src/clientlib/drillClientImpl.cpp#L374
> This prevents a client (like the ODBC driver) to pass extra properties to the 
> server (like extra metainformation, or some specific behavior for a given 
> software)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5190) Display planning time for a query in its profile page

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5190:
---
Labels:   (was: ready-to-commit)

> Display planning time for a query in its profile page
> -
>
> Key: DRILL-5190
> URL: https://issues.apache.org/jira/browse/DRILL-5190
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>
> Currently, the Web UI does not display the time spent in planning for a query 
> in its profile page. The estimate needs to be done by seeing how late did the 
> earliest major fragment start. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5230) Translation of millisecond duration into hours is incorrect

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5230:
---
Labels: easyfix  (was: easyfix ready-to-commit)

> Translation of millisecond duration into hours is incorrect
> ---
>
> Key: DRILL-5230
> URL: https://issues.apache.org/jira/browse/DRILL-5230
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>  Labels: easyfix
> Fix For: 1.10.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The method 
> {code:JAVA}org.apache.drill.exec.server.rest.profile.TableBuilder.appendMillis(long,
>  String){code}
> has a bug where the human readable translation of a 1+ hr duration in 
> milliseconds is reported incorrectly. 
> This has to do with the {code:JAVA}SimpleDateFormat.format() {code} method 
> incorectly translating it. 
> For e.g.
> {code:JAVA}
> long x = 4545342L; //1 hour 15 min 45.342 sec
> public void appendMillis(x, null);
> {code}
> This formats the value as {noformat}17h15m{noformat} instead of 
> {noformat}1h15m{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5207) Improve Parquet scan pipelining

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5207:
---
Labels: ready-to-commit  (was: )

> Improve Parquet scan pipelining
> ---
>
> Key: DRILL-5207
> URL: https://issues.apache.org/jira/browse/DRILL-5207
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.9.0
>Reporter: Parth Chandra
>Assignee: Parth Chandra
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> The parquet reader's async page reader is not quite efficiently pipelined. 
> The default size of the disk read buffer is 4MB while the page reader reads 
> ~1MB at a time. The Parquet decode is also processing 1MB at a time. This 
> means the disk is idle while the data is being processed. Reducing the buffer 
> to 1MB will reduce the time the processing thread waits for the disk read 
> thread.
> Additionally, since the data to process a page may be more or less than 1MB, 
> a queue of pages will help so that the disk scan does not block (until the 
> queue is full), waiting for the processing thread.
> Additionally, the BufferedDirectBufInputStream class reads from disk as soon 
> as it is initialized. Since this is called at setup time, this increases the 
> setup time for the query and query execution does not begin until this is 
> completed.
> There are a few other inefficiencies - options are read every time a page 
> reader is created. Reading options can be expensive.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5219) Remove DrillUserProperties filtering in C++ driver

[Sudheesh Katkam (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850922#comment-15850922
 ] 

Sudheesh Katkam commented on DRILL-5219:


I was planning to add filtering to the Java client ([see 
here|https://github.com/sudheeshkatkam/drill/blob/DRILL-4280-PR/common/src/main/java/org/apache/drill/common/config/DrillProperties.java]).
 How will these properties be used without code changes (in UserSession)?

> Remove DrillUserProperties filtering in C++ driver
> --
>
> Key: DRILL-5219
> URL: https://issues.apache.org/jira/browse/DRILL-5219
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - C++
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>
> Unlike the Java client, the C++ connector filter out unknown Drill user 
> properties:
> https://github.com/apache/drill/blob/master/contrib/native/client/src/clientlib/drillClientImpl.cpp#L374
> This prevents a client (like the ODBC driver) to pass extra properties to the 
> server (like extra metainformation, or some specific behavior for a given 
> software)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5220) Add api to set application name in C++ connector

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5220:
---
Labels: ready-to-commit  (was: )

> Add api to set application name in C++ connector
> 
>
> Key: DRILL-5220
> URL: https://issues.apache.org/jira/browse/DRILL-5220
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Client - C++
>Affects Versions: 1.8.0
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
>  Labels: ready-to-commit
>
> There's no API for a C++ connector user to specify the name of the 
> application, and to provide it to the server (optional field added in 
> DRILL-4369)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5238) CTTAS: unable to resolve temporary table if workspace is indicated without schema

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5238:
---
Labels: ready-to-commit  (was: )

> CTTAS: unable to resolve temporary table if workspace is indicated without 
> schema
> -
>
> Key: DRILL-5238
> URL: https://issues.apache.org/jira/browse/DRILL-5238
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>  Labels: ready-to-commit
> Fix For: 1.10.0
>
>
> Drill is unable to resolve temporary table if default temporary workspace is 
> partially indicated (schema was set using USE command and only workspace name 
> is used in query).
> {noformat}
> 0: jdbc:drill:zk=local> use dfs;
> +---+--+
> |  ok   | summary  |
> +---+--+
> | true  | Default schema changed to [dfs]  |
> +---+--+
> 1 row selected (0.156 seconds)
> 0: jdbc:drill:zk=local> create temporary table tmp.t as select 'A' from 
> (values(1));
> +---++
> | Fragment  | Number of records written  |
> +---++
> | 0_0   | 1  |
> +---++
> 1 row selected (1.525 seconds)
> 0: jdbc:drill:zk=local> select * from tmp.t;
> Feb 02, 2017 11:24:09 AM 
> org.apache.calcite.sql.validate.SqlValidatorException 
> SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table 'tmp.t' 
> not found
> Feb 02, 2017 11:24:09 AM org.apache.calcite.runtime.CalciteException 
> SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1, 
> column 15 to line 1, column 17: Table 'tmp.t' not found
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 17: Table 
> 'tmp.t' not found
> SQL Query null
> [Error Id: 5266cb67-9d37-4a94-9a4e-28a4a2f94be5 on localhost:31010] 
> (state=,code=0)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5190) Display planning time for a query in its profile page

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5190:
---
Labels: ready-to-commit  (was: )

> Display planning time for a query in its profile page
> -
>
> Key: DRILL-5190
> URL: https://issues.apache.org/jira/browse/DRILL-5190
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
>
> Currently, the Web UI does not display the time spent in planning for a query 
> in its profile page. The estimate needs to be done by seeing how late did the 
> earliest major fragment start. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (DRILL-5230) Translation of millisecond duration into hours is incorrect

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5230:
---
Labels: easyfix ready-to-commit  (was: easyfix)

> Translation of millisecond duration into hours is incorrect
> ---
>
> Key: DRILL-5230
> URL: https://issues.apache.org/jira/browse/DRILL-5230
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Web Server
>Affects Versions: 1.9.0
>Reporter: Kunal Khatua
>  Labels: easyfix, ready-to-commit
> Fix For: 1.10.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The method 
> {code:JAVA}org.apache.drill.exec.server.rest.profile.TableBuilder.appendMillis(long,
>  String){code}
> has a bug where the human readable translation of a 1+ hr duration in 
> milliseconds is reported incorrectly. 
> This has to do with the {code:JAVA}SimpleDateFormat.format() {code} method 
> incorectly translating it. 
> For e.g.
> {code:JAVA}
> long x = 4545342L; //1 hour 15 min 45.342 sec
> public void appendMillis(x, null);
> {code}
> This formats the value as {noformat}17h15m{noformat} instead of 
> {noformat}1h15m{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] (DRILL-3562) Query fails when using flatten on JSON data where some documents have an empty array

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Apache Drill /  DRILL-3562 
 
 
 
  Query fails when using flatten on JSON data where some documents have an empty array  
 
 
 
 
 
 
 
 
 

Change By:
 
 Sudheesh Katkam 
 
 
 

Labels:
 
 ready-to-commit 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-5126) Provide simplified, unified "cluster fixture" for tests

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam commented on  DRILL-5126 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Provide simplified, unified "cluster fixture" for tests  
 
 
 
 
 
 
 
 
 
 
Fixed in 5c3924c 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-4764) Parquet file with INT_16, etc. logical types not supported by simple SELECT

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam commented on  DRILL-4764 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Parquet file with INT_16, etc. logical types not supported by simple SELECT  
 
 
 
 
 
 
 
 
 
 
Fixed in 60624af 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-5218) Support Disabling Heartbeats in C++ Client

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam resolved as Fixed 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Apache Drill /  DRILL-5218 
 
 
 
  Support Disabling Heartbeats in C++ Client  
 
 
 
 
 
 
 
 
 

Change By:
 
 Sudheesh Katkam 
 
 
 

Resolution:
 
 Fixed 
 
 
 

Fix Version/s:
 
 1.10.0 
 
 
 

Status:
 
 Open Resolved 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-5218) Support Disabling Heartbeats in C++ Client

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam commented on  DRILL-5218 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Support Disabling Heartbeats in C++ Client  
 
 
 
 
 
 
 
 
 
 
Fixed in 837722c 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-3562) Query fails when using flatten on JSON data where some documents have an empty array

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Apache Drill /  DRILL-3562 
 
 
 
  Query fails when using flatten on JSON data where some documents have an empty array  
 
 
 
 
 
 
 
 
 

Change By:
 
 Sudheesh Katkam 
 
 
 

Labels:
 
 ready-to-commit 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-5218) Support Disabling Heartbeats in C++ Client

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Apache Drill /  DRILL-5218 
 
 
 
  Support Disabling Heartbeats in C++ Client  
 
 
 
 
 
 
 
 
 

Change By:
 
 Sudheesh Katkam 
 
 
 

Labels:
 
 ready-to-commit 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (DRILL-5215) CTTAS: disallow temp tables in view expansion logic

[Sudheesh Katkam (JIRA)]

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sudheesh Katkam updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Apache Drill /  DRILL-5215 
 
 
 
  CTTAS: disallow temp tables in view expansion logic  
 
 
 
 
 
 
 
 
 

Change By:
 
 Sudheesh Katkam 
 
 
 

Labels:
 
 ready-to-commit 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] [Updated] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures

[Sudheesh Katkam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sudheesh Katkam updated DRILL-5157:
---
Labels:   (was: ready-to-commit)

> Multiple Snappy versions on class path; causes unit test failures
> -
>
> Key: DRILL-5157
> URL: https://issues.apache.org/jira/browse/DRILL-5157
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>
> Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It 
> fails with the following stack trace and the memory leak trace shown second.
> Strangely, this error appears only if the test is run as part of the overall 
> suite. The error does not appear if the test is run individually in the 
> debugger. This suggests that the problem described here is a side-effect of a 
> problem created by an earlier test.
> Investigation (see notes below) reveal that this is a Maven dependency 
> management problem.
> Stack trace that seems to show that the code was trying to find a Snappy 
> native library:
> {code}
> java.lang.reflect.InvocationTargetException
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>   at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>   at org.xerial.snappy.Snappy.(Snappy.java:44)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479)
>   at 
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562)
>   at 
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178)
>   at 
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
>   at 
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> …
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)