[jira] [Commented] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.

[Kunal Khatua (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290482#comment-16290482
 ] 

Kunal Khatua commented on DRILL-5028:
-

This looks related to DRILL-5270 which has the same issue. I'm linking it for 
now, since DRILL-5270 attempts address this.

> Opening profiles page from web ui gets very slow when a lot of history files 
> have been stored in HDFS or Local FS.
> --
>
> Key: DRILL-5028
> URL: https://issues.apache.org/jira/browse/DRILL-5028
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Functions - Drill
>Affects Versions: 1.8.0
>Reporter: Hongze Zhang
>Priority: Minor
> Fix For: Future
>
>
> We have a Drill cluster with 20+ Nodes and we store all history profiles in 
> hdfs. Without doing periodically cleans for hdfs, the profiles page gets 
> slower while serving more queries.
> Code from LocalPersistentStore.java uses fs.list(false, basePath) for 
> fetching the latest 100 history profiles by default, I guess this operation 
> blocks the page loading (Millions small files can be stored in the basePath), 
> maybe we can try some other ways to reach the same goal.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua reopened DRILL-5028:
-

> Opening profiles page from web ui gets very slow when a lot of history files 
> have been stored in HDFS or Local FS.
> --
>
> Key: DRILL-5028
> URL: https://issues.apache.org/jira/browse/DRILL-5028
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Functions - Drill
>Affects Versions: 1.8.0
>Reporter: Hongze Zhang
>Priority: Minor
> Fix For: Future
>
>
> We have a Drill cluster with 20+ Nodes and we store all history profiles in 
> hdfs. Without doing periodically cleans for hdfs, the profiles page gets 
> slower while serving more queries.
> Code from LocalPersistentStore.java uses fs.list(false, basePath) for 
> fetching the latest 100 history profiles by default, I guess this operation 
> blocks the page loading (Millions small files can be stored in the basePath), 
> maybe we can try some other ways to reach the same goal.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5973) Support injection of time-bound pauses in server

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua updated DRILL-5973:

Labels: ready-to-commit  (was: )

> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>  Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-5973) Support injection of time-bound pauses in server

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua resolved DRILL-5973.
-
Resolution: Implemented

> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (DRILL-5973) Support injection of time-bound pauses in server

[Kunal Khatua (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kunal Khatua reopened DRILL-5973:
-

> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-4779) Kafka storage plugin support

[Bridget Bevens (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290109#comment-16290109
 ] 

Bridget Bevens commented on DRILL-4779:
---

Hi,
This JIRA is marked as having doc impact. Can you please provide some 
information in terms of how the user configures the storage plugin? For 
example, must the user edit the Storage configuration in the Drill Web UI 
(similar to the configuration for other storage plugins, such as Hive)? If so, 
what is the required configuration? 
Thanks,
Bridget


> Kafka storage plugin support
> 
>
> Key: DRILL-4779
> URL: https://issues.apache.org/jira/browse/DRILL-4779
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Storage - Other
>Affects Versions: 1.11.0
>Reporter: B Anil Kumar
>Assignee: B Anil Kumar
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.12.0
>
>
> Implement Kafka storage plugin will enable the strong SQL support for Kafka.
> Initially implementation can target for supporting json and avro message types



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6032) Use RecordBatchSizer to estimate size of columns in HashAgg

[Timothy Farkas (JIRA)]

Timothy Farkas created DRILL-6032:
-

 Summary: Use RecordBatchSizer to estimate size of columns in 
HashAgg
 Key: DRILL-6032
 URL: https://issues.apache.org/jira/browse/DRILL-6032
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Timothy Farkas
Assignee: Timothy Farkas






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-6031) Document support for dots in field names

[Bridget Bevens (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bridget Bevens resolved DRILL-6031.
---
Resolution: Fixed

Setting status to resolved. 

> Document support for dots in field names 
> -
>
> Key: DRILL-6031
> URL: https://issues.apache.org/jira/browse/DRILL-6031
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Documentation
>Affects Versions: 1.12.0
>Reporter: Bridget Bevens
>Assignee: Bridget Bevens
>Priority: Minor
> Fix For: 1.12.0
>
>
> Document support for dots in field names.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-6031) Document support for dots in field names

[Bridget Bevens (JIRA)]

Bridget Bevens created DRILL-6031:
-

 Summary: Document support for dots in field names 
 Key: DRILL-6031
 URL: https://issues.apache.org/jira/browse/DRILL-6031
 Project: Apache Drill
  Issue Type: Sub-task
  Components: Documentation
Affects Versions: 1.12.0
Reporter: Bridget Bevens
Assignee: Bridget Bevens
Priority: Minor
 Fix For: 1.12.0


Document support for dots in field names.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5973) Support injection of time-bound pauses in server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289776#comment-16289776
 ] 

ASF GitHub Bot commented on DRILL-5973:
---

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/1055
  
+1. 
No need to squash/rebase. 


> Support injection of time-bound pauses in server
> 
>
> Key: DRILL-5973
> URL: https://issues.apache.org/jira/browse/DRILL-5973
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build & Test
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.13.0
>
>
> While working on DRILL-3640 , when creating a unit test for a server-induced 
> timeout, the injecting a pause leaves the JUnit framework's DrillClient 
> without a handle to the query on the server. This is because we injected the 
> pause to occur before the server could send back a query ID, so the 
> DrillClient has no way to unpause the server.
> The workaround to support this unit test is to allow for injecting pauses 
> with a defined time-bound, after which the server would resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6028) Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error

[Pritesh Maker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pritesh Maker updated DRILL-6028:
-
Reviewer: Paul Rogers

> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error
> --
>
> Key: DRILL-6028
> URL: https://issues.apache.org/jira/browse/DRILL-6028
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.13.0
>
>
> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error.
> *REPRODUCE*
> File {{1200_columns.csv}}
> {noformat}
> 0,1,2,3...1200
> 0,1,2,3...1200
> {noformat}
> Query
> {noformat}
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> union
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> {noformat}
> Error
> {noformat}
> Error: SYSTEM ERROR: CompileException: File 
> 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', 
> Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
> public boolean isKeyMatchInternalBuild(int incomingRowIdx, int 
> htRowIdx)
>^ (compiler.err.limit.code)
> {noformat}
> *ROOT CAUSE*
> DRILL-4715 added ability to ensure that methods size won't go beyond the 64k 
> limit imposed by JVM. {{BlkCreateMode.TRUE_IF_BOUND}} was added to create new 
> block only if # of expressions added hit upper-bound defined by 
> {{exec.java.compiler.exp_in_method_size}}. Once number of expressions in 
> methods hits upper bound we create from call inner method.
> Example: 
> {noformat}
> public void doSetup(RecordBatch incomingBuild, RecordBatch incomingProbe) 
> throws SchemaChangeException {
> // some logic
> return doSetup0(incomingBuild, incomingProbe);
> }
> {noformat}
> During code generation {{ChainedHashTable}} added all code in its methods in 
> one block (using {{BlkCreateMode.FALSE}}) since {{getHashBuild}} and 
> {{getHashProbe}} methods contained state and thus could not be split. In 
> these methods hash was generated for each key expression. For the first key 
> seed was 0, subsequent keys hash was generated based on seed from previous 
> key.
> To allow splitting for there methods the following was done:
> 1. Method signatures was changed: added new parameter {{seedValue}}. 
> Initially starting seed value was hard-coded during code generation (set to 
> 0), now it is passed as method parameter.
> 2. Initially hash function call for all keys was transformed into one logical 
> expression which did not allow splitting. Now we create logical expression 
> for each key and thus splitting is possible. New {{seedValue}} parameter is 
> used as seed holder to pass seed value for the next key.
> 3. {{ParameterExpression}} was added to generate reference to method 
> parameter during code generation.
> Code example:
> {noformat}
> public int getHashBuild(int incomingRowIdx, int seedValue)
> throws SchemaChangeException
> {
> {
> NullableVarCharHolder out3 = new NullableVarCharHolder();
> {
> out3 .isSet = vv0 .getAccessor().isSet((incomingRowIdx));
> if (out3 .isSet == 1) {
> out3 .buffer = vv0 .getBuffer();
> long startEnd = vv0 
> .getAccessor().getStartEnd((incomingRowIdx));
> out3 .start = ((int) startEnd);
> out3 .end = ((int)(startEnd >> 32));
> }
> }
> IntHolder seedValue4 = new IntHolder();
> seedValue4 .value = seedValue;
> // start of eval portion of hash32 function. //
> IntHolder out5 = new IntHolder();
> {
> final IntHolder out = new IntHolder();
> NullableVarCharHolder in = out3;
> IntHolder seed = seedValue4;
>  
> Hash32FunctionsWithSeed$NullableVarCharHash_eval: {
> if (in.isSet == 0) {
> out.value = seed.value;
> } else
> {
> out.value = 
> org.apache.drill.exec.expr.fn.impl.HashHelper.hash32(in.start, in.end, 
> in.buffer, seed.value);
> }
> }
>  
> out5 = out;
> }
> // end of eval portion of hash32 function. //
> seedValue = out5 .value;
>return getHashBuild0((incomingRowIdx), (seedValue));
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6028) Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289581#comment-16289581
 ] 

ASF GitHub Bot commented on DRILL-6028:
---

Github user priteshm commented on the issue:

https://github.com/apache/drill/pull/1071
  
@paul-rogers can you please review this?


> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error
> --
>
> Key: DRILL-6028
> URL: https://issues.apache.org/jira/browse/DRILL-6028
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.13.0
>
>
> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error.
> *REPRODUCE*
> File {{1200_columns.csv}}
> {noformat}
> 0,1,2,3...1200
> 0,1,2,3...1200
> {noformat}
> Query
> {noformat}
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> union
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> {noformat}
> Error
> {noformat}
> Error: SYSTEM ERROR: CompileException: File 
> 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', 
> Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
> public boolean isKeyMatchInternalBuild(int incomingRowIdx, int 
> htRowIdx)
>^ (compiler.err.limit.code)
> {noformat}
> *ROOT CAUSE*
> DRILL-4715 added ability to ensure that methods size won't go beyond the 64k 
> limit imposed by JVM. {{BlkCreateMode.TRUE_IF_BOUND}} was added to create new 
> block only if # of expressions added hit upper-bound defined by 
> {{exec.java.compiler.exp_in_method_size}}. Once number of expressions in 
> methods hits upper bound we create from call inner method.
> Example: 
> {noformat}
> public void doSetup(RecordBatch incomingBuild, RecordBatch incomingProbe) 
> throws SchemaChangeException {
> // some logic
> return doSetup0(incomingBuild, incomingProbe);
> }
> {noformat}
> During code generation {{ChainedHashTable}} added all code in its methods in 
> one block (using {{BlkCreateMode.FALSE}}) since {{getHashBuild}} and 
> {{getHashProbe}} methods contained state and thus could not be split. In 
> these methods hash was generated for each key expression. For the first key 
> seed was 0, subsequent keys hash was generated based on seed from previous 
> key.
> To allow splitting for there methods the following was done:
> 1. Method signatures was changed: added new parameter {{seedValue}}. 
> Initially starting seed value was hard-coded during code generation (set to 
> 0), now it is passed as method parameter.
> 2. Initially hash function call for all keys was transformed into one logical 
> expression which did not allow splitting. Now we create logical expression 
> for each key and thus splitting is possible. New {{seedValue}} parameter is 
> used as seed holder to pass seed value for the next key.
> 3. {{ParameterExpression}} was added to generate reference to method 
> parameter during code generation.
> Code example:
> {noformat}
> public int getHashBuild(int incomingRowIdx, int seedValue)
> throws SchemaChangeException
> {
> {
> NullableVarCharHolder out3 = new NullableVarCharHolder();
> {
> out3 .isSet = vv0 .getAccessor().isSet((incomingRowIdx));
> if (out3 .isSet == 1) {
> out3 .buffer = vv0 .getBuffer();
> long startEnd = vv0 
> .getAccessor().getStartEnd((incomingRowIdx));
> out3 .start = ((int) startEnd);
> out3 .end = ((int)(startEnd >> 32));
> }
> }
> IntHolder seedValue4 = new IntHolder();
> seedValue4 .value = seedValue;
> // start of eval portion of hash32 function. //
> IntHolder out5 = new IntHolder();
> {
> final IntHolder out = new IntHolder();
> NullableVarCharHolder in = out3;
> IntHolder seed = seedValue4;
>  
> Hash32FunctionsWithSeed$NullableVarCharHash_eval: {
> if (in.isSet == 0) {
> out.value = seed.value;
> } else
> {
> out.value = 
> org.apache.drill.exec.expr.fn.impl.HashHelper.hash32(in.start, in.end, 
> in.buffer, seed.value);
> }
> }
>  
> out5 = out;
> }
> // end of eval portion of hash32 function. //
> seedValue = out5 .value;
>return getHashBuild0((incomingRowIdx), (seedValue));
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6028) Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289472#comment-16289472
 ] 

ASF GitHub Bot commented on DRILL-6028:
---

GitHub user arina-ielchiieva opened a pull request:

https://github.com/apache/drill/pull/1071

DRILL-6028: Allow splitting generated code in ChainedHashTable into b…

…locks to avoid "code too large" error

1. Added new parameter seedValue to getHashBuild and getHashProbe methods 
in HashTableTemplate.
2. Generate logical expression for each key so its  can be split into 
blocks if number of expressions in method exceeds upper limit.
3. ParameterExpression was added to generate reference to method parameter 
during code generation.

Details in [DRILL-6028](https://issues.apache.org/jira/browse/DRILL-6028)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arina-ielchiieva/drill DRILL-6028

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1071.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1071


commit 90dbd2db96bed8fa3cd3dab7575aadab240a621c
Author: Arina Ielchiieva 
Date:   2017-12-12T16:45:57Z

DRILL-6028: Allow splitting generated code in ChainedHashTable into blocks 
to avoid "code too large" error

1. Added new parameter seedValue to getHashBuild and getHashProbe methods 
in HashTableTemplate.
2. Generate logical expression for each key so its  can be split into 
blocks if number of expressions in method exceeds upper limit.
3. ParameterExpression was added to generate reference to method parameter 
during code generation.




> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error
> --
>
> Key: DRILL-6028
> URL: https://issues.apache.org/jira/browse/DRILL-6028
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.13.0
>
>
> Allow splitting generated code in ChainedHashTable into blocks to avoid "code 
> too large" error.
> *REPRODUCE*
> File {{1200_columns.csv}}
> {noformat}
> 0,1,2,3...1200
> 0,1,2,3...1200
> {noformat}
> Query
> {noformat}
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> union
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> {noformat}
> Error
> {noformat}
> Error: SYSTEM ERROR: CompileException: File 
> 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', 
> Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
> public boolean isKeyMatchInternalBuild(int incomingRowIdx, int 
> htRowIdx)
>^ (compiler.err.limit.code)
> {noformat}
> *ROOT CAUSE*
> DRILL-4715 added ability to ensure that methods size won't go beyond the 64k 
> limit imposed by JVM. {{BlkCreateMode.TRUE_IF_BOUND}} was added to create new 
> block only if # of expressions added hit upper-bound defined by 
> {{exec.java.compiler.exp_in_method_size}}. Once number of expressions in 
> methods hits upper bound we create from call inner method.
> Example: 
> {noformat}
> public void doSetup(RecordBatch incomingBuild, RecordBatch incomingProbe) 
> throws SchemaChangeException {
> // some logic
> return doSetup0(incomingBuild, incomingProbe);
> }
> {noformat}
> During code generation {{ChainedHashTable}} added all code in its methods in 
> one block (using {{BlkCreateMode.FALSE}}) since {{getHashBuild}} and 
> {{getHashProbe}} methods contained state and thus could not be split. In 
> these methods hash was generated for each key expression. For the first key 
> seed was 0, subsequent keys hash was generated based on seed from previous 
> key.
> To allow splitting for there methods the following was done:
> 1. Method signatures was changed: added new parameter {{seedValue}}. 
> Initially starting seed value was hard-coded during code generation (set to 
> 0), now it is passed as method parameter.
> 2. Initially hash function call for all keys was transformed into one logical 
> expression which did not allow splitting. Now we create logical expression 
> for each key and thus splitting is possible. New {{seedValue}} parameter is 
> used as seed holder to pass seed value for the next key.
> 3. {{ParameterExpression}} was added to generate reference to method 
> parameter during code generation.
> Code example:
> {noformat}
> public int getHashBuild(int incomingRowIdx, int seedValue)
> throws SchemaChangeException
> {
> {
> 

[jira] [Assigned] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-5425:
---

Assignee: Sorabh Hamirwasia  (was: Sindhuri Ramanarayan Rayavaram)

> Support HTTP Kerberos auth using SPNEGO
> ---
>
> Key: DRILL-5425
> URL: https://issues.apache.org/jira/browse/DRILL-5425
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Sudheesh Katkam
>Assignee: Sorabh Hamirwasia
> Fix For: 1.13.0
>
>
> DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests 
> to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for 
> HTTP connections.
> This requires creating "direct" web sessions; currently web sessions are 
> sessions over Java client sessions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5425:

Affects Version/s: 1.12.0

> Support HTTP Kerberos auth using SPNEGO
> ---
>
> Key: DRILL-5425
> URL: https://issues.apache.org/jira/browse/DRILL-5425
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Sudheesh Katkam
>Assignee: Sindhuri Ramanarayan Rayavaram
> Fix For: 1.13.0
>
>
> DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests 
> to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for 
> HTTP connections.
> This requires creating "direct" web sessions; currently web sessions are 
> sessions over Java client sessions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5425:

Fix Version/s: 1.13.0

> Support HTTP Kerberos auth using SPNEGO
> ---
>
> Key: DRILL-5425
> URL: https://issues.apache.org/jira/browse/DRILL-5425
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Sudheesh Katkam
>Assignee: Sindhuri Ramanarayan Rayavaram
> Fix For: 1.13.0
>
>
> DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests 
> to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for 
> HTTP connections.
> This requires creating "direct" web sessions; currently web sessions are 
> sessions over Java client sessions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289083#comment-16289083
 ] 

ASF GitHub Bot commented on DRILL-6020:
---

Github user vvysotskyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/1068#discussion_r156621897
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/FieldIdUtil.java
 ---
@@ -131,7 +131,7 @@ public static TypedFieldId 
getFieldIdIfMatches(ValueVector vector, TypedFieldId.
 } else if(v instanceof ListVector) {
   ListVector list = (ListVector) v;
   return getFieldIdIfMatches(list, builder, addToBreadCrumb, 
seg.getChild());
-} else if (v instanceof  UnionVector) {
+} else if (v instanceof  UnionVector && !seg.isLastPath()) {
--- End diff --

Good catch! 
I think we should add check for nulls into method 
`getFieldIdIfMatchesUnion()` as it was done for `getFieldIdIfMatches()`. Also 
please add a unit test for this change. You may use 
[testFieldWithDots()](https://github.com/apache/drill/blob/acc5ed927e1fa4011ac1c3724d15197484b9f45b/exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestJsonReader.java#L705)
 as an example.

But one more point: with this change error is not thrown, but only nulls 
are returned. I think we should also fix this issue on the borders of this pull 
request.


> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Labels: doc-impacting ready-to-commit  (was: doc-impacting)

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Priority: Minor
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Description: 
If the WebServer is launched on a machine that has more than 200 cores, you get 
the following stack trace:

{noformat}
Exception in thread "main" 
org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
startup of Drillbit:
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
Caused by: java.lang.IllegalStateException: Insufficient max threads in 
ThreadPool: max=200 < needed=206
at org.eclipse.jetty.server.Server.doStart(Server.java:321)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
... 2 more
{noformat}

The cause of this is that in the WebServer start method, a Server instance is 
created with the default constructor, which initializes a QueuedThreadPool with 
a default maxThreads value of 200, and there is no way to configure this value.

*For documentation*
New config option - drill.exec.web_server.thread_pool_max.

  was:
If the WebServer is launched on a machine that has more than 200 cores, you get 
the following stack trace:

{noformat}
Exception in thread "main" 
org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
startup of Drillbit:
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
Caused by: java.lang.IllegalStateException: Insufficient max threads in 
ThreadPool: max=200 < needed=206
at org.eclipse.jetty.server.Server.doStart(Server.java:321)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
... 2 more
{noformat}

The cause of this is that in the WebServer start method, a Server instance is 
created with the default constructor, which initializes a QueuedThreadPool with 
a default maxThreads value of 200, and there is no way to configure this value.


> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Priority: Minor
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Labels: doc-impacting  (was: )

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Priority: Minor
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-5994:
---

Assignee: (was: Arina Ielchiieva)

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Priority: Minor
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Affects Version/s: 1.11.0

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Issue Type: Improvement  (was: Bug)

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289015#comment-16289015
 ] 

ASF GitHub Bot commented on DRILL-5994:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/1069
  
+1, LGTM.


> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Fix Version/s: 1.13.0

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-5994:

Reviewer: Arina Ielchiieva

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-5994) Cannot start web server on a machine with more than 200 cores

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-5994:
---

Assignee: Arina Ielchiieva

> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you 
> get the following stack trace:
> {noformat}
> Exception in thread "main" 
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial 
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in 
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is 
> created with the default constructor, which initializes a QueuedThreadPool 
> with a default maxThreads value of 200, and there is no way to configure this 
> value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-6020:
---

Assignee: (was: Arina Ielchiieva)

> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-6020:

Reviewer: Volodymyr Vysotskyi

> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva reassigned DRILL-6020:
---

Assignee: Arina Ielchiieva

> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Arina Ielchiieva
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-6020:

Affects Version/s: 1.11.0

> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (DRILL-6020) NullPointerException with Union setting on when querying JSON untyped path

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-6020:

Fix Version/s: 1.13.0

> NullPointerException with Union setting on when querying JSON untyped path
> --
>
> Key: DRILL-6020
> URL: https://issues.apache.org/jira/browse/DRILL-6020
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
> Fix For: 1.13.0
>
>
> h1. Steps to reproduce
> alter session set `exec.enable_union_type`=true;
> select tb.level1.dta from dfs.`file.json` tb;
> *Content of file.json:*
> {noformat}
> {"level1":{"dta":{"test":"test"}}}
> {"level1":{"dta":"test"}}
> {noformat}
> h1. Stack trace
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: fe267584-32f3-413c-a77c-fc5b5c1ba513 on localhost:31010]
>   (java.lang.NullPointerException) null
> 
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatchesUnion():34
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():135
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldIdIfMatches():130
> org.apache.drill.exec.vector.complex.FieldIdUtil.getFieldId():201
> org.apache.drill.exec.record.SimpleVectorWrapper.getFieldIdIfMatches():102
> org.apache.drill.exec.record.VectorContainer.getValueVectorId():298
> org.apache.drill.exec.physical.impl.ScanBatch.getValueVectorId():313
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():289
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitSchemaPath():272
> org.apache.drill.common.expression.SchemaPath.accept():150
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():399
> 
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$AbstractMaterializeVisitor.visitFunctionCall():331
> org.apache.drill.common.expression.FunctionCall.accept():60
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():169
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize():147
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():421
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():78
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
> 
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)