[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884104#comment-15884104
 ] 

ASF GitHub Bot commented on DRILL-5274:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/760


> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>  Labels: ready-to-commit
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883915#comment-15883915
 ] 

ASF GitHub Bot commented on DRILL-5274:
---

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/760
  
+1


> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>  Labels: ready-to-commit
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882649#comment-15882649
 ] 

ASF GitHub Bot commented on DRILL-5274:
---

GitHub user arina-ielchiieva opened a pull request:

https://github.com/apache/drill/pull/760

DRILL-5274: Exception thrown in Drillbit shutdown in UDF cleanup code



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arina-ielchiieva/drill DRILL-5274

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/760.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #760


commit 55f7ae08f58a75db888052e617405cf4d10410dd
Author: Arina Ielchiieva 
Date:   2017-02-24T12:06:50Z

DRILL-5274: Exception thrown in Drillbit shutdown in UDF cleanup code




> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-24 Thread Arina Ielchiieva (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882493#comment-15882493
 ] 

Arina Ielchiieva commented on DRILL-5274:
-

[~paul-rogers] Did you delete local udf directory intentionally? Or how it got 
deleted?


> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-18 Thread Paul Rogers (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873321#comment-15873321
 ] 

Paul Rogers commented on DRILL-5274:



The following patch makes the problem go away, but I'm not sure if the above is 
an indication of deeper problems.

{code}
public class FunctionImplementationRegistry implements FunctionLookupContext, 
AutoCloseable {
  ...
  public void close() {
if (deleteTmpDir) {
  ...
} else {
  try {
File dir = new File(localUdfDir.toUri().getPath());
if (dir.exists()) {
  FileUtils.cleanDirectory(dir);
}
  ...
}
  }
{code}

> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-5274) Exception thrown in Drillbit shutdown in UDF cleanup code

2017-02-18 Thread Paul Rogers (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873319#comment-15873319
 ] 

Paul Rogers commented on DRILL-5274:


Example case. Query:

{code}
select * from (select * from dfs.`/drill/testdata/250wide.tbl`
order by columns[0])d where d.columns[0] = 'ljdfhwuehnoiueyf';
{code}

The data file is a large file consisting of a single column 250 characters wide.

Batch as seen by the external sort:

{code}
ExternalSortBatch - Actual batch schema & sizes {
  T0¦¦columns(std col. size: 54, actual col. size: 0, total size: 263942, 
vector size: 0, data size: 0, row capacity: 1023, density: 0)
  EXPR$1(std col. size: 54, actual col. size: 250, total size: 282624, vector 
size: 278528, data size: 255750, row capacity: 4095, density: 92)
  Records: 1023, Total size: 548614, Row width:538, Density:92}
{code}

The result is that the sort must buffer two copies of the data. Since this is 
an 18 GB input file, the sort must buffer (then spill) 36 GB of data. Note that 
there is actually no column alias or expression on the column.

> Exception thrown in Drillbit shutdown in UDF cleanup code
> -
>
> Key: DRILL-5274
> URL: https://issues.apache.org/jira/browse/DRILL-5274
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Paul Rogers
>Assignee: Arina Ielchiieva
>Priority: Minor
>
> I ran a very simple query: a single-line text file in an embedded Drillbit. 
> The UDF directory was placed in /tmp. During the run, the directory was 
> deleted. On Drillbit shutdown, the following occurred:
> {code}
> 25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
> 26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - 
> closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 
> ms
> 26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - 
> closed userServer in 1009 ms
> 26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - 
> closed dataPool in 1009 ms
> 26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
> java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
>   at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) 
> ~[commons-io-2.4.jar:2.4]
>   at 
> org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469)
>  ~[classes/:na]
>   at 
> org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) 
> ~[classes/:na]
>   at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) 
> ~[classes/:na]
>   at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) 
> ~[classes/:na]
>   at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) 
> ~[classes/:na]
> ...
> {code}
> The following patch makes the problem go away, but I'm not sure if the above 
> is an indication of deeper problems.
> {code}
> public class FunctionImplementationRegistry implements FunctionLookupContext, 
> AutoCloseable {
>   ...
>   public void close() {
> if (deleteTmpDir) {
>   ...
> } else {
>   try {
> File dir = new File(localUdfDir.toUri().getPath());
> if (dir.exists()) {
>   FileUtils.cleanDirectory(dir);
> }
>   ...
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)