[GitHub] drill issue #689: DRILL-4938: Need better error message

[zfong]

Github user zfong commented on the issue:

https://github.com/apache/drill/pull/689
  
Can you add a testcase for this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5124) Generated Java code methods throw schema change, calling code doesn't catch

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5124:
--

 Summary: Generated Java code methods throw schema change, calling 
code doesn't catch
 Key: DRILL-5124
 URL: https://issues.apache.org/jira/browse/DRILL-5124
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.8.0
Reporter: Paul Rogers
Assignee: Paul Rogers
Priority: Minor


Most generated code includes methods that throw {{}}. Example:

{code}
   public boolean resetValues()
throws SchemaChangeException
{ return true; }
{code}

But, most code in Drill that calls the generated methods does not catch the 
exception. The reason is that the Drill code is compiled against the template 
which does not declare exceptions:

{code}
  public abstract boolean resetValues();
{code}

The current behavior becomes an error when trying to compile the code as 
"Plain-Old Java" (DRILL-5052). The behavior is an error even otherwise as it 
causes the fragment to fail with an uncaught exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-5123) Write query profile after sending final response to client to improve latency

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5123:
--

 Summary: Write query profile after sending final response to 
client to improve latency
 Key: DRILL-5123
 URL: https://issues.apache.org/jira/browse/DRILL-5123
 Project: Apache Drill
  Issue Type: Improvement
Affects Versions: 1.8.0
Reporter: Paul Rogers
Assignee: Paul Rogers
Priority: Minor


In testing a particular query, I used a test setup that does not write to the 
"persistent store", causing query profiles to not be saved. I then changed the 
config to save them (to local disk). This produced about a 200ms difference in 
query run time as perceived by the client.

I then moved writing the query profile _after_ sending the client the final 
message. This resulted in an approximately 100ms savings, as perceived by the 
client, in query run time on short (~3 sec.) queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-5122) DrillBuf performs very expensive logging if JVM -ea option is set

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5122:
--

 Summary: DrillBuf performs very expensive logging if JVM -ea 
option is set
 Key: DRILL-5122
 URL: https://issues.apache.org/jira/browse/DRILL-5122
 Project: Apache Drill
  Issue Type: Improvement
  Components:  Server
Affects Versions: 1.8.0
Reporter: Paul Rogers
Assignee: Paul Rogers
Priority: Minor


The {{DrillBuf}} class is the foundation of Drill's direct memory model. It has 
a very sophisticated allocator mechanism behind it. {{DrillBuf}} provides 
logging to help track down memory leaks and other issues. Unfortunately, the 
trigger to turn on this option is based on detecting if assertions are enabled 
in the JVM.

Assertions are very handy tools: they are debug-only assertions not executed at 
run time. They are turned on with the "-ea" option to the JVM. But, in 
production, the assertions don't execute, avoiding production-time costs.

In Drill, once assertions are enabled, the dominant cost of a query becomes 
{{DrillBuf}} logging. On example showed that {{DrillBuf}} logging increases 
query cost by 10x.

Debugging of {{DrillBuf}} is handy for those few engineers who need to do so. 
Over time, {{DrillBuf}} has become quite stable so that, now, most developers, 
most of the time, do not work on {{DrillBuf}}.

One wonders: Drill uses Guava preconditions extensively. Is this perhaps due to 
the high cost of enabling assertions? Since assertions were unavailable, 
production-time checking via Guava may have become preferred. Assertions are, 
in fact, better because they incur no overhead in production, while Guava 
preconditions always execute, even in production. (Here we assume that the code 
errors that both check will find errors during development so that such errors 
do not sneak through into production.)

The code does have a way to turn off debugging, but only if a system property 
is set on every run. Since Drill has hundreds of unit tests that people might 
run, this requires creating custom run configurations for each.

This enhancement proposes to modify {{DrillBuf}} to enable logging only when a) 
assertions are enabled (so we know it is a debug environment) and b) when a 
boot-time configuration property is set (so that logging can be turned off for 
most developers most of the time.) Optionally, b) can be a system property that 
_enables_ (not disables) {{DrillBuf}} logging.

The code in question:
{code}
class BaseAllocator ... {
  public static final boolean DEBUG = AssertionUtil.isAssertionsEnabled()
  || Boolean.parseBoolean(System.getProperty(DEBUG_ALLOCATOR, "false"));
...
class DrillBuf ... {
  public boolean release(int decrement) {
...
if (BaseAllocator.DEBUG) {
  historicalLog.recordEvent("release(%d). original value: %d", decrement, 
refCnt + decrement);
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill issue #660: DRILL-5052: Option to debug generated Java code using an I...

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/660
  
Rebased on master as master had drifted along in the three weeks this PR 
has been active.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #690: DRILL-5121 A memory leak is observed when exact cas...

[bitblender]

GitHub user bitblender opened a pull request:

https://github.com/apache/drill/pull/690

DRILL-5121 A memory leak is observed when exact case is not specified for a 
column in a filter condition

Fix for https://issues.apache.org/jira/browse/DRILL-5121.

Changes fieldVectorMap in ScanBatch to a CaseInsensitiveMap to workaround 
the memory leak described in the bug.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bitblender/drill DRILL-5121

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/690.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #690


commit 8178e2ac620cdde534b8235e1b699b92bbad87c8
Author: karthik 
Date:   2016-11-14T18:36:53Z

DRILL-5121 Fix for memory leak. Changes fieldVectorMap in ScanBatch to a 
CaseInsensitiveMap




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---