[jira] [Commented] (DRILL-6088) MainLoginPageModel errors out when http.auth.mechanisms is not configured
[
https://issues.apache.org/jira/browse/DRILL-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331717#comment-16331717
]
ASF GitHub Bot commented on DRILL-6088:
---
Github user sohami commented on the issue:
https://github.com/apache/drill/pull/1092
@arina-ielchiieva - I have rebased the commit's on latest master and made
the change as suggested. I have also made the MainLoginPageModel class public
since it was causing the issue without that at runtime. Looks like during
testing I kept it as public but later while opening the PR I might have changed
it to package-private and was seeing issues today again during test.
> MainLoginPageModel errors out when http.auth.mechanisms is not configured
> -
>
> Key: DRILL-6088
> URL: https://issues.apache.org/jira/browse/DRILL-6088
> Project: Apache Drill
> Issue Type: Bug
> Components: Web Server
>Affects Versions: 1.13.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>Priority: Major
> Fix For: 1.13.0
>
>
> Reported by [~mandoskippy]:
> I am probably missing something minor here, but I am working with Ted
> Dunning on some PCAP plugin stuff, so I built his 1.13 SNAPSHOT, and when I
> try to login I see
> {
> "errorMessage" : "No configuration setting found for key
> 'drill.exec.http.auth'"
> }
>
> With DRILL-5425 SPNEGO supports was provided for Drill WebServer. With this
> check-in a backward compatibility change is missing in
> [MainLoginPageModel|https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/LogInLogOutResources.java#L151],
> where in absence of drill.exec.http.auth.mechanisms property it errors out.
> It should default to Form authentication in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6098) Make Drill Failure Handling Consistent
[ https://issues.apache.org/jira/browse/DRILL-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331620#comment-16331620 ] Timothy Farkas commented on DRILL-6098: --- Design Doc [https://docs.google.com/document/d/18RjRyrcla1d2REL8gu3nuwUsleU_Q66e-z54sTpvcS8] > Make Drill Failure Handling Consistent > -- > > Key: DRILL-6098 > URL: https://issues.apache.org/jira/browse/DRILL-6098 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Attachments: Drill Operator Error Handling Redesign.pdf > > > As described by [~Paul.Rogers] > " > We have multiple ways of reporting errors: > * Throw a UserException explaining the error > * Throw an unchecked exception and and let the fragment executor guess the > cause. > * Return STOP > * Tell the fragment executor to fail. (A we also required to return STOP?) > * Return OUT_OF_MEMORY status > The proposal is to replace them all with a single solution: throw a > UserException. Each operator catches other exceptions and translates them to > UserException. > Java unwinds the stack just fine; no reason for us to write code to do it via > STOP. > Then, the Fragment Executor calls close() on all operators. No reason to try > to do this cleanup on STOP. (Even if we do, the lower-level operators won't > have seen the STOP.) > Since failures are hard to test, and have cause no end of problems, having > multiple ways to do the same thing is really not that helpful to Drill users. > " -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331613#comment-16331613
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162521845
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -86,37 +117,35 @@
* Multiple threads of execution.
*
*/
-
public class OperatorFixture extends BaseFixture implements AutoCloseable {
+ public OperatorContext operatorContext(PhysicalOperator config) {
+return new MockOperatorContext(context, allocator(), config);
+ }
+
/**
* Builds an operator fixture based on a set of config options and
system/session
* options.
*/
-
- public static class OperatorFixtureBuilder
+ public static class Builder
--- End diff --
It's a nested class in the master branch currently
https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java
. So I didn't change that in this PR. Did you want me to pull it out into a
top level class as part of this PR?
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6098) Make Drill Failure Handling Consistent
[ https://issues.apache.org/jira/browse/DRILL-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331604#comment-16331604 ] Paul Rogers commented on DRILL-6098: Thanks for taking on this task! Posted a design document from an earlier attempt in case if offers any useful ideas. > Make Drill Failure Handling Consistent > -- > > Key: DRILL-6098 > URL: https://issues.apache.org/jira/browse/DRILL-6098 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Attachments: Drill Operator Error Handling Redesign.pdf > > > As described by [~Paul.Rogers] > " > We have multiple ways of reporting errors: > * Throw a UserException explaining the error > * Throw an unchecked exception and and let the fragment executor guess the > cause. > * Return STOP > * Tell the fragment executor to fail. (A we also required to return STOP?) > * Return OUT_OF_MEMORY status > The proposal is to replace them all with a single solution: throw a > UserException. Each operator catches other exceptions and translates them to > UserException. > Java unwinds the stack just fine; no reason for us to write code to do it via > STOP. > Then, the Fragment Executor calls close() on all operators. No reason to try > to do this cleanup on STOP. (Even if we do, the lower-level operators won't > have seen the STOP.) > Since failures are hard to test, and have cause no end of problems, having > multiple ways to do the same thing is really not that helpful to Drill users. > " -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6098) Make Drill Failure Handling Consistent
[ https://issues.apache.org/jira/browse/DRILL-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated DRILL-6098: --- Attachment: Drill Operator Error Handling Redesign.pdf > Make Drill Failure Handling Consistent > -- > > Key: DRILL-6098 > URL: https://issues.apache.org/jira/browse/DRILL-6098 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Attachments: Drill Operator Error Handling Redesign.pdf > > > As described by [~Paul.Rogers] > " > We have multiple ways of reporting errors: > * Throw a UserException explaining the error > * Throw an unchecked exception and and let the fragment executor guess the > cause. > * Return STOP > * Tell the fragment executor to fail. (A we also required to return STOP?) > * Return OUT_OF_MEMORY status > The proposal is to replace them all with a single solution: throw a > UserException. Each operator catches other exceptions and translates them to > UserException. > Java unwinds the stack just fine; no reason for us to write code to do it via > STOP. > Then, the Fragment Executor calls close() on all operators. No reason to try > to do this cleanup on STOP. (Even if we do, the lower-level operators won't > have seen the STOP.) > Since failures are hard to test, and have cause no end of problems, having > multiple ways to do the same thing is really not that helpful to Drill users. > " -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331575#comment-16331575
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162518120
--- Diff: exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java
---
@@ -58,23 +55,24 @@
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
-public class PlanningBase extends ExecTest{
- //private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(PlanningBase.class);
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
--- End diff --
Thanks!
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5879) Optimize "Like" operator
[
https://issues.apache.org/jira/browse/DRILL-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331571#comment-16331571
]
ASF GitHub Bot commented on DRILL-5879:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1072#discussion_r162517870
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/expr/fn/impl/TestSqlPatterns.java
---
@@ -446,6 +446,61 @@ public void testSqlPatternComplex() {
assertEquals(1, sqlPatternComplex.match(0, byteBuffer.limit(),
drillBuf)); // should match
}
+ @Test
+ public void testSqlPatternContainsMUltipleMatchers() {
--- End diff --
If tests exist, and you have verified that each of the lengths is, in fact,
tested by those tests, then I'm fine with adding a comment identifying the
cases and where to find the existing tests. The point is to make sure all
variations are fully tested -- one way or another.
> Optimize "Like" operator
>
>
> Key: DRILL-5879
> URL: https://issues.apache.org/jira/browse/DRILL-5879
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Environment: *
>Reporter: salim achouche
>Assignee: salim achouche
>Priority: Minor
> Fix For: 1.13.0
>
>
> Query: select from where colA like '%a%' or colA like
> '%xyz%';
> Improvement Opportunities
> # Avoid isAscii computation (full access of the input string) since we're
> dealing with the same column twice
> # Optimize the "contains" for-loop
> Implementation Details
> 1)
> * Added a new integer variable "asciiMode" to the VarCharHolder class
> * The default value is -1 which indicates this info is not known
> * Otherwise this value will be set to either 1 or 0 based on the string being
> in ASCII mode or Unicode
> * The execution plan already shares the same VarCharHolder instance for all
> evaluations of the same column value
> * The asciiMode will be correctly set during the first LIKE evaluation and
> will be reused across other LIKE evaluations
> 2)
> * The "Contains" LIKE operation is quite expensive as the code needs to
> access the input string to perform character based comparisons
> * Created 4 versions of the same for-loop to a) make the loop simpler to
> optimize (Vectorization) and b) minimize comparisons
> Benchmarks
> * Lineitem table 100GB
> * Query: select l_returnflag, count(*) from dfs.`` where l_comment
> not like '%a%' or l_comment like '%the%' group by l_returnflag
> * Before changes: 33sec
> * After changes: 27sec
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331567#comment-16331567
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162517423
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/xsort/managed/TestSorter.java
---
@@ -59,7 +59,7 @@
@Category(OperatorTest.class)
public class TestSorter extends DrillTest {
- public static OperatorFixture fixture;
+ public volatile static OperatorFixture fixture;
--- End diff --
Strange... So, if having volatile does solve a problem, just leave it in,
but maybe add a comment that includes you note above so that we know what's
what in the future.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[
https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331561#comment-16331561
]
ASF GitHub Bot commented on DRILL-6090:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1094#discussion_r162516548
--- Diff:
exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillConnectionImpl.java ---
@@ -132,10 +132,12 @@ protected DrillConnectionImpl(DriverImpl driver,
AvaticaFactory factory,
bit = new Drillbit(dConfig, serviceSet);
bit.run();
} catch (final UserException e) {
+cleanup();
throw new SQLException(
"Failure in starting embedded Drillbit: " + e.getMessage(),
e);
} catch (Exception e) {
+cleanup();
--- End diff --
One trick is this: used nested exceptions. The inner set "parses" the
exception types, the outer does cleanup:
```
try {
try {
// do something
} catch (ExceptionA e) {
// Do something
} catch (ExceptionB e) {
// Do something else
... }
} catch (Throwable t) {
cleanup();
throw t;
}
```
> While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of
> "Curator-Framework-0" threads are created if connection to drill-bit is not
> successful(no drill-bits are up/reachable)
> ---
>
> Key: DRILL-6090
> URL: https://issues.apache.org/jira/browse/DRILL-6090
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.12.0
> Environment: Centos 65, Java 7, Drill JDBC 1.12.0
>Reporter: Milind Takawale
>Assignee: Milind Takawale
>Priority: Major
> Fix For: 1.13.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the
> available drill-bits using Zookeepers. When drill-bits are not up or not
> reachable, the connection is failed with exception: "Failure in connecting to
> Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for
> client", which is expected, but number of threads created by
> ZKClusterCoordinator just keeps on increasing.
> Steps to reproduce the issue
> # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0
> through Zookeeper hosts(port 5181)
> # Now stop the drill-bit services or block the drill-bit IPs using iptable
> rules
> # Truncate catalina logs
> # Try to connect to the drill-bit/hit a code path that requires connection
> to drill-bits.
> # Take thread dump using kill -QUIT
> # grep -c "Curator-Framework-0" catalina.out
> Observe that the curator framework thread just keep on accumulating
> RCA:
> # ZKClusterCoordinator creates curator threads in the constructor
> # ZKClusterCoordinator is instantiated by DrillClient.connect
> # DrillClient.connect is called in DrillConnectionImpl constructor
> Fix:
> Call DrillConnectionImpl .cleanup() from all the catch blocks in the
> DrillConnectionImpl constructor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331545#comment-16331545
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r162512667
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java ---
@@ -278,22 +273,62 @@ public void addDoubleMetrics(OperatorProfile.Builder
builder) {
}
}
- @Override
+ /**
+ * Set a stat to the specified long value. Creates the stat
+ * if the stat does not yet exist.
+ *
+ * @param metric the metric to update
+ * @param value the value to set
+ */
+
public void addLongStat(MetricDef metric, long value){
longMetrics.putOrAdd(metric.metricId(), value, value);
}
- @Override
+ @VisibleForTesting
+ public long getLongStat(MetricDef metric) {
+return longMetrics.get(metric.metricId());
--- End diff --
Presumably fail. But, since this used in testing to ensure that a long
metric was, in fact, written, failure is actually a useful result.
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Major
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[ https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331546#comment-16331546 ] ASF GitHub Bot commented on DRILL-6049: --- Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1085#discussion_r162513175 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/ExecTest.java --- @@ -38,20 +38,26 @@ import org.apache.drill.exec.server.options.SystemOptionManager; import org.apache.drill.exec.store.sys.store.provider.LocalPersistentStoreProvider; import org.apache.drill.exec.util.GuavaPatcher; +import org.apache.drill.test.BaseDirTestWatcher; --- End diff -- Checked the source. No hidden characters. For me, formatting does appear below this line. I suspect your browser hit some limit. Large PRs make Safari struggle to display the code. Suggestion: refresh the page. search for this file, and keep going. Perhaps, if you don't expand prior files, your browser will handle the formatting. (GitHub should provide some solution for large PRs...) > Rollup of hygiene changes from "batch size" project > --- > > Key: DRILL-6049 > URL: https://issues.apache.org/jira/browse/DRILL-6049 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Fix For: 1.13.0 > > > PR to include all non-core changes related to the batch size project to allow > simpler review of the core changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331547#comment-16331547
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r162512386
--- Diff: common/src/main/java/org/apache/drill/common/types/Types.java ---
@@ -728,4 +733,49 @@ public static boolean isLaterType(MajorType type) {
return type.getMinorType() == MinorType.LATE;
}
+ public static boolean isEquivalent(MajorType type1, MajorType type2) {
+
+// Requires full type equality, including fields such as precision and
scale.
+// But, unset fields are equivalent to 0. Can't use the
protobuf-provided
+// isEquals() which treats set and unset fields as different.
+
+if (type1.getMinorType() != type2.getMinorType() ||
+type1.getMode() != type2.getMode() ||
+type1.getScale() != type2.getScale() ||
+type1.getPrecision() != type2.getPrecision()) {
+ return false;
+}
+
+// Subtypes are only for unions and are seldom used.
+
+if (type1.getMinorType() != MinorType.UNION) {
+ return true;
+}
+
+List subtypes1 = type1.getSubTypeList();
+List subtypes2 = type2.getSubTypeList();
+if (subtypes1 == subtypes2) { // Only occurs if both are null
+ return true;
+}
+if (subtypes1 == null || subtypes2 == null) {
+ return false;
+}
+if (subtypes1.size() != subtypes2.size()) {
+ return false;
+}
+
+// Now it gets slow because subtype lists are not ordered.
+
+List copy1 = new ArrayList<>();
+List copy2 = new ArrayList<>();
+copy1.addAll(subtypes1);
+copy2.addAll(subtypes2);
+Collections.sort(copy1);
+Collections.sort(copy2);
+return copy1.equals(copy2);
+ }
+
+ public static String typeKey(MinorType type) {
--- End diff --
Done.
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Major
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331544#comment-16331544
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r162512111
--- Diff:
common/src/main/java/org/apache/drill/common/exceptions/UserException.java ---
@@ -83,23 +83,17 @@ public static Builder memoryError() {
* The cause message will be used unless {@link
Builder#message(String, Object...)} is called.
* If the wrapped exception is, or wraps, a user exception it will be
returned by {@link Builder#build(Logger)}
* instead of creating a new exception. Any added context will be added
to the user exception as well.
- *
- * This exception, previously deprecated, has been repurposed to
indicate unspecified
- * errors. In particular, the case in which a lower level bit of code
throws an
- * exception other than UserException. The catching code then only knows
"something went
- * wrong", but not enough information to categorize the error.
- *
- * System errors also indicate illegal internal states, missing
functionality, and other
- * code-related errors -- all of which "should never occur."
*
* @see
org.apache.drill.exec.proto.UserBitShared.DrillPBError.ErrorType#SYSTEM
*
* @param cause exception we want the user exception to wrap. If cause
is, or wrap, a user exception it will be
* returned by the builder instead of creating a new user
exception
* @return user exception builder
*
+ * @deprecated This method should never need to be used explicitly,
unless you are passing the exception to the
+ * Rpc layer or UserResultListener.submitFailed()
*/
-
+ @Deprecated
--- End diff --
Good catch. Merge error. It is deprecated in master; I have undeprecated it
in my branch...
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Major
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6099) Drill does not push limit past project (flatten) if it cannot be pushed into scan
Gautam Kumar Parai created DRILL-6099:
-
Summary: Drill does not push limit past project (flatten) if it
cannot be pushed into scan
Key: DRILL-6099
URL: https://issues.apache.org/jira/browse/DRILL-6099
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.12.0
Reporter: Gautam Kumar Parai
Assignee: Gautam Kumar Parai
Fix For: 1.13.0
It would be useful to have pushdown occur past flatten(project). Here is an
example to illustrate the issue:
{{explain plan without implementation for }}{{select name, flatten(categories)
as category from dfs.`/tmp/t_json_20` LIMIT 1;}}
{{DrillScreenRel}}{{ }}
{{ DrillLimitRel(fetch=[1])}}{{ }}
{{ DrillProjectRel(name=[$0], category=[FLATTEN($1)])}}
{{ DrillScanRel(table=[[dfs, /tmp/t_json_20]], groupscan=[EasyGroupScan
[selectionRoot=maprfs:/tmp/t_json_20, numFiles=1, columns=[`name`,
`categories`], files=[maprfs:///tmp/t_json_20/0_0_0.json]]])}}
=
Content of 0_0_0.json
=
{
"name" : "Eric Goldberg, MD",
"categories" : [ "Doctors", "Health & Medical" ]
} {
"name" : "Pine Cone Restaurant",
"categories" : [ "Restaurants" ]
} {
"name" : "Deforest Family Restaurant",
"categories" : [ "American (Traditional)", "Restaurants" ]
} {
"name" : "Culver's",
"categories" : [ "Food", "Ice Cream & Frozen Yogurt", "Fast Food",
"Restaurants" ]
} {
"name" : "Chang Jiang Chinese Kitchen",
"categories" : [ "Chinese", "Restaurants" ]
}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331536#comment-16331536
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162513655
--- Diff:
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java
---
@@ -288,7 +288,7 @@ public static void populateVector(final ValueVector
vector, final DrillBuf manag
}
}
- public static MajorType getMajorTypeFromHiveTypeInfo(final TypeInfo
typeInfo, final OptionManager options) {
+ public static MajorType getMajorTypeFromHiveTypeInfo(final TypeInfo
typeInfo, final OptionSet options) {
--- End diff --
I'll flip to using the OptionManager.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331534#comment-16331534
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162513475
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
---
@@ -118,7 +118,7 @@ public IterOutcome innerNext() {
} catch(IOException ex) {
logger.error("Failure during query", ex);
kill(false);
- context.fail(ex);
+ context.getExecutorState().fail(ex);
--- End diff --
Sounds fun. Created https://issues.apache.org/jira/browse/DRILL-6098
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6098) Make Drill Failure Handling Consistent
Timothy Farkas created DRILL-6098: - Summary: Make Drill Failure Handling Consistent Key: DRILL-6098 URL: https://issues.apache.org/jira/browse/DRILL-6098 Project: Apache Drill Issue Type: Improvement Reporter: Timothy Farkas Assignee: Timothy Farkas As described by [~Paul.Rogers] " We have multiple ways of reporting errors: * Throw a UserException explaining the error * Throw an unchecked exception and and let the fragment executor guess the cause. * Return STOP * Tell the fragment executor to fail. (A we also required to return STOP?) * Return OUT_OF_MEMORY status The proposal is to replace them all with a single solution: throw a UserException. Each operator catches other exceptions and translates them to UserException. Java unwinds the stack just fine; no reason for us to write code to do it via STOP. Then, the Fragment Executor calls close() on all operators. No reason to try to do this cleanup on STOP. (Even if we do, the lower-level operators won't have seen the STOP.) Since failures are hard to test, and have cause no end of problems, having multiple ways to do the same thing is really not that helpful to Drill users. " -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331528#comment-16331528
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162512891
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
---
@@ -172,7 +172,7 @@ public RawFragmentBatch getNext() throws IOException {
} catch (final InterruptedException e) {
// We expect that the interrupt means the fragment is canceled or
failed, so we should kill this buffer
- if (!context.shouldContinue()) {
+ if (!context.getExecutorState().shouldContinue()) {
--- End diff --
ExecutorState is an interface. I removed those methods from the
FragmentContext interface because it felt redundant to replicate the
ExecutorState interface methods in the FragmentContext interface.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331526#comment-16331526
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162512429
--- Diff: exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java
---
@@ -58,23 +55,24 @@
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
-public class PlanningBase extends ExecTest{
- //private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(PlanningBase.class);
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
--- End diff --
https://issues.apache.org/jira/browse/DRILL-6097 will be the next step
forward to removing our dependence on Mocking libraries :).
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331525#comment-16331525
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162512220
--- Diff: exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java
---
@@ -58,23 +55,24 @@
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
-public class PlanningBase extends ExecTest{
- //private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(PlanningBase.class);
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
--- End diff --
This test was previously using JMockit and I have simply replaced it with
Mockito which is Eclipse friendly and a step forward for developers using
Eclipse. So the amount of Mocking going on here is the same :) the only
difference is that Eclipse users should now be able to run these tests. I have
also completely removed mocking from some tests which no longer need it. As we
take more steps to properly use interfaces for more classes, we can
incrementally remove Mockito from even more tests. However, this is an
incremental process and shouldn't be done all in one shot.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331518#comment-16331518
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162511432
--- Diff: exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java
---
@@ -84,28 +82,17 @@ protected void testSqlPlan(String sqlCommands) throws
Exception {
systemOptions.init();
@SuppressWarnings("resource")
final UserSession userSession =
UserSession.Builder.newBuilder().withOptionManager(systemOptions).build();
-final SessionOptionManager sessionOptions = (SessionOptionManager)
userSession.getOptions();
+final SessionOptionManager sessionOptions = userSession.getOptions();
final QueryOptionManager queryOptions = new
QueryOptionManager(sessionOptions);
final ExecutionControls executionControls = new
ExecutionControls(queryOptions, DrillbitEndpoint.getDefaultInstance());
-new NonStrictExpectations() {
- {
-dbContext.getMetrics();
-result = new MetricRegistry();
-dbContext.getAllocator();
-result = allocator;
-dbContext.getConfig();
-result = config;
-dbContext.getOptionManager();
-result = systemOptions;
-dbContext.getStoreProvider();
-result = provider;
-dbContext.getClasspathScan();
-result = scanResult;
-dbContext.getLpPersistence();
-result = logicalPlanPersistence;
- }
-};
+when(dbContext.getMetrics()).thenReturn(new MetricRegistry());
+when(dbContext.getAllocator()).thenReturn(allocator);
+when(dbContext.getConfig()).thenReturn(config);
+when(dbContext.getOptionManager()).thenReturn(systemOptions);
+when(dbContext.getStoreProvider()).thenReturn(provider);
+when(dbContext.getClasspathScan()).thenReturn(scanResult);
+when(dbContext.getLpPersistence()).thenReturn(logicalPlanPersistence);
--- End diff --
I completely agree with this in the long term. Since this code deals with
the QueryContext I did not dive into creating an interface for the QueryContext
and a Mock implementation of the interface. I want to limit the scope of this
PR since it is already quite large, but I have created a follow up ticket to
handle the mocking of the QueryContext correctly
https://issues.apache.org/jira/browse/DRILL-6097 . As an additional note, the
only reason why this code was changed was to take a step towards removing
JMockit which does not play nice with Eclipse as you have noticed. Removing
JMockit and replacing it with Mockito was the easiest thing to do at this
point. Once this change goes in we can revisit the QueryContext in DRILL-6097.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6097) Create an interface for the QueryContext
Timothy Farkas created DRILL-6097: - Summary: Create an interface for the QueryContext Key: DRILL-6097 URL: https://issues.apache.org/jira/browse/DRILL-6097 Project: Apache Drill Issue Type: Improvement Environment: Currently the QueryContext does not implement an interface and the concrete class is passed around everywhere. Additionally Mockito is used in tests to mock it. Ideally we would make the QueryContext implement an interface and create a mock implementation of it that is used in the tests, just like what we did for the FragmentContext. Reporter: Timothy Farkas Assignee: Timothy Farkas -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5879) Optimize "Like" operator
[ https://issues.apache.org/jira/browse/DRILL-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331459#comment-16331459 ] ASF GitHub Bot commented on DRILL-5879: --- Github user priteshm commented on the issue: https://github.com/apache/drill/pull/1072 @paul-rogers Is this ready for commit? > Optimize "Like" operator > > > Key: DRILL-5879 > URL: https://issues.apache.org/jira/browse/DRILL-5879 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Relational Operators > Environment: * >Reporter: salim achouche >Assignee: salim achouche >Priority: Minor > Fix For: 1.13.0 > > > Query: select from where colA like '%a%' or colA like > '%xyz%'; > Improvement Opportunities > # Avoid isAscii computation (full access of the input string) since we're > dealing with the same column twice > # Optimize the "contains" for-loop > Implementation Details > 1) > * Added a new integer variable "asciiMode" to the VarCharHolder class > * The default value is -1 which indicates this info is not known > * Otherwise this value will be set to either 1 or 0 based on the string being > in ASCII mode or Unicode > * The execution plan already shares the same VarCharHolder instance for all > evaluations of the same column value > * The asciiMode will be correctly set during the first LIKE evaluation and > will be reused across other LIKE evaluations > 2) > * The "Contains" LIKE operation is quite expensive as the code needs to > access the input string to perform character based comparisons > * Created 4 versions of the same for-loop to a) make the loop simpler to > optimize (Vectorization) and b) minimize comparisons > Benchmarks > * Lineitem table 100GB > * Query: select l_returnflag, count(*) from dfs.`` where l_comment > not like '%a%' or l_comment like '%the%' group by l_returnflag > * Before changes: 33sec > * After changes: 27sec -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-2556) support delimiters of multiple characters
[ https://issues.apache.org/jira/browse/DRILL-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331445#comment-16331445 ] Kunal Khatua commented on DRILL-2556: - I think DRILL-4660 addresses this. Can you confirm, [~vicenteg]? > support delimiters of multiple characters > - > > Key: DRILL-2556 > URL: https://issues.apache.org/jira/browse/DRILL-2556 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Text CSV >Affects Versions: 0.7.0 >Reporter: Vince Gonzalez >Priority: Major > Fix For: Future > > > Support for multi-character delimiters would be nice to have. Here's a sample > data set: https://gist.github.com/vicenteg/111372a42b0989d43dca -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331425#comment-16331425
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162501868
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/xsort/managed/TestSorter.java
---
@@ -59,7 +59,7 @@
@Category(OperatorTest.class)
public class TestSorter extends DrillTest {
- public static OperatorFixture fixture;
+ public volatile static OperatorFixture fixture;
--- End diff --
I was running into an issue where the fixture variable was consistently
null when the test ran. This seemed impossible and I hypothesized that JUnit is
secretly using two threads, one to call the static initializers and then
another to actually execute the test methods. So I made the variable volatile
and the issue went away. I agree this is weird and it raises the question why
other tests don't have the same issue. I'll remove the volatile for now, if the
bizarre issue surfaces again, I will get more details and post them here.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6027) Implement spill to disk for the Hash Join
[ https://issues.apache.org/jira/browse/DRILL-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331416#comment-16331416 ] Boaz Ben-Zvi commented on DRILL-6027: - The code changes up to now are at: [https://github.com/apache/drill/compare/master...Ben-Zvi:HashJoinSpill] The work includes splitting the Hash-Join work into partitions (both Build and Probe sides), and replacing the generated code for moving rows around with the new appendRow() calls. All the user regressions are now passing. > Implement spill to disk for the Hash Join > - > > Key: DRILL-6027 > URL: https://issues.apache.org/jira/browse/DRILL-6027 > Project: Apache Drill > Issue Type: New Feature > Components: Execution - Relational Operators >Affects Versions: 1.11.0 >Reporter: Boaz Ben-Zvi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.13.0 > > > Implement the spill memory to disk (as needed) feature for the Hash Join > operator (similar to the prior work on the Hash Aggregate). > A design draft document has been published: > [https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing] > Functional spec is available: > [https://docs.google.com/document/d/1bPAddVCRxKHxi2RjqUvISIWXNqLdbRzZT9CWmanh4h0/edit] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331411#comment-16331411
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162499201
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/store/TestAffinityCalculator.java
---
@@ -21,18 +21,14 @@
import org.apache.drill.exec.ExecTest;
import org.apache.drill.exec.proto.CoordinationProtos;
-import org.apache.drill.exec.store.parquet.ParquetGroupScan;
import org.apache.hadoop.fs.BlockLocation;
import org.junit.Test;
import com.google.common.collect.ImmutableRangeMap;
import com.google.common.collect.Range;
public class TestAffinityCalculator extends ExecTest {
- static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(TestAffinityCalculator.class);
-
- String port = "1234";
- final String path = "path";
+ private final String port = "1234";
--- End diff --
- Two tests were blatantly commented out.
- testSetEndpointBytes
- testApplyAssignments
- Several methods were unused according to IntelliJ and the java compiler.
- buildRowGroups
- buildEndpoints
- buildBlockLocations2
After removing the unused code there is shockingly almost nothing left.
There are basically no functioning unit tests for the AffinityCalculator.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-3993) Rebase Drill on Calcite master branch
[ https://issues.apache.org/jira/browse/DRILL-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331410#comment-16331410 ] ASF GitHub Bot commented on DRILL-3993: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1066 > Rebase Drill on Calcite master branch > - > > Key: DRILL-3993 > URL: https://issues.apache.org/jira/browse/DRILL-3993 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.2.0 >Reporter: Sudheesh Katkam >Assignee: Roman Kulyk >Priority: Major > > Calcite keeps moving, and now we need to catch up to Calcite 1.5, and ensure > there are no regressions. > Also, how do we resolve this 'catching up' issue in the long term? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331406#comment-16331406
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162497247
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -86,37 +117,35 @@
* Multiple threads of execution.
*
*/
-
public class OperatorFixture extends BaseFixture implements AutoCloseable {
+ public OperatorContext operatorContext(PhysicalOperator config) {
+return new MockOperatorContext(context, allocator(), config);
+ }
+
/**
* Builds an operator fixture based on a set of config options and
system/session
* options.
*/
-
- public static class OperatorFixtureBuilder
+ public static class Builder
--- End diff --
Yes, of course. But one of the changes was to pull this out to a new
top-level class. In that case, "Builder" by itself is rather generic.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331403#comment-16331403
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162496772
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -86,37 +117,35 @@
* Multiple threads of execution.
*
*/
-
public class OperatorFixture extends BaseFixture implements AutoCloseable {
+ public OperatorContext operatorContext(PhysicalOperator config) {
+return new MockOperatorContext(context, allocator(), config);
+ }
+
/**
* Builds an operator fixture based on a set of config options and
system/session
* options.
*/
-
- public static class OperatorFixtureBuilder
+ public static class Builder
--- End diff --
There are no collisions. Java has a nice syntax for handling names like
this. Specifically you can use this name for the class
**OperatorFixture.Builder** in all your variable declarations. This removes the
redundancy of prefixing the name of an inner class with the name of the outer
class.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6096) Provide mechanisms to specify field delimiters and quoted text for TextRecordWriter
Kunal Khatua created DRILL-6096: --- Summary: Provide mechanisms to specify field delimiters and quoted text for TextRecordWriter Key: DRILL-6096 URL: https://issues.apache.org/jira/browse/DRILL-6096 Project: Apache Drill Issue Type: Bug Components: Storage - Text CSV Affects Versions: 1.12.0 Reporter: Kunal Khatua Currently, there is no way for a user to specify the field delimiter for the writing records as a text output. Further more, if the fields contain the delimiter, we have no mechanism of specifying quotes. By default, quotes should be used to enclose non-numeric fields being written. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331397#comment-16331397
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162495937
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -127,20 +156,26 @@ public OperatorFixture build() {
* uses the same code generation mechanism as the full Drill, but
* provide test-specific versions of various other services.
*/
-
- public static class TestFragmentContext extends BaseFragmentContext {
-
+ public static class MockFragmentContext extends BaseFragmentContext {
--- End diff --
I like prefixing Mock classes with **Mock** instead of **Test**.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331395#comment-16331395
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162495701
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -351,8 +603,110 @@ public OperatorStats getStats() {
}
}
- public OperatorContext operatorContext(PhysicalOperator config) {
-return new TestOperatorContext(context, allocator(), config, stats);
+ public static class MockPhysicalOperator implements PhysicalOperator {
--- End diff --
Thanks for catching this. This class is left over from some previous work
and is not used. I've deleted it.
> Fix Unit Test failures on JDK 8 And Some JDK 7 versions
> ---
>
> Key: DRILL-5730
> URL: https://issues.apache.org/jira/browse/DRILL-5730
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
>
> Tests fail on JDK 8 and oracle JDK 7 on my mac
> Failed tests:
> TestMetadataProvider.tables:153 expected: but was:
> TestMetadataProvider.tablesWithTableNameFilter:212 expected: but
> was:
> TestMetadataProvider.tablesWithSystemTableFilter:187 expected: but
> was:
> TestMetadataProvider.tablesWithTableFilter:176 expected: but
> was:
> Tests in error:
> TestInfoSchema.selectFromAllTables » UserRemote SYSTEM ERROR:
> URISyntaxExcepti...
> TestCustomUserAuthenticator.positiveUserAuth » UserRemote SYSTEM ERROR:
> URISyn...
> TestCustomUserAuthenticator.positiveUserAuthAfterNegativeUserAuth »
> UserRemote
> TestViewSupport.infoSchemaWithView:350->BaseTestQuery.testRunAndReturn:344
> » Rpc
> TestParquetScan.testSuccessFile:58->BaseTestQuery.testRunAndReturn:344 »
> Rpc o...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5730) Fix Unit Test failures on JDK 8 And Some JDK 7 versions
[
https://issues.apache.org/jira/browse/DRILL-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331394#comment-16331394
]
ASF GitHub Bot commented on DRILL-5730:
---
Github user ilooner commented on a diff in the pull request:
https://github.com/apache/drill/pull/1045#discussion_r162495593
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/OperatorFixture.java ---
@@ -175,19 +210,189 @@ public DrillConfig getConfig() {
}
@Override
-public DrillbitContext getDrillbitContext() {
- throw new UnsupportedOperationException("Drillbit context not
available for operator unit tests");
+public ExecutorService getScanDecodeExecutor() {
+ return null;
+}
+
+@Override
+public ExecutorService getScanExecutor() {
+ return null;
+}
+
+@Override
+public ExecutorService getExecutor() {
+ return null;
+}
+
+@Override
+public ExecutorState getExecutorState() {
+ return executorState;
+}
+
+@Override
+public BufferAllocator getNewChildAllocator(String operatorName, int
operatorId,
+long initialReservation,
long maximumReservation) {
+ return allocator.newChildAllocator(
+"op:" + operatorId + ":" + operatorName,
+initialReservation,
+maximumReservation);
}
@Override
-protected CodeCompiler getCompiler() {
+public ExecProtos.FragmentHandle getHandle() {
+ return ExecProtos.FragmentHandle.newBuilder().build();
+}
+
+@Override
+public BufferAllocator getAllocator() {
+ return allocator;
+}
+
+@Override
+public OperatorContext newOperatorContext(PhysicalOperator popConfig) {
+ return mockOperatorContext;
+}
+
+@Override
+public OperatorContext newOperatorContext(PhysicalOperator popConfig,
OperatorStats stats) {
+ return mockOperatorContext;
+}
+
+@Override
+public SchemaPlus getFullRootSchema() {
+ return null;
+}
+
+@Override
+public String getQueryUserName() {
+ return null;
+}
+
+@Override
+public String getFragIdString() {
+ return null;
+}
+
+@Override
+public CodeCompiler getCompiler() {
return compiler;
}
@Override
protected BufferManager getBufferManager() {
return bufferManager;
}
+
+@Override
+public void close() {
+ bufferManager.close();
+}
+
+@Override
+public ContextInformation getContextInformation() {
+ return null;
+}
+
+@Override
+public PartitionExplorer getPartitionExplorer() {
+ return null;
+}
+
+@Override
+public ValueHolder getConstantValueHolder(String value,
TypeProtos.MinorType type, Function holderInitializer) {
+ return null;
+}
+ }
+
+ public static class MockExecutorFragmentContext extends
MockFragmentContext implements ExecutorFragmentContext {
+
+public MockExecutorFragmentContext(DrillConfig config, OptionManager
optionManager, BufferAllocator allocator) {
+ super(config, optionManager, allocator);
+}
+
+@Override
+public PhysicalPlanReader getPlanReader() {
+ throw new UnsupportedOperationException();
+}
+
+@Override
+public ClusterCoordinator getClusterCoordinator() {
+ throw new UnsupportedOperationException();
+}
+
+@Override
+public CoordinationProtos.DrillbitEndpoint getForemanEndpoint() {
+ throw new UnsupportedOperationException();
+}
+
+@Override
+public CoordinationProtos.DrillbitEndpoint getEndpoint() {
+ throw new UnsupportedOperationException();
+}
+
+@Override
+public Collection getBits() {
+ throw new UnsupportedOperationException();
+}
+
+@Override
+public OperatorCreatorRegistry getOperatorCreatorRegistry() {
+ return null;
+}
+
+@Override
+public void setBuffers(IncomingBuffers buffers) {
+
+}
+
+@Override
+public Set> getUserConnections() {
+ return null;
+}
+
+@Override
+public QueryProfileStoreContext getProfileStoreContext() {
+ return null;
+}
+
+@Override
+public void waitForSendComplete() {
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331280#comment-16331280
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1084#discussion_r162478583
--- Diff: exec/java-exec/src/main/resources/rest/profile/profile.ftl ---
@@ -77,16 +84,17 @@ table.sortable thead .sorting_desc { background-image:
url("/static/img/black-de
<#if model.isOnlyImpersonationEnabled()>
+ ${model.getProfile().query}
+
User Name
-
-${model.getProfile().query}
-
+ ${model.getProfile().query}
+
--- End diff --
Patched and Verified that with Impersonation enabled, there is only one
editor for the profile page.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6032) Use RecordBatchSizer to estimate size of columns in HashAgg
[ https://issues.apache.org/jira/browse/DRILL-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6032: - Description: We need to use the RecordBatchSize to estimate the size of columns in the Partition batches created by HashAgg. (was: [~timothyfarkas] please do add a description of this change.) > Use RecordBatchSizer to estimate size of columns in HashAgg > --- > > Key: DRILL-6032 > URL: https://issues.apache.org/jira/browse/DRILL-6032 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.13.0 > > > We need to use the RecordBatchSize to estimate the size of columns in the > Partition batches created by HashAgg. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6032) Use RecordBatchSizer to estimate size of columns in HashAgg
[ https://issues.apache.org/jira/browse/DRILL-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6032: - Description: [~timothyfarkas] please do add a description of this change. > Use RecordBatchSizer to estimate size of columns in HashAgg > --- > > Key: DRILL-6032 > URL: https://issues.apache.org/jira/browse/DRILL-6032 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.13.0 > > > [~timothyfarkas] please do add a description of this change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6027) Implement spill to disk for the Hash Join
[ https://issues.apache.org/jira/browse/DRILL-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331267#comment-16331267 ] Pritesh Maker commented on DRILL-6027: -- [~ben-zvi] ,[~timothyfarkas] - please do share the link to the work in progress branch. > Implement spill to disk for the Hash Join > - > > Key: DRILL-6027 > URL: https://issues.apache.org/jira/browse/DRILL-6027 > Project: Apache Drill > Issue Type: New Feature > Components: Execution - Relational Operators >Affects Versions: 1.11.0 >Reporter: Boaz Ben-Zvi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.13.0 > > > Implement the spill memory to disk (as needed) feature for the Hash Join > operator (similar to the prior work on the Hash Aggregate). > A design draft document has been published: > [https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing] > Functional spec is available: > [https://docs.google.com/document/d/1bPAddVCRxKHxi2RjqUvISIWXNqLdbRzZT9CWmanh4h0/edit] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6027) Implement spill to disk for the Hash Join
[ https://issues.apache.org/jira/browse/DRILL-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6027: - Description: Implement the spill memory to disk (as needed) feature for the Hash Join operator (similar to the prior work on the Hash Aggregate). A design draft document has been published: [https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing] Functional spec is available: [https://docs.google.com/document/d/1bPAddVCRxKHxi2RjqUvISIWXNqLdbRzZT9CWmanh4h0/edit] was: Implement the spill memory to disk (as needed) feature for the Hash Join operator (similar to the prior work on the Hash Aggregate). A design draft document has been published: https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing > Implement spill to disk for the Hash Join > - > > Key: DRILL-6027 > URL: https://issues.apache.org/jira/browse/DRILL-6027 > Project: Apache Drill > Issue Type: New Feature > Components: Execution - Relational Operators >Affects Versions: 1.11.0 >Reporter: Boaz Ben-Zvi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.13.0 > > > Implement the spill memory to disk (as needed) feature for the Hash Join > operator (similar to the prior work on the Hash Aggregate). > A design draft document has been published: > [https://docs.google.com/document/d/1-c_oGQY4E5d58qJYv_zc7ka834hSaB3wDQwqKcMoSAI/edit?usp=sharing] > Functional spec is available: > [https://docs.google.com/document/d/1bPAddVCRxKHxi2RjqUvISIWXNqLdbRzZT9CWmanh4h0/edit] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunhui Shi updated DRILL-6078: --- Labels: (was: ready-to-commit) > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi >Priority: Major > Fix For: 1.13.0 > > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4834) decimal implementation is vulnerable to overflow errors, and extremely complex
[ https://issues.apache.org/jira/browse/DRILL-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331053#comment-16331053 ] ASF GitHub Bot commented on DRILL-4834: --- Github user daveoshinsky commented on the issue: https://github.com/apache/drill/pull/570 The new VARDECIMAL one-size-fits-all decimal type, which this PR implements, will now be incorporated into the following new JIRA with additional changes and fixes for Drill 1.13: https://issues.apache.org/jira/browse/DRILL-6094 So, development on this PR will now cease. But VARDECIMAL lives on Dave Oshinsky > decimal implementation is vulnerable to overflow errors, and extremely complex > -- > > Key: DRILL-4834 > URL: https://issues.apache.org/jira/browse/DRILL-4834 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.6.0 > Environment: Drill 1.7 on any platform >Reporter: Dave Oshinsky >Assignee: Dave Oshinsky >Priority: Major > Fix For: 1.13.0 > > > While working on a fix for DRILL-4704, logic was added to CastIntDecimal.java > template to handle the situation where a precision is not supplied (i.e., the > supplied precision is zero) for an integer value that is to be casted to a > decimal. The Drill decimal implementation uses a limited selection of fixed > decimal precision data types (the total number of decimal digits, i.e., > Decimal9, 18, 28, 38) to represent decimal values. If the destination > precision is too small to represent the input integer that is being casted, > there is no clean way to deal with the overflow error properly. > While using fixed decimal precisions as is being done currently can lead to > more efficient use of memory, it often will actually lead to less efficient > use of memory (when the fixed precision is specified significantly larger > than is actually needed to represent the numbers), and it results in a > tremendous mushrooming of the complexity of the code. For each fixed > precision (and there are only a limited set of selections, 9, 18, 28, 38, > which itself leads to memory inefficiency), there is a separate set of code > generated from templates. For each pairwise combination of decimal or > non-decimal numeric types, there are multiple places in the code where > conversions must be handled, or conditions must be included to handle the > difference in precision between the two types. A one-size-fits-all approach > (using a variable width vector to represent any decimal precision) would > usually be more memory-efficient (since precisions are often over-specified), > and would greatly simplify the code. > Also see the DRILL-4184 issue, which is related. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dr
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker reassigned DRILL-6090: Assignee: Milind Takawale > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Assignee: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5994) Cannot start web server on a machine with more than 200 cores
[
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330962#comment-16330962
]
Mitchel Labonte commented on DRILL-5994:
[~priteshm] I applied the fix I proposed on my installation and it works, I
didn't have time to try [~vrozov]'s proposal, but I assume it would probably
work too indeed.
> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Mitchel Labonte
>Priority: Minor
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you
> get the following stack trace:
> {noformat}
> Exception in thread "main"
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is
> created with the default constructor, which initializes a QueuedThreadPool
> with a default maxThreads value of 200, and there is no way to configure this
> value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6004) Direct buffer bounds checking should be disabled by default
[
https://issues.apache.org/jira/browse/DRILL-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6004:
-
Labels: docs-impacting ready-to-commit (was: doc-impacting ready-to-commit)
> Direct buffer bounds checking should be disabled by default
> ---
>
> Key: DRILL-6004
> URL: https://issues.apache.org/jira/browse/DRILL-6004
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Labels: docs-impacting, ready-to-commit
>
> Direct buffer bounds checking is enabled either when assertions are enabled
> (see DRILL-6001) or when {{drill.enable_unsafe_memory_access}} property is
> not set to true, so it is enabled in production as by default
> {{drill.enable_unsafe_memory_access}} is not set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6004) Direct buffer bounds checking should be disabled by default
[
https://issues.apache.org/jira/browse/DRILL-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-6004:
Labels: doc-impacting ready-to-commit (was: ready-to-commit)
> Direct buffer bounds checking should be disabled by default
> ---
>
> Key: DRILL-6004
> URL: https://issues.apache.org/jira/browse/DRILL-6004
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Labels: doc-impacting, ready-to-commit
>
> Direct buffer bounds checking is enabled either when assertions are enabled
> (see DRILL-6001) or when {{drill.enable_unsafe_memory_access}} property is
> not set to true, so it is enabled in production as by default
> {{drill.enable_unsafe_memory_access}} is not set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5994) Cannot start web server on a machine with more than 200 cores
[
https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330945#comment-16330945
]
Pritesh Maker commented on DRILL-5994:
--
[~arina], [~Mitchel] do you have any more comments on this one? Seems like
[~vrozov] is saying that we should have an alternate fix.
> Cannot start web server on a machine with more than 200 cores
> -
>
> Key: DRILL-5994
> URL: https://issues.apache.org/jira/browse/DRILL-5994
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Mitchel Labonte
>Assignee: Mitchel Labonte
>Priority: Minor
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you
> get the following stack trace:
> {noformat}
> Exception in thread "main"
> org.apache.drill.exec.exception.DrillStartupException: Failure during initial
> startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in
> ThreadPool: max=200 < needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is
> created with the default constructor, which initializes a QueuedThreadPool
> with a default maxThreads value of 200, and there is no way to configure this
> value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
[
https://issues.apache.org/jira/browse/DRILL-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sekhon updated DRILL-6095:
---
Description:
Drill's documentation on PStore configuration to HBase shows sample config with
4 ZooKeeper ip addresses in the {{hbase.zookeeper.quorum field}}, this should
be fixed to 3 or 5 as per ZooKeeper best practices.
[https://drill.apache.org/docs/persistent-configuration-storage/]
was:
Drill's documentation on PStore configuration to HBase shows sample config with
4 ZooKeeper servers in a quorum, this should be fixed to 3 or 5 as per
ZooKeeper best practices.
[https://drill.apache.org/docs/persistent-configuration-storage/]
> Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per
> standard practice
> ---
>
> Key: DRILL-6095
> URL: https://issues.apache.org/jira/browse/DRILL-6095
> Project: Apache Drill
> Issue Type: Improvement
> Components: Documentation
>Affects Versions: 1.12.0
> Environment: MapR 5.2
>
>Reporter: Hari Sekhon
>Priority: Trivial
>
> Drill's documentation on PStore configuration to HBase shows sample config
> with 4 ZooKeeper ip addresses in the {{hbase.zookeeper.quorum field}}, this
> should be fixed to 3 or 5 as per ZooKeeper best practices.
> [https://drill.apache.org/docs/persistent-configuration-storage/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
[ https://issues.apache.org/jira/browse/DRILL-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated DRILL-6095: --- Description: Drill's documentation on PStore configuration to HBase shows sample config with 4 ZooKeeper servers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. [https://drill.apache.org/docs/persistent-configuration-storage/] was: Drill's documentation on PStore configuration to HBase shows sample config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. [https://drill.apache.org/docs/persistent-configuration-storage/] > Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per > standard practice > --- > > Key: DRILL-6095 > URL: https://issues.apache.org/jira/browse/DRILL-6095 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.12.0 > Environment: MapR 5.2 > >Reporter: Hari Sekhon >Priority: Trivial > > Drill's documentation on PStore configuration to HBase shows sample config > with 4 ZooKeeper servers in a quorum, this should be fixed to 3 or 5 as per > ZooKeeper best practices. > [https://drill.apache.org/docs/persistent-configuration-storage/] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
[ https://issues.apache.org/jira/browse/DRILL-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated DRILL-6095: --- Description: Drill's documentation on PStore configuration to HBase shows sample config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. [https://drill.apache.org/docs/persistent-configuration-storage/] was: Drill's PStore doc for HBase shows config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. [https://drill.apache.org/docs/persistent-configuration-storage/] > Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per > standard practice > --- > > Key: DRILL-6095 > URL: https://issues.apache.org/jira/browse/DRILL-6095 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.12.0 > Environment: MapR 5.2 > >Reporter: Hari Sekhon >Priority: Trivial > > Drill's documentation on PStore configuration to HBase shows sample config > with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per > ZooKeeper best practices. > [https://drill.apache.org/docs/persistent-configuration-storage/] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
[ https://issues.apache.org/jira/browse/DRILL-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated DRILL-6095: --- Description: Drill's PStore doc for HBase shows config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. [https://drill.apache.org/docs/persistent-configuration-storage/] > Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per > standard practice > --- > > Key: DRILL-6095 > URL: https://issues.apache.org/jira/browse/DRILL-6095 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.12.0 > Environment: MapR 5.2 > >Reporter: Hari Sekhon >Priority: Trivial > > Drill's PStore doc for HBase shows config with 4 zookeepers in a quorum, this > should be fixed to 3 or 5 as per ZooKeeper best practices. > [https://drill.apache.org/docs/persistent-configuration-storage/] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
Hari Sekhon created DRILL-6095: -- Summary: Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice Key: DRILL-6095 URL: https://issues.apache.org/jira/browse/DRILL-6095 Project: Apache Drill Issue Type: Improvement Components: Documentation Affects Versions: 1.12.0 Environment: Drill's PStore doc for HBase shows config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. https://drill.apache.org/docs/persistent-configuration-storage/ Reporter: Hari Sekhon -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6095) Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per standard practice
[ https://issues.apache.org/jira/browse/DRILL-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated DRILL-6095: --- Environment: MapR 5.2 was: Drill's PStore doc for HBase shows config with 4 zookeepers in a quorum, this should be fixed to 3 or 5 as per ZooKeeper best practices. https://drill.apache.org/docs/persistent-configuration-storage/ > Drill PStore HBase doc shows 4 zookeepers in quorum, fix to 3 or 5 as per > standard practice > --- > > Key: DRILL-6095 > URL: https://issues.apache.org/jira/browse/DRILL-6095 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.12.0 > Environment: MapR 5.2 > >Reporter: Hari Sekhon >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-3993) Rebase Drill on Calcite master branch
[ https://issues.apache.org/jira/browse/DRILL-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330682#comment-16330682 ] ASF GitHub Bot commented on DRILL-3993: --- Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/1066 FYI: changes in pom files: We observed issue with Avatica similar to the issue described in https://issues.apache.org/jira/browse/CALCITE-1694. Therefore we had to make changes in our pom files similar to the changes, made in CALCITE-1694 to fix NoClassDefFoundError. > Rebase Drill on Calcite master branch > - > > Key: DRILL-3993 > URL: https://issues.apache.org/jira/browse/DRILL-3993 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.2.0 >Reporter: Sudheesh Katkam >Assignee: Roman Kulyk >Priority: Major > > Calcite keeps moving, and now we need to catch up to Calcite 1.5, and ensure > there are no regressions. > Also, how do we resolve this 'catching up' issue in the long term? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330665#comment-16330665
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1084#discussion_r162384019
--- Diff: exec/java-exec/src/main/resources/rest/profile/profile.ftl ---
@@ -77,16 +84,17 @@ table.sortable thead .sorting_desc { background-image:
url("/static/img/black-de
<#if model.isOnlyImpersonationEnabled()>
+ ${model.getProfile().query}
+
User Name
-
-${model.getProfile().query}
-
+ ${model.getProfile().query}
+
--- End diff --
Let me take a look. This might have slipped past during a merge conflict.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-5798) Fix Flakey Tests
[ https://issues.apache.org/jira/browse/DRILL-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-5798: Labels: ready-to-commit (was: ) > Fix Flakey Tests > > > Key: DRILL-5798 > URL: https://issues.apache.org/jira/browse/DRILL-5798 > Project: Apache Drill > Issue Type: Bug >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.12.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-5798) Fix Flakey Tests
[ https://issues.apache.org/jira/browse/DRILL-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-5798: Fix Version/s: (was: 1.13.0) 1.12.0 > Fix Flakey Tests > > > Key: DRILL-5798 > URL: https://issues.apache.org/jira/browse/DRILL-5798 > Project: Apache Drill > Issue Type: Bug >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.12.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-5798) Fix Flakey Tests
[ https://issues.apache.org/jira/browse/DRILL-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-5798: Fix Version/s: 1.13.0 > Fix Flakey Tests > > > Key: DRILL-5798 > URL: https://issues.apache.org/jira/browse/DRILL-5798 > Project: Apache Drill > Issue Type: Bug >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.12.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6088) MainLoginPageModel errors out when http.auth.mechanisms is not configured
[
https://issues.apache.org/jira/browse/DRILL-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330486#comment-16330486
]
ASF GitHub Bot commented on DRILL-6088:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1092#discussion_r162331112
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/LogInLogOutResources.java
---
@@ -132,24 +133,26 @@ public Viewable getMainLoginPage(@Context
HttpServletRequest request, @Context H
@Context SecurityContext sc, @Context
UriInfo uriInfo,
@QueryParam(WebServerConstants.REDIRECT_QUERY_PARM) String redirect) throws
Exception {
updateSessionRedirectInfo(redirect, request);
-final DrillConfig drillConfig = workManager.getContext().getConfig();
-MainLoginPageModel model = new MainLoginPageModel(null, drillConfig);
+final MainLoginPageModel model = new MainLoginPageModel(null);
return ViewableWithPermissions.createMainLoginPage(model);
}
- private class MainLoginPageModel {
+ @VisibleForTesting
+ class MainLoginPageModel {
private final String error;
private final boolean authEnabled;
+private final DrillConfig config;
--- End diff --
It looks like you are using config only in constructor, so it can not store
it in class.
> MainLoginPageModel errors out when http.auth.mechanisms is not configured
> -
>
> Key: DRILL-6088
> URL: https://issues.apache.org/jira/browse/DRILL-6088
> Project: Apache Drill
> Issue Type: Bug
> Components: Web Server
>Affects Versions: 1.13.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>Priority: Major
> Fix For: 1.13.0
>
>
> Reported by [~mandoskippy]:
> I am probably missing something minor here, but I am working with Ted
> Dunning on some PCAP plugin stuff, so I built his 1.13 SNAPSHOT, and when I
> try to login I see
> {
> "errorMessage" : "No configuration setting found for key
> 'drill.exec.http.auth'"
> }
>
> With DRILL-5425 SPNEGO supports was provided for Drill WebServer. With this
> check-in a backward compatibility change is missing in
> [MainLoginPageModel|https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/LogInLogOutResources.java#L151],
> where in absence of drill.exec.http.auth.mechanisms property it errors out.
> It should default to Form authentication in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6088) MainLoginPageModel errors out when http.auth.mechanisms is not configured
[
https://issues.apache.org/jira/browse/DRILL-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6088:
Fix Version/s: 1.13.0
> MainLoginPageModel errors out when http.auth.mechanisms is not configured
> -
>
> Key: DRILL-6088
> URL: https://issues.apache.org/jira/browse/DRILL-6088
> Project: Apache Drill
> Issue Type: Bug
> Components: Web Server
>Affects Versions: 1.13.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>Priority: Major
> Fix For: 1.13.0
>
>
> Reported by [~mandoskippy]:
> I am probably missing something minor here, but I am working with Ted
> Dunning on some PCAP plugin stuff, so I built his 1.13 SNAPSHOT, and when I
> try to login I see
> {
> "errorMessage" : "No configuration setting found for key
> 'drill.exec.http.auth'"
> }
>
> With DRILL-5425 SPNEGO supports was provided for Drill WebServer. With this
> check-in a backward compatibility change is missing in
> [MainLoginPageModel|https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/LogInLogOutResources.java#L151],
> where in absence of drill.exec.http.auth.mechanisms property it errors out.
> It should default to Form authentication in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330451#comment-16330451
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1084#discussion_r162329512
--- Diff: exec/java-exec/src/main/resources/rest/profile/profile.ftl ---
@@ -77,16 +84,17 @@ table.sortable thead .sorting_desc { background-image:
url("/static/img/black-de
<#if model.isOnlyImpersonationEnabled()>
+ ${model.getProfile().query}
+
User Name
-
-${model.getProfile().query}
-
+ ${model.getProfile().query}
+
--- End diff --
It looks like you have the same code on lines 87-88. These code lines are
included when only impersonation is enabled. So if I am not mistaken, you'll
end up with two query editors when only impersonation is enabled. Please check.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6085) Travis build sometimes fails becomes vm suddenly exits.
[
https://issues.apache.org/jira/browse/DRILL-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6085:
Labels: ready-to-commit (was: )
> Travis build sometimes fails becomes vm suddenly exits.
> ---
>
> Key: DRILL-6085
> URL: https://issues.apache.org/jira/browse/DRILL-6085
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Travis fails with the following error.
> {code}
> Failed to execute goal
> [32morg.apache.maven.plugins:maven-surefire-plugin:2.17:test[m
> [1m(default-test)[m on project [36mdrill-java-exec[m: [1;31mExecution
> default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test
> failed: The forked VM terminated without properly saying goodbye. VM crash or
> System.exit called?[m
> {code}
> I believe this error is caused by the vm running out of memory. I will try
> bumping up the memory slightly and doing several runs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6085) Travis build sometimes fails becomes vm suddenly exits.
[
https://issues.apache.org/jira/browse/DRILL-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330441#comment-16330441
]
ASF GitHub Bot commented on DRILL-6085:
---
Github user arina-ielchiieva commented on the issue:
https://github.com/apache/drill/pull/1095
+1
> Travis build sometimes fails becomes vm suddenly exits.
> ---
>
> Key: DRILL-6085
> URL: https://issues.apache.org/jira/browse/DRILL-6085
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
> Fix For: 1.13.0
>
>
> Travis fails with the following error.
> {code}
> Failed to execute goal
> [32morg.apache.maven.plugins:maven-surefire-plugin:2.17:test[m
> [1m(default-test)[m on project [36mdrill-java-exec[m: [1;31mExecution
> default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test
> failed: The forked VM terminated without properly saying goodbye. VM crash or
> System.exit called?[m
> {code}
> I believe this error is caused by the vm running out of memory. I will try
> bumping up the memory slightly and doing several runs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6094) Decimal data type enhancements
[ https://issues.apache.org/jira/browse/DRILL-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6094: Affects Version/s: 1.12.0 > Decimal data type enhancements > -- > > Key: DRILL-6094 > URL: https://issues.apache.org/jira/browse/DRILL-6094 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Volodymyr Vysotskyi >Assignee: Volodymyr Vysotskyi >Priority: Major > > Currently, Decimal types are disabled by default since existing Decimal > implementation has a lot of flaws and performance problems. The goal of this > Jira to describe majority of them and possible ways of improving existing > implementation to be able to enable Decimal data types by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6085) Travis build sometimes fails becomes vm suddenly exits.
[
https://issues.apache.org/jira/browse/DRILL-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6085:
Fix Version/s: 1.13.0
> Travis build sometimes fails becomes vm suddenly exits.
> ---
>
> Key: DRILL-6085
> URL: https://issues.apache.org/jira/browse/DRILL-6085
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
> Fix For: 1.13.0
>
>
> Travis fails with the following error.
> {code}
> Failed to execute goal
> [32morg.apache.maven.plugins:maven-surefire-plugin:2.17:test[m
> [1m(default-test)[m on project [36mdrill-java-exec[m: [1;31mExecution
> default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test
> failed: The forked VM terminated without properly saying goodbye. VM crash or
> System.exit called?[m
> {code}
> I believe this error is caused by the vm running out of memory. I will try
> bumping up the memory slightly and doing several runs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6085) Travis build sometimes fails becomes vm suddenly exits.
[
https://issues.apache.org/jira/browse/DRILL-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6085:
Affects Version/s: 1.13.0
> Travis build sometimes fails becomes vm suddenly exits.
> ---
>
> Key: DRILL-6085
> URL: https://issues.apache.org/jira/browse/DRILL-6085
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Major
> Fix For: 1.13.0
>
>
> Travis fails with the following error.
> {code}
> Failed to execute goal
> [32morg.apache.maven.plugins:maven-surefire-plugin:2.17:test[m
> [1m(default-test)[m on project [36mdrill-java-exec[m: [1;31mExecution
> default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test
> failed: The forked VM terminated without properly saying goodbye. VM crash or
> System.exit called?[m
> {code}
> I believe this error is caused by the vm running out of memory. I will try
> bumping up the memory slightly and doing several runs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6094) Decimal data type enhancements
Volodymyr Vysotskyi created DRILL-6094: -- Summary: Decimal data type enhancements Key: DRILL-6094 URL: https://issues.apache.org/jira/browse/DRILL-6094 Project: Apache Drill Issue Type: Improvement Reporter: Volodymyr Vysotskyi Assignee: Volodymyr Vysotskyi Currently, Decimal types are disabled by default since existing Decimal implementation has a lot of flaws and performance problems. The goal of this Jira to describe majority of them and possible ways of improving existing implementation to be able to enable Decimal data types by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[
https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330386#comment-16330386
]
ASF GitHub Bot commented on DRILL-6090:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1094#discussion_r162311140
--- Diff:
exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillConnectionImpl.java ---
@@ -132,10 +132,12 @@ protected DrillConnectionImpl(DriverImpl driver,
AvaticaFactory factory,
bit = new Drillbit(dConfig, serviceSet);
bit.run();
} catch (final UserException e) {
+cleanup();
throw new SQLException(
"Failure in starting embedded Drillbit: " + e.getMessage(),
e);
} catch (Exception e) {
+cleanup();
--- End diff --
What if you use flag `doCleanup` which will be true by default and after
`this.client.connect(connect, info);` succeedes, it would be set to false. In
finally block you'll do clean up only if flag is true?
> While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of
> "Curator-Framework-0" threads are created if connection to drill-bit is not
> successful(no drill-bits are up/reachable)
> ---
>
> Key: DRILL-6090
> URL: https://issues.apache.org/jira/browse/DRILL-6090
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.12.0
> Environment: Centos 65, Java 7, Drill JDBC 1.12.0
>Reporter: Milind Takawale
>Priority: Major
> Fix For: 1.13.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the
> available drill-bits using Zookeepers. When drill-bits are not up or not
> reachable, the connection is failed with exception: "Failure in connecting to
> Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for
> client", which is expected, but number of threads created by
> ZKClusterCoordinator just keeps on increasing.
> Steps to reproduce the issue
> # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0
> through Zookeeper hosts(port 5181)
> # Now stop the drill-bit services or block the drill-bit IPs using iptable
> rules
> # Truncate catalina logs
> # Try to connect to the drill-bit/hit a code path that requires connection
> to drill-bits.
> # Take thread dump using kill -QUIT
> # grep -c "Curator-Framework-0" catalina.out
> Observe that the curator framework thread just keep on accumulating
> RCA:
> # ZKClusterCoordinator creates curator threads in the constructor
> # ZKClusterCoordinator is instantiated by DrillClient.connect
> # DrillClient.connect is called in DrillConnectionImpl constructor
> Fix:
> Call DrillConnectionImpl .cleanup() from all the catch blocks in the
> DrillConnectionImpl constructor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-3993) Rebase Drill on Calcite master branch
[ https://issues.apache.org/jira/browse/DRILL-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330379#comment-16330379 ] ASF GitHub Bot commented on DRILL-3993: --- Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/1066 @amansinha100, regarding HashAggregate OOM related change, it was done in the scope of this pull request since with new Calcite a physical plan for the query was changed to the correct one but it caused an infinite loop in HashAgg operator. Therefore I made these changes in order to prevent this infinite loop for the queries that previously worked. I still think that it is better to keep this change in this PR. > Rebase Drill on Calcite master branch > - > > Key: DRILL-3993 > URL: https://issues.apache.org/jira/browse/DRILL-3993 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.2.0 >Reporter: Sudheesh Katkam >Assignee: Roman Kulyk >Priority: Major > > Calcite keeps moving, and now we need to catch up to Calcite 1.5, and ensure > there are no regressions. > Also, how do we resolve this 'catching up' issue in the long term? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330322#comment-16330322
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162294377
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/JoinTestBase.java
---
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.physical.impl.join;
+
+import org.apache.drill.categories.OperatorTest;
+import org.apache.drill.PlanTestBase;
+import org.apache.drill.common.exceptions.UserRemoteException;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.nio.file.Paths;
+import org.apache.drill.exec.planner.physical.PlannerSettings;
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.junit.Assert.assertThat;
+
+
+@Category(OperatorTest.class)
+public class JoinTestBase extends PlanTestBase {
+
+ private static final String testEmptyJoin = "select count(*) as cnt from
cp.`employee.json` emp %s join dfs.`dept.json` " +
+ "as dept on dept.manager = emp.`last_name`";
+
+ /**
+ * This function runs a join query with one of the table generated as an
--- End diff --
function -> method
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330325#comment-16330325
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162293231
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
---
@@ -535,4 +541,8 @@ public void close() {
}
super.close();
}
+
+ private boolean isFurtherProcessingRequired(IterOutcome upStream) {
--- End diff --
It will be good to have here a small java-doc
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330324#comment-16330324
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162289830
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/JoinTestBase.java
---
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.physical.impl.join;
+
+import org.apache.drill.categories.OperatorTest;
+import org.apache.drill.PlanTestBase;
+import org.apache.drill.common.exceptions.UserRemoteException;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.nio.file.Paths;
+import org.apache.drill.exec.planner.physical.PlannerSettings;
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.junit.Assert.assertThat;
+
+
+@Category(OperatorTest.class)
+public class JoinTestBase extends PlanTestBase {
+
+ private static final String testEmptyJoin = "select count(*) as cnt from
cp.`employee.json` emp %s join dfs.`dept.json` " +
+ "as dept on dept.manager = emp.`last_name`";
+
+ /**
+ * This function runs a join query with one of the table generated as an
+ * empty json file.
+ * @param testDir in which the empty json file is generated.
+ * @param joinType to be executed.
+ * @param joinPattern to look for the pattern in the successful run.
+ * @param result number of the output rows.
+ */
+ public void testJoinWithEmptyFile(File testDir, String joinType,
+ String joinPattern, long result) throws Exception
{
+buildFile("dept.json", new String[0], testDir);
+String query = String.format(testEmptyJoin, joinType);
+testPlanMatchingPatterns(query, new String[]{joinPattern}, new
String[]{});
+testBuilder()
+.sqlQuery(query)
+.unOrdered()
+.baselineColumns("cnt")
+.baselineValues(result)
+.build().run();
+ }
+
+ private void buildFile(String fileName, String[] data, File testDir)
throws IOException {
+try(PrintWriter out = new PrintWriter(new FileWriter(new File(testDir,
fileName {
+ for (String line : data) {
+out.println(line);
+ }
+}
+ }
+}
--- End diff --
Put a new line in the end of file
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330321#comment-16330321
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162292571
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestNestedLoopJoin.java
---
@@ -19,19 +19,17 @@
package org.apache.drill.exec.physical.impl.join;
import org.apache.drill.categories.OperatorTest;
-import org.apache.drill.PlanTestBase;
import org.apache.drill.common.exceptions.UserRemoteException;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-
import java.nio.file.Paths;
-
+import org.apache.drill.exec.planner.physical.PlannerSettings;
import static org.hamcrest.CoreMatchers.containsString;
import static org.junit.Assert.assertThat;
@Category(OperatorTest.class)
-public class TestNestedLoopJoin extends PlanTestBase {
+public class TestNestedLoopJoin extends JoinTestBase {
private static String nlpattern = "NestedLoopJoin";
--- End diff --
Can you edit this as well? --> final, NLJ_PATTERN
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330320#comment-16330320
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162292685
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoinAdvanced.java
---
@@ -19,20 +19,22 @@
package org.apache.drill.exec.physical.impl.join;
-import org.apache.drill.test.BaseTestQuery;
import org.apache.drill.categories.OperatorTest;
import org.apache.drill.categories.UnlikelyTest;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
+
@Category(OperatorTest.class)
-public class TestHashJoinAdvanced extends BaseTestQuery {
+public class TestHashJoinAdvanced extends JoinTestBase {
+
+ private static String hjpattern = "HashJoin";
--- End diff --
--> final, HJ_PATTERN
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330326#comment-16330326
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162292084
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoinAdvanced.java
---
@@ -38,13 +37,16 @@
import java.util.Random;
@Category(OperatorTest.class)
-public class TestMergeJoinAdvanced extends BaseTestQuery {
+public class TestMergeJoinAdvanced extends JoinTestBase {
private static final String LEFT = "merge-join-left.json";
private static final String RIGHT = "merge-join-right.json";
+ private static String mjpattern = "MergeJoin";
--- End diff --
Make it as a constant --> private static final String MJ_PATTERN =
"MergeJoin";
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330323#comment-16330323
]
ASF GitHub Bot commented on DRILL-5851:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1059#discussion_r162293578
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
---
@@ -228,4 +228,20 @@ public WritableBatch getWritableBatch() {
public VectorContainer getOutgoingContainer() {
throw new UnsupportedOperationException(String.format(" You should not
call getOutgoingContainer() for class %s", this.getClass().getCanonicalName()));
}
+
+ public void drainStream(IterOutcome stream, int input, RecordBatch
batch) {
--- End diff --
Please add java-doc for this method
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-5851:
Fix Version/s: 1.13.0
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[
https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330289#comment-16330289
]
ASF GitHub Bot commented on DRILL-6090:
---
Github user milindt commented on a diff in the pull request:
https://github.com/apache/drill/pull/1094#discussion_r162286759
--- Diff:
exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillConnectionImpl.java ---
@@ -132,10 +132,12 @@ protected DrillConnectionImpl(DriverImpl driver,
AvaticaFactory factory,
bit = new Drillbit(dConfig, serviceSet);
bit.run();
} catch (final UserException e) {
+cleanup();
throw new SQLException(
"Failure in starting embedded Drillbit: " + e.getMessage(),
e);
} catch (Exception e) {
+cleanup();
--- End diff --
We don't want clean up to run on successful connection though, we just want
to cover all the exceptions. On a successful connection, the "Curator" threads
will be created but the number of threads will be proportional to number of
connections, which can be be controlled by using a connection pool, hence it
won't cause this issue.
On the other hand, calling cleanup on a successful connection may
terminate that connection. Thoughts?
> While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of
> "Curator-Framework-0" threads are created if connection to drill-bit is not
> successful(no drill-bits are up/reachable)
> ---
>
> Key: DRILL-6090
> URL: https://issues.apache.org/jira/browse/DRILL-6090
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.12.0
> Environment: Centos 65, Java 7, Drill JDBC 1.12.0
>Reporter: Milind Takawale
>Priority: Major
> Fix For: 1.13.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the
> available drill-bits using Zookeepers. When drill-bits are not up or not
> reachable, the connection is failed with exception: "Failure in connecting to
> Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for
> client", which is expected, but number of threads created by
> ZKClusterCoordinator just keeps on increasing.
> Steps to reproduce the issue
> # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0
> through Zookeeper hosts(port 5181)
> # Now stop the drill-bit services or block the drill-bit IPs using iptable
> rules
> # Truncate catalina logs
> # Try to connect to the drill-bit/hit a code path that requires connection
> to drill-bits.
> # Take thread dump using kill -QUIT
> # grep -c "Curator-Framework-0" catalina.out
> Observe that the curator framework thread just keep on accumulating
> RCA:
> # ZKClusterCoordinator creates curator threads in the constructor
> # ZKClusterCoordinator is instantiated by DrillClient.connect
> # DrillClient.connect is called in DrillConnectionImpl constructor
> Fix:
> Call DrillConnectionImpl .cleanup() from all the catch blocks in the
> DrillConnectionImpl constructor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6093) Unneeded columns in Drill logical project
[ https://issues.apache.org/jira/browse/DRILL-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautam Kumar Parai updated DRILL-6093: -- Labels: ready-to-commit (was: ) > Unneeded columns in Drill logical project > - > > Key: DRILL-6093 > URL: https://issues.apache.org/jira/browse/DRILL-6093 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0, 1.12.0 >Reporter: Gautam Kumar Parai >Assignee: Gautam Kumar Parai >Priority: Major > Labels: ready-to-commit > Fix For: 1.13.0 > > > Here is an example query with the corresponding logical plan. The project > contains unnecessary columns L_ORDERKEY, O_ORDERKEY in the projection even > when it is not required by subsequent operators e.g. DrillJoinRel. > EXPLAIN PLAN without implementation FOR SELECT L.L_QUANTITY FROM > cp.`tpch/lineitem.parquet` L, cp.`tpch/orders.parquet` O WHERE > cast(L.L_ORDERKEY as int) = cast(O.O_ORDERKEY as int); > *+--+--+* > *|* *text* *|* *json* *|* > *+--+--+* > *|* DrillScreenRel > DrillProjectRel(L_QUANTITY=[$1]) > DrillJoinRel(condition=[=($2, $4)], joinType=[inner]) > DrillProjectRel(L_ORDERKEY=[$0], L_QUANTITY=[$1], > $f2=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/lineitem.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/lineitem.parquet]], > selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`L_ORDERKEY`, `L_QUANTITY`]]]) > DrillProjectRel(O_ORDERKEY=[$0], $f1=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`O_ORDERKEY`]]]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330275#comment-16330275 ] ASF GitHub Bot commented on DRILL-6090: --- Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1094 @milindt release 1.12.0 is out. Regarding this fix, once you address code review comments it will be added in 1.13.0. > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[
https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330273#comment-16330273
]
ASF GitHub Bot commented on DRILL-6090:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1094#discussion_r162282830
--- Diff:
exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillConnectionImpl.java ---
@@ -132,10 +132,12 @@ protected DrillConnectionImpl(DriverImpl driver,
AvaticaFactory factory,
bit = new Drillbit(dConfig, serviceSet);
bit.run();
} catch (final UserException e) {
+cleanup();
throw new SQLException(
"Failure in starting embedded Drillbit: " + e.getMessage(),
e);
} catch (Exception e) {
+cleanup();
--- End diff --
I suppose you can use finally block to perform clean up instead of
repeating `cleanup` method call after each cached exception.
> While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of
> "Curator-Framework-0" threads are created if connection to drill-bit is not
> successful(no drill-bits are up/reachable)
> ---
>
> Key: DRILL-6090
> URL: https://issues.apache.org/jira/browse/DRILL-6090
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.12.0
> Environment: Centos 65, Java 7, Drill JDBC 1.12.0
>Reporter: Milind Takawale
>Priority: Major
> Fix For: 1.13.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the
> available drill-bits using Zookeepers. When drill-bits are not up or not
> reachable, the connection is failed with exception: "Failure in connecting to
> Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for
> client", which is expected, but number of threads created by
> ZKClusterCoordinator just keeps on increasing.
> Steps to reproduce the issue
> # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0
> through Zookeeper hosts(port 5181)
> # Now stop the drill-bit services or block the drill-bit IPs using iptable
> rules
> # Truncate catalina logs
> # Try to connect to the drill-bit/hit a code path that requires connection
> to drill-bits.
> # Take thread dump using kill -QUIT
> # grep -c "Curator-Framework-0" catalina.out
> Observe that the curator framework thread just keep on accumulating
> RCA:
> # ZKClusterCoordinator creates curator threads in the constructor
> # ZKClusterCoordinator is instantiated by DrillClient.connect
> # DrillClient.connect is called in DrillConnectionImpl constructor
> Fix:
> Call DrillConnectionImpl .cleanup() from all the catch blocks in the
> DrillConnectionImpl constructor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dr
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6090: --- Assignee: (was: Arina Ielchiieva) > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330268#comment-16330268 ] ASF GitHub Bot commented on DRILL-6090: --- Github user milindt commented on the issue: https://github.com/apache/drill/pull/1094 @paul-rogers can you please review this fix? > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dri
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6090: Reviewer: Arina Ielchiieva > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dri
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6090: Labels: (was: patch) > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Assignee: Arina Ielchiieva >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dri
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6090: Fix Version/s: (was: 1.12.0) 1.13.0 > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no dr
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6090: --- Assignee: Arina Ielchiieva > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Assignee: Arina Ielchiieva >Priority: Major > Fix For: 1.13.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6090) While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no d
[ https://issues.apache.org/jira/browse/DRILL-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330263#comment-16330263 ] ASF GitHub Bot commented on DRILL-6090: --- Github user milindt commented on the issue: https://github.com/apache/drill/pull/1094 Details in [DRILL-6090](https://issues.apache.org/jira/browse/DRILL-6090) > While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of > "Curator-Framework-0" threads are created if connection to drill-bit is not > successful(no drill-bits are up/reachable) > --- > > Key: DRILL-6090 > URL: https://issues.apache.org/jira/browse/DRILL-6090 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.12.0 > Environment: Centos 65, Java 7, Drill JDBC 1.12.0 >Reporter: Milind Takawale >Priority: Major > Labels: patch > Fix For: 1.12.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the > available drill-bits using Zookeepers. When drill-bits are not up or not > reachable, the connection is failed with exception: "Failure in connecting to > Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for > client", which is expected, but number of threads created by > ZKClusterCoordinator just keeps on increasing. > Steps to reproduce the issue > # Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 > through Zookeeper hosts(port 5181) > # Now stop the drill-bit services or block the drill-bit IPs using iptable > rules > # Truncate catalina logs > # Try to connect to the drill-bit/hit a code path that requires connection > to drill-bits. > # Take thread dump using kill -QUIT > # grep -c "Curator-Framework-0" catalina.out > Observe that the curator framework thread just keep on accumulating > RCA: > # ZKClusterCoordinator creates curator threads in the constructor > # ZKClusterCoordinator is instantiated by DrillClient.connect > # DrillClient.connect is called in DrillConnectionImpl constructor > Fix: > Call DrillConnectionImpl .cleanup() from all the catch blocks in the > DrillConnectionImpl constructor. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6078: --- Assignee: Chunhui Shi (was: Arina Ielchiieva) > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi >Priority: Major > Labels: ready-to-commit > Fix For: 1.13.0 > > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6078: --- Assignee: Arina Ielchiieva (was: Chunhui Shi) > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Arina Ielchiieva >Priority: Major > Labels: ready-to-commit > Fix For: 1.13.0 > > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6093) Unneeded columns in Drill logical project
[ https://issues.apache.org/jira/browse/DRILL-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6093: Reviewer: Aman Sinha Fix Version/s: (was: 1.12.0) 1.13.0 > Unneeded columns in Drill logical project > - > > Key: DRILL-6093 > URL: https://issues.apache.org/jira/browse/DRILL-6093 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0, 1.12.0 >Reporter: Gautam Kumar Parai >Assignee: Gautam Kumar Parai >Priority: Major > Fix For: 1.13.0 > > > Here is an example query with the corresponding logical plan. The project > contains unnecessary columns L_ORDERKEY, O_ORDERKEY in the projection even > when it is not required by subsequent operators e.g. DrillJoinRel. > EXPLAIN PLAN without implementation FOR SELECT L.L_QUANTITY FROM > cp.`tpch/lineitem.parquet` L, cp.`tpch/orders.parquet` O WHERE > cast(L.L_ORDERKEY as int) = cast(O.O_ORDERKEY as int); > *+--+--+* > *|* *text* *|* *json* *|* > *+--+--+* > *|* DrillScreenRel > DrillProjectRel(L_QUANTITY=[$1]) > DrillJoinRel(condition=[=($2, $4)], joinType=[inner]) > DrillProjectRel(L_ORDERKEY=[$0], L_QUANTITY=[$1], > $f2=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/lineitem.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/lineitem.parquet]], > selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`L_ORDERKEY`, `L_QUANTITY`]]]) > DrillProjectRel(O_ORDERKEY=[$0], $f1=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`O_ORDERKEY`]]]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6078: Reviewer: Gautam Kumar Parai Fix Version/s: 1.13.0 > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi >Priority: Major > Labels: ready-to-commit > Fix For: 1.13.0 > > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6078: Labels: ready-to-commit (was: ) > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi >Priority: Major > Labels: ready-to-commit > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6093) Unneeded columns in Drill logical project
[ https://issues.apache.org/jira/browse/DRILL-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330219#comment-16330219 ] ASF GitHub Bot commented on DRILL-6093: --- Github user gparai commented on the issue: https://github.com/apache/drill/pull/1093 @amansinha100 I have addressed your comments. > Unneeded columns in Drill logical project > - > > Key: DRILL-6093 > URL: https://issues.apache.org/jira/browse/DRILL-6093 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0, 1.12.0 >Reporter: Gautam Kumar Parai >Assignee: Gautam Kumar Parai >Priority: Major > Fix For: 1.12.0 > > > Here is an example query with the corresponding logical plan. The project > contains unnecessary columns L_ORDERKEY, O_ORDERKEY in the projection even > when it is not required by subsequent operators e.g. DrillJoinRel. > EXPLAIN PLAN without implementation FOR SELECT L.L_QUANTITY FROM > cp.`tpch/lineitem.parquet` L, cp.`tpch/orders.parquet` O WHERE > cast(L.L_ORDERKEY as int) = cast(O.O_ORDERKEY as int); > *+--+--+* > *|* *text* *|* *json* *|* > *+--+--+* > *|* DrillScreenRel > DrillProjectRel(L_QUANTITY=[$1]) > DrillJoinRel(condition=[=($2, $4)], joinType=[inner]) > DrillProjectRel(L_ORDERKEY=[$0], L_QUANTITY=[$1], > $f2=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/lineitem.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/lineitem.parquet]], > selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`L_ORDERKEY`, `L_QUANTITY`]]]) > DrillProjectRel(O_ORDERKEY=[$0], $f1=[CAST($0):INTEGER]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`O_ORDERKEY`]]]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
