[GitHub] drill pull request #684: DRILL-5070: Code gen: create methods in fixed order...

[paul-rogers]

Github user paul-rogers closed the pull request at:

https://github.com/apache/drill/pull/684


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #684: DRILL-5070: Code gen: create methods in fixed order to all...

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/684
  
Closing pull request since we don't want to have golden copies of generated 
code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #665: DRILL-5056: UserException does not write full message to l...

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/665
  
Used a generated serial id for UserException (even though it is never 
serialized.) Remove the code cleanup. Changed code to use a string builder.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #685: Drill 5043: Function that returns a unique id per s...

[nagarajanchinnasamy]

Github user nagarajanchinnasamy commented on a diff in the pull request:

https://github.com/apache/drill/pull/685#discussion_r91826699
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java ---
@@ -115,6 +118,7 @@ public UserSession build() {
 
   private UserSession() {
 queryCount = new AtomicInteger(0);
+sessionId = nextSessionId.getAndIncrement();
   }
--- End diff --

Thanks Arina. Just realized you are the author of #666 :) Could you please 
tell me how I can merge your changes into my branch. Am new to this process. 
Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91823124
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java ---
@@ -357,10 +357,54 @@ protected void afterExecute(final Runnable r, final 
Throwable t) {
 super.afterExecute(r, t);
   }
 };
-client = new UserClient(clientName, config, supportComplexTypes, 
allocator, eventLoopGroup, executor);
-logger.debug("Connecting to server {}:{}", endpoint.getAddress(), 
endpoint.getUserPort());
-connect(endpoint);
-connected = true;
+
+// "tries" is max number of unique drillbit to try connecting until 
successfully connected to one of them
+final String connectTriesConf = (props != null) ? 
props.getProperty("tries", "5") : "5";
+
+int connectTriesVal;
+try {
+  connectTriesVal = Math.min(endpoints.size(), 
Integer.parseInt(connectTriesConf));
+} catch (NumberFormatException e) {
+  throw new InvalidConnectionInfoException("Invalid tries value: " + 
connectTriesConf + " specified in " +
+   "connection string");
+}
+
+// If the value provided in the connection string is <=0 then override 
with 1 since we want to try connecting
+// at least once
+connectTriesVal = Math.max(1, connectTriesVal);
+
+int triedEndpointIndex = 0;
+DrillbitEndpoint endpoint;
+
+while (triedEndpointIndex < connectTriesVal) {
+  client = new UserClient(clientName, config, supportComplexTypes, 
allocator, eventLoopGroup, executor);
+  endpoint = endpoints.get(triedEndpointIndex);
+  logger.debug("Connecting to server {}:{}", endpoint.getAddress(), 
endpoint.getUserPort());
+
+  try {
+connect(endpoint);
+connected = true;
+logger.info("Successfully connected to server {}:{}", 
endpoint.getAddress(), endpoint.getUserPort());
+break;
+  } catch (InvalidConnectionInfoException ex) {
+logger.error("Connection to {}:{} failed with error {}. Not 
retrying anymore", endpoint.getAddress(),
+ endpoint.getUserPort(), ex.getMessage());
+throw ex;
+  } catch (RpcException ex) {
+++triedEndpointIndex;
+logger.error("Attempt {}: Failed to connect to server {}:{}", 
triedEndpointIndex, endpoint.getAddress(),
+ endpoint.getUserPort());
+
+// Close the connection
+if (client.isActive()) {
--- End diff --

The loop recreates a new client anyway. So this should be closed, 
regardless of being active.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sohami]

Github user sohami commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91821466
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/client/ConnectTriesPropertyTestClusterBits.java
 ---
@@ -0,0 +1,244 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.client;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Properties;
+import java.util.concurrent.ExecutionException;
+
+import org.apache.drill.common.config.DrillConfig;
+import org.apache.drill.exec.ZookeeperHelper;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.DrillbitStartupException;
+import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
+import org.apache.drill.exec.rpc.InvalidConnectionInfoException;
+import org.apache.drill.exec.rpc.RpcException;
+import org.apache.drill.exec.server.Drillbit;
+
+import org.apache.drill.exec.server.RemoteServiceSet;
+
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static junit.framework.TestCase.assertTrue;
+import static junit.framework.TestCase.fail;
+
+public class ConnectTriesPropertyTestClusterBits {
+
+  public static StringBuilder bitInfo;
+  public static final String fakeBitsInfo = 
"127.0.0.1:5000,127.0.0.1:5001";
+  public static List drillbits;
+  public static final int drillBitCount = 1;
+  public static ZookeeperHelper zkHelper;
+  public static RemoteServiceSet remoteServiceSet;
+  public static DrillConfig drillConfig;
+
+  @BeforeClass
+  public static void testSetUp() throws Exception {
+remoteServiceSet = RemoteServiceSet.getLocalServiceSet();
+zkHelper = new ZookeeperHelper();
+zkHelper.startZookeeper(1);
+
+// Creating Drillbits
+drillConfig = zkHelper.getConfig();
+try {
+  int drillBitStarted = 0;
+  drillbits = new ArrayList<>();
+  while(drillBitStarted < drillBitCount){
+drillbits.add(Drillbit.start(drillConfig, remoteServiceSet));
+++drillBitStarted;
+  }
+} catch (DrillbitStartupException e) {
+  throw new RuntimeException("Failed to start drillbits.", e);
+}
+bitInfo = new StringBuilder();
+
+for (int i = 0; i < drillBitCount; ++i) {
+  final DrillbitEndpoint currentEndPoint = 
drillbits.get(i).getContext().getEndpoint();
+  final String currentBitIp = currentEndPoint.getAddress();
+  final int currentBitPort = currentEndPoint.getUserPort();
+  bitInfo.append(",");
+  bitInfo.append(currentBitIp);
+  bitInfo.append(":");
+  bitInfo.append(currentBitPort);
+}
+  }
+
+  @AfterClass
+  public static void testCleanUp(){
+for(int i=0; i < drillBitCount; ++i){
+  drillbits.get(i).close();
--- End diff --

Fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sohami]

Github user sohami commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91822093
  
--- Diff: 
exec/jdbc/src/test/java/org/apache/drill/jdbc/test/JdbcConnectTriesTestEmbeddedBits.java
 ---
@@ -0,0 +1,162 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.jdbc.test;
+
+import org.apache.drill.exec.rpc.InvalidConnectionInfoException;
+import org.apache.drill.exec.rpc.RpcException;
+import org.apache.drill.jdbc.Driver;
+import org.apache.drill.jdbc.JdbcTestBase;
+
+import org.junit.Test;
+
+import java.sql.SQLException;
+import java.sql.Connection;
+
+import java.util.concurrent.ExecutionException;
+
+import static junit.framework.Assert.assertNotNull;
+import static junit.framework.TestCase.fail;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+public class JdbcConnectTriesTestEmbeddedBits extends JdbcTestBase {
+
+  @Test
+  public void testDirectConnectionConnectTriesEqualsDrillbitCount() throws 
SQLException {
+Connection connection = null;
+try {
+  connection = new 
Driver().connect("jdbc:drill:drillbit=127.0.0.1:5000,127.0.0.1:5001;" + 
"tries=2",
--- End diff --

Fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #665: DRILL-5056: UserException does not write full messa...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/665#discussion_r91820335
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
 ---
@@ -224,6 +224,7 @@ public void run() {
   ImpersonationUtil.getProcessUserUGI();
 
   queryUserUgi.doAs(new PrivilegedExceptionAction() {
+@Override
--- End diff --

Please undo these overrides. Although useful, these are unrelated to the 
issue. Maybe another ticket?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #681: DRILL-5112: Unit tests derived from PopUnitTestBase fail i...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/681
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #680: DRILL-5108: Reduce output from Maven git-commit-id-plugin

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/680
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #669: DRILL-5044: After the dynamic registration of multiple jar...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/669
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5121) A memory leak is observed when exact case is not specified for a column in a filter condition

[Karthikeyan Manivannan (JIRA)]

Karthikeyan Manivannan created DRILL-5121:
-

 Summary: A memory leak is observed when exact case is not 
specified for a column in a filter condition
 Key: DRILL-5121
 URL: https://issues.apache.org/jira/browse/DRILL-5121
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.8.0, 1.6.0
Reporter: Karthikeyan Manivannan
Assignee: Karthikeyan Manivannan
 Fix For: Future


When the query SELECT XYZ from dfs.`/tmp/foo' where xYZ like "abc", is executed 
on a setup where /tmp/foo has 2 Parquet files, 1.parquet and 2.parquet, where 
1.parquet has the column XYZ but 2.parquet does not, then there is a memory 
leak. 

This seems to happen because xYZ seem to be treated as a new column. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill issue #676: DRILL-5091: JDBC unit test fail on Java 8

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/676
  
Opened DRILL-5120 for the JDBC Driver update for JDBC 4.2. Added TODOs as 
requested. Squashed commits to simplify commit to master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5120) Upgrade JDBC Driver for new Java 8 methods

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5120:
--

 Summary: Upgrade JDBC Driver for new Java 8 methods
 Key: DRILL-5120
 URL: https://issues.apache.org/jira/browse/DRILL-5120
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.8.0
Reporter: Paul Rogers
Priority: Minor


Java 8 has been released for some time. Included in Java 8 is a new version of 
the JDBC interface: JDBC 4.2. Consult the JDBC spec for details.

The JDBC unit tests were modified to pass with the default JDBC 4.2 
implementations. The "known not implemented" code (marked with TODO) should be 
replaced to test the real implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill issue #688: DRILL-5119 - Update MapR version to 5.2.0.40963-mapr

[adityakishore]

Github user adityakishore commented on the issue:

https://github.com/apache/drill/pull/688
  
LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #688: DRILL-5119 - Update MapR version to 5.2.0.40963-mapr

[spanchamiamapr]

Github user spanchamiamapr commented on the issue:

https://github.com/apache/drill/pull/688
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #680: DRILL-5108: Reduce output from Maven git-commit-id-plugin

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/680
  
The non-verbose output is, indeed, blank. The value of the plugin seems to 
be that it creates Maven variables that can be used elsewhere in the POM for 
various purposes. Dumping to console is an "extra."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #688: DRILL-5119 - Update MapR version to 5.2.0.40963-mapr

[Agirish]

Github user Agirish commented on the issue:

https://github.com/apache/drill/pull/688
  
@spanchamiamapr, @adityakishore can one of you please review this change?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #688: DRILL-5119 - Update MapR version to 5.2.0.40963-mapr

[Agirish]

Github user Agirish commented on the issue:

https://github.com/apache/drill/pull/688
  
+1 (non-binding). Thanks for making the change. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #688: DRILL-5119 - Update MapR version to 5.2.0.40963-map...

[pwong-mapr]

GitHub user pwong-mapr opened a pull request:

https://github.com/apache/drill/pull/688

DRILL-5119 - Update MapR version to 5.2.0.40963-mapr

Change for https://issues.apache.org/jira/browse/DRILL-5119

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pwong-mapr/incubator-drill patch-4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/688.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #688


commit c3f03b186e786dc92ad88ee459c93e581fc52c26
Author: Patrick Wong 
Date:   2016-12-09T22:26:31Z

DRILL-5119 - Update MapR version to 5.2.0.40963-mapr




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #687: DRILL-4996: Parquet Date auto-correction is not wor...

[vdiravka]

GitHub user vdiravka opened a pull request:

https://github.com/apache/drill/pull/687

DRILL-4996: Parquet Date auto-correction is not working in auto-parti…

…tioned parquet files generated by drill-1.6

- Changed detection approach of corrupted date values for the case, when 
parquet files are generated by drill:
  the corruption status is determined by looking at the min/max values in 
the metadata;
- Appropriate refactoring of TestCorruptParquetDateCorrection.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vdiravka/drill DRILL-4996

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/687.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #687


commit 22f4c3dcbe00d47055185d802e88fecaff0d252d
Author: Vitalii Diravka 
Date:   2016-12-09T08:00:48Z

DRILL-4996: Parquet Date auto-correction is not working in auto-partitioned 
parquet files generated by drill-1.6
- Changed detection approach of corrupted date values for the case, when 
parquet files are generated by drill:
  the corruption status is determined by looking at the min/max values in 
the metadata;
- Appropriate refactoring of TestCorruptParquetDateCorrection.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #657: DRILL-5048: AssertionError when case statement is used wit...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/657
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #672: DRILL-5085: Add / update description for dynamic UDFs dire...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/672
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #676: DRILL-5091: JDBC unit test fail on Java 8

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/676
  
+1, pending minor comment


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/676#discussion_r91797180
  
--- Diff: 
exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2489CallsAfterCloseThrowExceptionsTest.java
 ---
@@ -477,18 +468,32 @@ public void testClosedConnectionMethodsThrowRight() {
 }
 
 @Override
+protected boolean isOkayNonthrowingMethod(Method method) {
+  // Java 8 method
+  if ("getLargeUpdateCount".equals(method.getName())) {
+return true; }
+  return super.isOkayNonthrowingMethod(method);
+}
+
+@Override
 protected boolean isOkaySpecialCaseException(Method method, Throwable 
cause) {
   final boolean result;
   if (super.isOkaySpecialCaseException(method, cause)) {
 result = true;
   }
+  else if (   method.getName().equals("executeLargeBatch")
+   || method.getName().equals("executeLargeUpdate")) {
+// New Java 8 methods not implemented in Avatica.
--- End diff --

Please open a ticket for this, so we make changes once Avatica supports 
Java 8, add TODOs instead of comments for easy tracking 
([example](https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/FragmentWrapper.java#L103)).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91768005
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java ---
@@ -357,10 +357,54 @@ protected void afterExecute(final Runnable r, final 
Throwable t) {
 super.afterExecute(r, t);
   }
 };
-client = new UserClient(clientName, config, supportComplexTypes, 
allocator, eventLoopGroup, executor);
-logger.debug("Connecting to server {}:{}", endpoint.getAddress(), 
endpoint.getUserPort());
-connect(endpoint);
-connected = true;
+
+// "tries" is max number of unique drillbit to try connecting until 
successfully connected to one of them
+final String connectTriesConf = (props != null) ? 
props.getProperty("tries", "5") : "5";
+
+int connectTriesVal;
+try {
+  connectTriesVal = Math.min(endpoints.size(), 
Integer.parseInt(connectTriesConf));
+} catch (NumberFormatException e) {
+  throw new InvalidConnectionInfoException("Invalid tries value: " + 
connectTriesConf + " specified in " +
+   "connection string");
+}
+
+// If the value provided in the connection string is <=0 then override 
with 1 since we want to try connecting
+// at least once
+connectTriesVal = Math.max(1, connectTriesVal);
+
+int triedEndpointIndex = 0;
+DrillbitEndpoint endpoint;
+
+while (triedEndpointIndex < connectTriesVal) {
+  client = new UserClient(clientName, config, supportComplexTypes, 
allocator, eventLoopGroup, executor);
+  endpoint = endpoints.get(triedEndpointIndex);
+  logger.debug("Connecting to server {}:{}", endpoint.getAddress(), 
endpoint.getUserPort());
+
+  try {
+connect(endpoint);
+connected = true;
+logger.info("Successfully connected to server {}:{}", 
endpoint.getAddress(), endpoint.getUserPort());
+break;
+  } catch (InvalidConnectionInfoException ex) {
+logger.error("Connection to {}:{} failed with error {}. Not 
retrying anymore", endpoint.getAddress(),
+ endpoint.getUserPort(), ex.getMessage());
+throw ex;
+  } catch (RpcException ex) {
+++triedEndpointIndex;
+logger.error("Attempt {}: Failed to connect to server {}:{}", 
triedEndpointIndex, endpoint.getAddress(),
+ endpoint.getUserPort());
+
+// Close the connection
+if (client.isActive()) {
--- End diff --

Shouldn't the client be closed regardless of being active? There maybe 
other resources to close.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91795818
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/client/ConnectTriesPropertyTestClusterBits.java
 ---
@@ -0,0 +1,244 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.client;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Properties;
+import java.util.concurrent.ExecutionException;
+
+import org.apache.drill.common.config.DrillConfig;
+import org.apache.drill.exec.ZookeeperHelper;
+import org.apache.drill.exec.coord.ClusterCoordinator;
+import org.apache.drill.exec.exception.DrillbitStartupException;
+import org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint;
+import org.apache.drill.exec.rpc.InvalidConnectionInfoException;
+import org.apache.drill.exec.rpc.RpcException;
+import org.apache.drill.exec.server.Drillbit;
+
+import org.apache.drill.exec.server.RemoteServiceSet;
+
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static junit.framework.TestCase.assertTrue;
+import static junit.framework.TestCase.fail;
+
+public class ConnectTriesPropertyTestClusterBits {
+
+  public static StringBuilder bitInfo;
+  public static final String fakeBitsInfo = 
"127.0.0.1:5000,127.0.0.1:5001";
+  public static List drillbits;
+  public static final int drillBitCount = 1;
+  public static ZookeeperHelper zkHelper;
+  public static RemoteServiceSet remoteServiceSet;
+  public static DrillConfig drillConfig;
+
+  @BeforeClass
+  public static void testSetUp() throws Exception {
+remoteServiceSet = RemoteServiceSet.getLocalServiceSet();
+zkHelper = new ZookeeperHelper();
+zkHelper.startZookeeper(1);
+
+// Creating Drillbits
+drillConfig = zkHelper.getConfig();
+try {
+  int drillBitStarted = 0;
+  drillbits = new ArrayList<>();
+  while(drillBitStarted < drillBitCount){
+drillbits.add(Drillbit.start(drillConfig, remoteServiceSet));
+++drillBitStarted;
+  }
+} catch (DrillbitStartupException e) {
+  throw new RuntimeException("Failed to start drillbits.", e);
+}
+bitInfo = new StringBuilder();
+
+for (int i = 0; i < drillBitCount; ++i) {
+  final DrillbitEndpoint currentEndPoint = 
drillbits.get(i).getContext().getEndpoint();
+  final String currentBitIp = currentEndPoint.getAddress();
+  final int currentBitPort = currentEndPoint.getUserPort();
+  bitInfo.append(",");
+  bitInfo.append(currentBitIp);
+  bitInfo.append(":");
+  bitInfo.append(currentBitPort);
+}
+  }
+
+  @AfterClass
+  public static void testCleanUp(){
+for(int i=0; i < drillBitCount; ++i){
+  drillbits.get(i).close();
--- End diff --

+ spacing in the signature
+ Use `AutoCloseables.close(drillbits);`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #679: DRILL-5098: Improving fault tolerance for connectio...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/679#discussion_r91796042
  
--- Diff: 
exec/jdbc/src/test/java/org/apache/drill/jdbc/test/JdbcConnectTriesTestEmbeddedBits.java
 ---
@@ -0,0 +1,162 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.jdbc.test;
+
+import org.apache.drill.exec.rpc.InvalidConnectionInfoException;
+import org.apache.drill.exec.rpc.RpcException;
+import org.apache.drill.jdbc.Driver;
+import org.apache.drill.jdbc.JdbcTestBase;
+
+import org.junit.Test;
+
+import java.sql.SQLException;
+import java.sql.Connection;
+
+import java.util.concurrent.ExecutionException;
+
+import static junit.framework.Assert.assertNotNull;
+import static junit.framework.TestCase.fail;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+public class JdbcConnectTriesTestEmbeddedBits extends JdbcTestBase {
+
+  @Test
+  public void testDirectConnectionConnectTriesEqualsDrillbitCount() throws 
SQLException {
+Connection connection = null;
+try {
+  connection = new 
Driver().connect("jdbc:drill:drillbit=127.0.0.1:5000,127.0.0.1:5001;" + 
"tries=2",
--- End diff --

Create one driver and use across tests.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #680: DRILL-5108: Reduce output from Maven git-commit-id-plugin

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/680
  
What does the non verbose message look like?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #682: DRILL-5113: Upgrade Maven RAT plugin to avoid annoying XML...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/682
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #673: DRILL-4764: Parquet file with INT_16, etc. logical ...

[Serhii-Harnyk]

Github user Serhii-Harnyk commented on a diff in the pull request:

https://github.com/apache/drill/pull/673#discussion_r91753304
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReaders.java
 ---
@@ -56,6 +58,31 @@ protected void readField(long recordsToReadInThisPass) {
 }
--- End diff --

done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #594: DRILL-4842: SELECT * on JSON data results in NumberFormatE...

[Serhii-Harnyk]

Github user Serhii-Harnyk commented on the issue:

https://github.com/apache/drill/pull/594
  
@chunhui-shi, could you please review new changes?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #686: DRILL-5117: Compile error when query a json file wi...

[Serhii-Harnyk]

GitHub user Serhii-Harnyk opened a pull request:

https://github.com/apache/drill/pull/686

DRILL-5117: Compile error when query a json file with 1000+columns



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Serhii-Harnyk/drill DRILL-5117

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/686.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #686


commit 00eaf30fd662530d8bd62059b85b0ad179768fdb
Author: Serhii-Harnyk 
Date:   2016-12-08T20:08:34Z

DRILL-5117: Compile error when query a json file with 1000+columns




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5118) Select Query Limit is not working in Drill 1.8

[Gopal Nagar (JIRA)]

Gopal Nagar created DRILL-5118:
--

 Summary: Select Query Limit is not working in Drill 1.8
 Key: DRILL-5118
 URL: https://issues.apache.org/jira/browse/DRILL-5118
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - CLI, Client - HTTP
Affects Versions: 1.8.0
Reporter: Gopal Nagar


Hi All,

Drill 1.8.0 has been installed on a AWS node which has 32 GB RAM and 80 GB 
storage. I didn't specify memory separately to Drill. I am trying to join two 
tables have rows 4607818 & 14273378 respectively and I have put limit 100 in 
query.

But after displaying the 100 rows on Drill CLI, Query doesn't terminated and it 
doesn't go back to CLI prompt and keep processing the data in Background. 
Please help.

Join Query 
--
select t1.col FROM hive.table1 as t1 join hive.table2 as t2 on t1.col = t2.col 
limit 100;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-5117) Compile error when query a json file with 1000+columns

[Serhii Harnyk (JIRA)]

Serhii Harnyk created DRILL-5117:


 Summary: Compile error when query a json file with 1000+columns
 Key: DRILL-5117
 URL: https://issues.apache.org/jira/browse/DRILL-5117
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Codegen
Affects Versions: 1.8.0
Reporter: Serhii Harnyk
Assignee: Serhii Harnyk
 Fix For: Future


Query failed with compile error when we querying a json file with 1000+columns:
{noformat}
0: jdbc:drill:zk=local> select * from dfs.`/tmp/tooManyFields.json` limit 1;
Error: SYSTEM ERROR: JaninoRuntimeException: Code attribute in class 
"org.apache.drill.exec.test.generated.CopierGen0" grows beyond 64 KB

Fragment 0:0

[Error Id: a1306543-4d66-4bb0-b687-5802002833b2 on user515050-pc:31010] 
(state=,code=0)
{noformat}

Stack trace from sqlline.log:
{noformat}
2016-12-09 13:43:38,207 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
27b54af4-b41f-0682-e50d-626de4eff68e: select * from 
dfs.`/tmp/tooManyFields.json` limit 1
2016-12-09 13:43:38,340 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,340 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,341 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,341 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,341 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,341 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,532 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2016-12-09 13:43:38,547 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host 
localhost.  Skipping affinity to that host.
2016-12-09 13:43:38,548 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using 
1 threads. Time: 13ms total, 13.922965ms avg, 13ms max.
2016-12-09 13:43:38,548 [27b54af4-b41f-0682-e50d-626de4eff68e:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using 
1 threads. Earliest start: 6.956000 μs, Latest start: 6.956000 μs, Average 
start: 6.956000 μs .
2016-12-09 13:43:38,750 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 27b54af4-b41f-0682-e50d-626de4eff68e:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2016-12-09 13:43:38,761 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 27b54af4-b41f-0682-e50d-626de4eff68e:0:0: 
State to report: RUNNING
2016-12-09 13:43:39,375 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] WARN  
o.a.d.exec.compile.JDKClassCompiler - JDK Java compiler not available - 
probably you're running Drill with a JRE and not a JDK
2016-12-09 13:43:40,533 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 27b54af4-b41f-0682-e50d-626de4eff68e:0:0: 
State change requested RUNNING --> FAILED
2016-12-09 13:43:40,550 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 27b54af4-b41f-0682-e50d-626de4eff68e:0:0: 
State change requested FAILED --> FINISHED
2016-12-09 13:43:40,552 [27b54af4-b41f-0682-e50d-626de4eff68e:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: JaninoRuntimeException: 
Code attribute in class "org.apache.drill.exec.test.generated.CopierGen0" grows 
beyond 64 KB

Fragment 0:0

[Error Id: a1306543-4d66-4bb0-b687-5802002833b2 on user515050-pc:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
JaninoRuntimeException: Code attribute in class 
"org.apache.drill.exec.test.generated.CopierGen0" grows beyond 64 KB

Fragment 0:0

[Error Id: a1306543-4d66-4bb0-b687-5802002833b2 on user515050-pc:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
 ~[drill-common-1.8.0.jar:1.8.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)
 [drill-java-exec-1.8.0.jar:1.8.0]
at 

[GitHub] drill issue #685: Drill 5043: Function that returns a unique id per session/...

[nagarajanchinnasamy]

Github user nagarajanchinnasamy commented on the issue:

https://github.com/apache/drill/pull/685
  
@paul-rogers as per my requirement the SessionId needs to be unique. Thats 
all. It does not have to be a monotonically increasing sequence. So, UUID 
should be fine. Will look into #666 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #685: Drill 5043: Function that returns a unique id per s...

[nagarajanchinnasamy]

Github user nagarajanchinnasamy commented on a diff in the pull request:

https://github.com/apache/drill/pull/685#discussion_r91685705
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java ---
@@ -115,6 +118,7 @@ public UserSession build() {
 
   private UserSession() {
 queryCount = new AtomicInteger(0);
+sessionId = nextSessionId.getAndIncrement();
   }
--- End diff --

Yes. This approach doesn't take distribution in consideration. Looking for 
inputs on what should be the approach for distributed environment. As suggested 
by @paul-rogers I will look into #666 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #660: DRILL-5052: Option to debug generated Java code using an I...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/660
  
Looks good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #660: DRILL-5052: Option to debug generated Java code usi...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/660#discussion_r91679324
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/ClassGenerator.java ---
@@ -246,6 +246,12 @@ public void rotateBlock() {
 rotateBlock(BlkCreateMode.TRUE);
   }
 
+  /**
+   * Create a new code block, closing the current block.
+   *
+   * @param mode
+   */
--- End diff --

IntelliJ, it's more strict :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---