[jira] [Created] (DRILL-5868) Support SQL syntax highlighting and formatting in the Edit Query textbox

[Kunal Khatua (JIRA)]

Kunal Khatua created DRILL-5868:
---

 Summary: Support SQL syntax highlighting and formatting in the 
Edit Query textbox 
 Key: DRILL-5868
 URL: https://issues.apache.org/jira/browse/DRILL-5868
 Project: Apache Drill
  Issue Type: New Feature
  Components: Web Server
Reporter: Kunal Khatua
Priority: Minor
 Fix For: Future


It would be nice to have the Query Editor support syntax highlighting.
An autocomplete would be even better as new functions are introduced in Drill



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (DRILL-5867) List profiles in pages rather than a long verbose listing

[Kunal Khatua (JIRA)]

Kunal Khatua created DRILL-5867:
---

 Summary: List profiles in pages rather than a long verbose listing
 Key: DRILL-5867
 URL: https://issues.apache.org/jira/browse/DRILL-5867
 Project: Apache Drill
  Issue Type: Sub-task
Affects Versions: 1.11.0
Reporter: Kunal Khatua
Assignee: Kunal Khatua
Priority: Minor
 Fix For: 1.12.0






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[ilooner]

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144195098
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
 ---
@@ -95,7 +91,9 @@ public TopNBatch(TopN popConfig, FragmentContext context, 
RecordBatch incoming)
 super(popConfig, context);
 this.incoming = incoming;
 this.config = popConfig;
-batchPurgeThreshold = 
context.getConfig().getInt(ExecConstants.BATCH_PURGE_THRESHOLD);
+DrillConfig drillConfig = context.getConfig();
+batchPurgeThreshold = 
drillConfig.getInt(ExecConstants.BATCH_PURGE_THRESHOLD);
+codegenDump = 
drillConfig.getBoolean(CodeCompiler.ENABLE_SAVE_CODE_FOR_DEBUG);
--- End diff --

I will rename the option to drill.debug.codegen.TopN


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[ilooner]

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144194940
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java ---
@@ -110,6 +109,11 @@ public CodeGenCompiler(final DrillConfig config, final 
OptionSet optionManager)
   public static final String DISABLE_CACHE_CONFIG = COMPILE_BASE + 
".disable_cache";
 
   /**
+   * Enables saving generated code for debugging
+   */
+  public static final String ENABLE_SAVE_CODE_FOR_DEBUG = COMPILE_BASE + 
".codegen.dump";
--- End diff --

This is the name of the option which holds the true/false flag to enable 
saving code for debugging. The destination directory is governed by the 
ClassBuilder.CODE_DIR_OPTION property.


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[ilooner]

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144194617
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueue.java
 ---
@@ -20,22 +20,58 @@
 import org.apache.drill.exec.compile.TemplateClassDefinition;
 import org.apache.drill.exec.exception.SchemaChangeException;
 import org.apache.drill.exec.memory.BufferAllocator;
-import org.apache.drill.exec.ops.FragmentContext;
 import org.apache.drill.exec.physical.impl.sort.RecordBatchData;
 import org.apache.drill.exec.record.VectorContainer;
 import org.apache.drill.exec.record.selection.SelectionVector4;
 
 public interface PriorityQueue {
-  public void add(FragmentContext context, RecordBatchData batch) throws 
SchemaChangeException;
-  public void init(int limit, FragmentContext context, BufferAllocator 
allocator, boolean hasSv2) throws SchemaChangeException;
-  public void generate() throws SchemaChangeException;
-  public VectorContainer getHyperBatch();
-  public SelectionVector4 getHeapSv4();
-  public SelectionVector4 getFinalSv4();
-  public boolean validate();
-  public void resetQueue(VectorContainer container, SelectionVector4 
vector4) throws SchemaChangeException;
-  public void cleanup();
-
-  public static TemplateClassDefinition TEMPLATE_DEFINITION 
= new TemplateClassDefinition(PriorityQueue.class, 
PriorityQueueTemplate.class);
+  /**
+   * The elements in the given batch are added to the priority queue. Note 
that the priority queue
+   * only retains the top elements that fit within the size specified by 
the {@link #init(int, BufferAllocator, boolean)}
+   * method.
+   * @param batch The batch containing elements we want to add.
+   * @throws SchemaChangeException
+   */
+  void add(RecordBatchData batch) throws SchemaChangeException;
 
+  /**
+   * Initializes the priority queue. This method must be called before any 
other methods on the priority
+   * queue are called.
+   * @param limit The size of the priority queue.
+   * @param allocator The {@link BufferAllocator} to use when creating the 
priority queue.
+   * @param hasSv2 True when incoming batches have v2 selection vectors. 
False otherwise.
+   * @throws SchemaChangeException
+   */
+  void init(int limit, BufferAllocator allocator, boolean hasSv2) throws 
SchemaChangeException;
+
+  /**
+   * This method must be called before fetching the final heap hyper batch 
and final Sv4 vector.
--- End diff --

Old habit, I'm used to PriorityQueues being called Heaps. I agree it's 
confusing in this context I will fix it.


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[ilooner]

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144194241
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/CodeGenerator.java ---
@@ -90,12 +89,11 @@
   private String generatedCode;
   private String generifiedCode;
 
-  CodeGenerator(TemplateClassDefinition definition, 
FunctionImplementationRegistry funcRegistry, OptionSet optionManager) {
-this(ClassGenerator.getDefaultMapping(), definition, funcRegistry, 
optionManager);
+  CodeGenerator(TemplateClassDefinition definition, OptionSet 
optionManager) {
+this(ClassGenerator.getDefaultMapping(), definition, optionManager);
--- End diff --

I traced through the code and asked IntelliJ where the variable is used. 
This is what I found:

1. CodeGenerator accepts it as an argument to its constructor
1. It is then passed to the constructor of the EvaluationVisitor
1. A private field is set with the value in the constructor of 
EvaluationVisitor
1. This private field of EvaluationVisitor is then unused.

If your not convinced let's walk through the code when I'm in the office 
next week.


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[ilooner]

Github user ilooner commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144193392
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java ---
@@ -110,6 +109,11 @@ public CodeGenCompiler(final DrillConfig config, final 
OptionSet optionManager)
   public static final String DISABLE_CACHE_CONFIG = COMPILE_BASE + 
".disable_cache";
 
   /**
+   * Enables saving generated code for debugging
+   */
+  public static final String ENABLE_SAVE_CODE_FOR_DEBUG = COMPILE_BASE + 
".codegen.dump";
--- End diff --

The directory that generated code is dumped into is determined by the 
ClassBuilder.CODE_DIR_OPTION property. The DirTestWatcher createw a directory 
of the following form:

```
/target//
```

So if the ClassBuilder.CODE_DIR_OPTION property is configured to use the 
directory generated by the DirTestWatcher, the code is dumped into the same 
well defined location every test run. Also the generated class file has names 
like:

```
Gen.java
```


---

[jira] [Created] (DRILL-5866) Methods supportsFullOuterJoins and supportsLimitedOuterJoins of DatabaseMetaData return incorrect values

[second88 (JIRA)]

second88 created DRILL-5866:
---

 Summary: Methods supportsFullOuterJoins and 
supportsLimitedOuterJoins of DatabaseMetaData return incorrect values
 Key: DRILL-5866
 URL: https://issues.apache.org/jira/browse/DRILL-5866
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC, Metadata, Storage - JDBC
Affects Versions: 1.10.0
Reporter: second88
Priority: Minor


Currently, supportsFullOuterJoins() returns true and supportsLimitedOuterJoins 
returns false.
The expected result should be supportsFullOuterJoins() returns false and 
supportsLimitedOuterJoins returns true.

My interpretations to the descriptions of the methods in [JDBC 
API|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html]:

*supportsFullOuterJoins*
If "LEFT OUTER JOIN", "RIGHT OUTER JOIN" and "FULL OUTER JOIN" clauses and 
nested joins are supported, returns true, otherwise, returns false.

*supportsLimitedOuterJoins*
If "LEFT OUTER JOIN", "RIGHT OUTER JOIN" and "FULL OUTER JOIN" clauses, returns 
true, otherwise, returns false.

The [current 
implementation|https://github.com/apache/drill/blob/master/exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillDatabaseMetaDataImpl.java]
 seems to be wrong.
supportsFullOuterJoins() does not check if nested joins are supported.
supportsLimitedOuterJoins() contradicts the description of the method that it 
returns false while supportsFullOuterJoins() returns true.

>From 
>[MySQL|https://dev.mysql.com/doc/refman/5.7/en/nested-join-optimization.html] 
>and [join_clause of 
>Oracle|http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_10002.htm#i2080416],
> I think nested join in general means using parentheses to override default 
>precedence.

An example of nested join

{code:sql}
--Given that the involved schema / default workspace, tables and columns are 
available, this select statment is tested valid in Oracle but not in Drill 
(PARSE ERROR: Non-query expression encountered in illegal context)
select a.id 
from dfs.a a
  left outer join (
dfs.b b
full outer join dfs.c c
on b.id = c.id
  )
  on a.id = b.id
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-5819) Default value of security.admin.user_groups and security.admin.users is "true"

[Paul Rogers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Rogers resolved DRILL-5819.

Resolution: Fixed
  Assignee: Karthikeyan Manivannan
  Reviewer: Paul Rogers

> Default value of security.admin.user_groups and security.admin.users is "true"
> --
>
> Key: DRILL-5819
> URL: https://issues.apache.org/jira/browse/DRILL-5819
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Karthikeyan Manivannan
> Fix For: 1.12.0
>
>
> Default value of security.admin.user_groups and security.admin.users is "true"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-5865) build broken with commit fe79a63

[Paul Rogers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Rogers resolved DRILL-5865.

Resolution: Fixed
  Assignee: Parth Chandra
  Reviewer: Paul Rogers

> build broken with commit fe79a63
> 
>
> Key: DRILL-5865
> URL: https://issues.apache.org/jira/browse/DRILL-5865
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Parth Chandra
>Assignee: Parth Chandra
>
> Looks like the combination of fe79a63 and 42f7af2 broke the build. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Jenkins build is back to normal : drill-scm #896

[Apache Jenkins Server]

See 

[GitHub] drill pull request #986: DRILL-5865: fix build

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/986


---

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/983


---

[GitHub] drill pull request #987: DRILL-5863: Sortable table incorrectly sorts fragme...

[kkhatua]

GitHub user kkhatua opened a pull request:

https://github.com/apache/drill/pull/987

DRILL-5863: Sortable table incorrectly sorts fragments/time lexically

The DataTables jQuery library sorts data based on the value of the element 
in a column.
However, since Drill publishes sortable items like fragment IDs and time 
durations as non-numeric text, the sorting is incorrect. 
This PR fixes the fragment and duration ordering based on their implicit 
numeric values (minor ID and millisecond representation, respectively).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kkhatua/drill DRILL-5863

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/987.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #987


commit dee20aae602f86d14ddb832e7185499556c6a1e1
Author: Kunal Khatua 
Date:   2017-10-12T00:17:03Z

DRILL-5863: Sortable table incorrectly sorts fragments/time lexically

The DataTables jQuery library sorts data based on the value of the element 
in a column.
However, since Drill publishes sortable items like fragment IDs and time 
durations as non-numeric text, the sorting is incorrect. 
This PR fixes the fragment and duration ordering based on their implicit 
numeric values (minor ID and millisecond representation, respectively).




---

Re: Possible submission for BI tools documentation

[Parth Chandra]

I'm sure this would be useful for many users. Please go ahead and submit it.

On Wed, Oct 11, 2017 at 2:37 PM, Jack Ingoldsby 
wrote:

> Hi,
> I'm a Sales Engineer with Sisense, a growing BI solution provider.
>
> As part of a internal project, I used Drill to query s3 buckets
> successfully and bring the results into our solution through ODBC/JDBC.
>
> Drill configuration didn't take long, I was able to bring in 37m records in
> 22 minutes in embedded mode, on a fairly modest box.
>
> I'm currently writing a blog post on Drill for our organization, but it
> occurred to me it might be nice to include a short note on connecting
> Sisense to Drill for the
> https://drill.apache.org/docs/using-drill-with-bi-tools-introduction/
> documentation.
>
> If I were to write short guide on connecting Sisense to Drill,  would you
> have any interest in adding it to the BI section?
>
> Regards,
> Jack
>

Re: Possible submission for BI tools documentation

[Saurabh Mahapatra]

Thanks Jack for taking the initiative!!! This is great stuff.

I would actually support this. I think you should go ahead and create the pull 
request on the documentation. I think the Drill documentation needs to be more 
inclusive of the work done by the community. We need more companies and private 
individuals to advertise their work on Apache Drill. 

There was some talk about having a plugin page like Presto does where anyone 
from the community can advertise their work. 

I think this is very important and I support this. I am not a committee though 
but I hope others in the community agree with me. I am sure there are many 
others we should document instead of having them scattered around the web in 
blog posts and private documentation.

Thoughts?

Best,
Saurabh





> On Oct 11, 2017, at 5:37 PM, Jack Ingoldsby  wrote:
> 
> Hi,
> I'm a Sales Engineer with Sisense, a growing BI solution provider.
> 
> As part of a internal project, I used Drill to query s3 buckets
> successfully and bring the results into our solution through ODBC/JDBC.
> 
> Drill configuration didn't take long, I was able to bring in 37m records in
> 22 minutes in embedded mode, on a fairly modest box.
> 
> I'm currently writing a blog post on Drill for our organization, but it
> occurred to me it might be nice to include a short note on connecting
> Sisense to Drill for the
> https://drill.apache.org/docs/using-drill-with-bi-tools-introduction/
> documentation.
> 
> If I were to write short guide on connecting Sisense to Drill,  would you
> have any interest in adding it to the BI section?
> 
> Regards,
> Jack

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144147623
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
 ---
@@ -335,20 +333,32 @@ private void purge() throws SchemaChangeException {
 logger.debug("Took {} us to purge", 
watch.elapsed(TimeUnit.MICROSECONDS));
   }
 
-  public PriorityQueue createNewPriorityQueue(FragmentContext context, 
List orderings,
- VectorAccessible 
batch, MappingSet mainMapping, MappingSet leftMapping, MappingSet rightMapping)
-  throws ClassTransformationException, IOException, 
SchemaChangeException{
-CodeGenerator cg = 
CodeGenerator.get(PriorityQueue.TEMPLATE_DEFINITION, 
context.getFunctionRegistry(), context.getOptions());
+  private PriorityQueue createNewPriorityQueue(VectorAccessible batch, int 
limit)
+throws SchemaChangeException, ClassTransformationException, 
IOException {
+return createNewPriorityQueue(context.getOptionSet(), 
context.getFunctionRegistry(), context.getDrillbitContext().getCompiler(),
+  config.getOrderings(), batch, unionTypeEnabled, codegenDump, limit, 
oContext.getAllocator(), schema.getSelectionVectorMode());
+  }
+
+  public static PriorityQueue createNewPriorityQueue(
+OptionSet optionSet, FunctionLookupContext functionLookupContext, 
CodeCompiler codeCompiler,
+List orderings, VectorAccessible batch, boolean 
unionTypeEnabled, boolean codegenDump,
+int limit, BufferAllocator allocator, SelectionVectorMode mode)
+  throws ClassTransformationException, IOException, 
SchemaChangeException {
+final MappingSet mainMapping = new MappingSet((String) null, null, 
ClassGenerator.DEFAULT_SCALAR_MAP, ClassGenerator.DEFAULT_SCALAR_MAP);
+final MappingSet leftMapping = new MappingSet("leftIndex", null, 
ClassGenerator.DEFAULT_SCALAR_MAP, ClassGenerator.DEFAULT_SCALAR_MAP);
+final MappingSet rightMapping = new MappingSet("rightIndex", null, 
ClassGenerator.DEFAULT_SCALAR_MAP, ClassGenerator.DEFAULT_SCALAR_MAP);
--- End diff --

Not sure these should be inside the method. Can we still grab hold of the 
variable for testing or debugging? Might we want to leave this in the class 
name space?


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144132366
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/CodeGenerator.java ---
@@ -90,12 +89,11 @@
   private String generatedCode;
   private String generifiedCode;
 
-  CodeGenerator(TemplateClassDefinition definition, 
FunctionImplementationRegistry funcRegistry, OptionSet optionManager) {
-this(ClassGenerator.getDefaultMapping(), definition, funcRegistry, 
optionManager);
+  CodeGenerator(TemplateClassDefinition definition, OptionSet 
optionManager) {
+this(ClassGenerator.getDefaultMapping(), definition, optionManager);
--- End diff --

Does the code actually work without this item? Isn't the registry where we 
store UDF functions? And, doesn't the expression materializer make use of this 
info to copy UDFs inline into the generated code?

If we can remove this, that is a big convenience. But, I'm skeptical that 
it is, in fact, not needed.


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144131729
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java ---
@@ -110,6 +109,11 @@ public CodeGenCompiler(final DrillConfig config, final 
OptionSet optionManager)
   public static final String DISABLE_CACHE_CONFIG = COMPILE_BASE + 
".disable_cache";
 
   /**
+   * Enables saving generated code for debugging
+   */
+  public static final String ENABLE_SAVE_CODE_FOR_DEBUG = COMPILE_BASE + 
".codegen.dump";
--- End diff --

When saving generated code, I have to point to the directory from Eclipse. 
This means two things:

1. It works best if the same directory is used from one run to the next. 
(That is, no randomly generated temp directory.)
2. The directory has to be visible to the Eclipse UI. (That is, no hidden 
directories starting with dots.)

Can we make the directory in a known location, and can we use a nice simple 
name like "codegen"?


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144142548
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueue.java
 ---
@@ -20,22 +20,58 @@
 import org.apache.drill.exec.compile.TemplateClassDefinition;
 import org.apache.drill.exec.exception.SchemaChangeException;
 import org.apache.drill.exec.memory.BufferAllocator;
-import org.apache.drill.exec.ops.FragmentContext;
 import org.apache.drill.exec.physical.impl.sort.RecordBatchData;
 import org.apache.drill.exec.record.VectorContainer;
 import org.apache.drill.exec.record.selection.SelectionVector4;
 
 public interface PriorityQueue {
-  public void add(FragmentContext context, RecordBatchData batch) throws 
SchemaChangeException;
-  public void init(int limit, FragmentContext context, BufferAllocator 
allocator, boolean hasSv2) throws SchemaChangeException;
-  public void generate() throws SchemaChangeException;
-  public VectorContainer getHyperBatch();
-  public SelectionVector4 getHeapSv4();
-  public SelectionVector4 getFinalSv4();
-  public boolean validate();
-  public void resetQueue(VectorContainer container, SelectionVector4 
vector4) throws SchemaChangeException;
-  public void cleanup();
-
-  public static TemplateClassDefinition TEMPLATE_DEFINITION 
= new TemplateClassDefinition(PriorityQueue.class, 
PriorityQueueTemplate.class);
+  /**
+   * The elements in the given batch are added to the priority queue. Note 
that the priority queue
+   * only retains the top elements that fit within the size specified by 
the {@link #init(int, BufferAllocator, boolean)}
+   * method.
+   * @param batch The batch containing elements we want to add.
+   * @throws SchemaChangeException
+   */
+  void add(RecordBatchData batch) throws SchemaChangeException;
 
+  /**
+   * Initializes the priority queue. This method must be called before any 
other methods on the priority
+   * queue are called.
+   * @param limit The size of the priority queue.
+   * @param allocator The {@link BufferAllocator} to use when creating the 
priority queue.
+   * @param hasSv2 True when incoming batches have v2 selection vectors. 
False otherwise.
+   * @throws SchemaChangeException
+   */
+  void init(int limit, BufferAllocator allocator, boolean hasSv2) throws 
SchemaChangeException;
+
+  /**
+   * This method must be called before fetching the final heap hyper batch 
and final Sv4 vector.
--- End diff --

"heap hyper batch" seems an oxymoron. All batches, including hyper-batches, 
exist in direct memory. What is meant by "heap"?


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144147991
  
--- Diff: 
common/src/test/java/org/apache/drill/testutils/SubDirTestWatcher.java ---
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.testutils;
--- End diff --

You were a brave soul and moved the `DrillTest` and `BaseTestQuery` classes 
into `org.apache.drill.test`. Do we want to move `testutils` under `test` as 
well?


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144131909
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java ---
@@ -110,6 +109,11 @@ public CodeGenCompiler(final DrillConfig config, final 
OptionSet optionManager)
   public static final String DISABLE_CACHE_CONFIG = COMPILE_BASE + 
".disable_cache";
 
   /**
+   * Enables saving generated code for debugging
+   */
+  public static final String ENABLE_SAVE_CODE_FOR_DEBUG = COMPILE_BASE + 
".codegen.dump";
--- End diff --

Also, how does a static string enable code generation? Isn't this just the 
destination folder once saving code is enabled?


---

[GitHub] drill pull request #984: DRILL-5783 Made a unit test for generated Priority ...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/984#discussion_r144147334
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
 ---
@@ -95,7 +91,9 @@ public TopNBatch(TopN popConfig, FragmentContext context, 
RecordBatch incoming)
 super(popConfig, context);
 this.incoming = incoming;
 this.config = popConfig;
-batchPurgeThreshold = 
context.getConfig().getInt(ExecConstants.BATCH_PURGE_THRESHOLD);
+DrillConfig drillConfig = context.getConfig();
+batchPurgeThreshold = 
drillConfig.getInt(ExecConstants.BATCH_PURGE_THRESHOLD);
+codegenDump = 
drillConfig.getBoolean(CodeCompiler.ENABLE_SAVE_CODE_FOR_DEBUG);
--- End diff --

Not sure we want to enable this in this way. This would globally enable all 
code to be dumped. In practice, we want to enable module by module depending on 
what we are debugging.

I've been doing this the crude-but-effective way: just uncommenting the 
magic line of code. This usually works because I have to be in the debugger 
anyway to step through the code. But, if we wanted to be fancier, we could have 
config settings for each package. So, here, maybe you would enable 
`drill.debug.codegen.org.apache.drill.exec.physical.impl.TopN`. Or, we could be 
a bit lighter weight and just assign names: `drill.debug.codegen.TopN`.

Because a typical run will generate zillions of files, reducing the number 
of saved files to those of interest will help to keep things tidy.


---

[GitHub] drill issue #986: DRILL-5865: fix build

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/986
  
@paul-rogers please review (and merge). 


---

[GitHub] drill pull request #986: DRILL-5865: fix build

[parthchandra]

GitHub user parthchandra opened a pull request:

https://github.com/apache/drill/pull/986

DRILL-5865: fix build



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/parthchandra/drill DRILL-5865

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/986.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #986


commit d1bfa16317659269077dc82e896941f41bb22e3c
Author: Parth Chandra 
Date:   2017-10-11T23:40:12Z

DRILL-5865: fix build




---

[jira] [Created] (DRILL-5865) build broken with commit fe79a63

[Parth Chandra (JIRA)]

Parth Chandra created DRILL-5865:


 Summary: build broken with commit fe79a63
 Key: DRILL-5865
 URL: https://issues.apache.org/jira/browse/DRILL-5865
 Project: Apache Drill
  Issue Type: Bug
Reporter: Parth Chandra


Looks like the combination of fe79a63 and 42f7af2 broke the build. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #973: DRILL-5775: Select * query on a maprdb binary table...

[vdiravka]

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/973#discussion_r144158971
  
--- Diff: 
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseUtils.java
 ---
@@ -139,4 +142,26 @@ public static Filter orFilterAtIndex(Filter 
currentFilter, int index, Filter new
 return Bytes.compareTo(left, right) < 0 ? left : right;
   }
 
+
+  /**
+   * Verify the presence of a column family in the schema path of the 
hbase table or whether the schema path is
+   * the row key column.
+   *
+   * @param columns List of the selected columns
+   * @param hTableDesc HTableDescriptor of HBase/MapR-DB_binary table 
(consists the details about that table)
+   * @throws DrillRuntimeException if column family does not exist, or is 
not row_key column.
+   */
+  public static void verifyColumns(List columns, 
HTableDescriptor hTableDesc) {
--- End diff --

DRILL-5546 introduced expanding wildcard in the `verifyColumns()` method 
with renaming to `verifyColumnsAndConvertStar()`. But this PR keeps the 
original `verifyColumns()` method and the same in DRILL-5830 (the star isn't 
expanded here). Just the method have started to be common to HBase and MapR-DB. 
And MapR-DB plugin for now leverages the same way of project push down as 
HBase due to the `MapRDBBinaryTable` introducing.
Or did I miss something?


---

[jira] [Created] (DRILL-5864) Selecting a non-existing field from a MapR-DB JSON table fails with NPE

[Abhishek Girish (JIRA)]

Abhishek Girish created DRILL-5864:
--

 Summary: Selecting a non-existing field from a MapR-DB JSON table 
fails with NPE
 Key: DRILL-5864
 URL: https://issues.apache.org/jira/browse/DRILL-5864
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators, Storage - MapRDB
Affects Versions: 1.12.0
Reporter: Abhishek Girish
Assignee: Hanumath Rao Maduri
 Attachments: OrderByNPE.log, OrderByNPE2.log

Query 1
{code}
> select C_FIRST_NAME,C_BIRTH_COUNTRY,C_BIRTH_YEAR,C_BIRTH_MONTH,C_BIRTH_DAY 
> from customer ORDER BY C_BIRTH_COUNTRY ASC, C_FIRST_NAME ASC LIMIT 10;
Error: SYSTEM ERROR: NullPointerException

  (java.lang.NullPointerException) null
org.apache.drill.exec.record.SchemaUtil.coerceContainer():176

org.apache.drill.exec.physical.impl.xsort.managed.BufferedBatches.convertBatch():124
org.apache.drill.exec.physical.impl.xsort.managed.BufferedBatches.add():90
org.apache.drill.exec.physical.impl.xsort.managed.SortImpl.addBatch():265

org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.loadBatch():421

org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load():357

org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.innerNext():302
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext():93
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext():115
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext():115
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext():93
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():164
org.apache.drill.exec.physical.impl.BaseRootExec.next():105
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
org.apache.drill.exec.physical.impl.BaseRootExec.next():95
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1595
org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748 (state=,code=0)
{code}
Plan
{code}
00-00Screen
00-01  Project(C_FIRST_NAME=[$0], C_BIRTH_COUNTRY=[$1], C_BIRTH_YEAR=[$2], 
C_BIRTH_MONTH=[$3], C_BIRTH_DAY=[$4])
00-02SelectionVectorRemover
00-03  Limit(fetch=[10])
00-04Limit(fetch=[10])
00-05  SelectionVectorRemover
00-06Sort(sort0=[$1], sort1=[$0], dir0=[ASC], dir1=[ASC])
00-07  Scan(groupscan=[JsonTableGroupScan 
[ScanSpec=JsonScanSpec 
[tableName=maprfs:///drill/testdata/tpch/sf1/maprdb/json/range/customer, 
condition=null], columns=[`C_FIRST_NAME`, `C_BIRTH_COUNTRY`, `C_BIRTH_YEAR`, 
`C_BIRTH_MONTH`, `C_BIRTH_DAY`]]])
{code}

Query 2
{code}
> select C_SALUTATION,C_FIRST_NAME,C_LAST_NAME from customer ORDER BY 
> C_BIRTH_COUNTRY DESC, C_FIRST_NAME DESC LIMIT 10;
Error: SYSTEM ERROR: NullPointerException

Fragment 0:0

[Error Id: fc02de86-ce9b-418a-80ce-b7f5365911e1 on sidrill1:31010]

  (java.lang.NullPointerException) null

[jira] [Created] (DRILL-5863) Sortable table incorrectly sorts minor fragments and time elements lexically instead of sorting by implicit value

[Kunal Khatua (JIRA)]

Kunal Khatua created DRILL-5863:
---

 Summary: Sortable table incorrectly sorts minor fragments and time 
elements lexically instead of sorting by implicit value
 Key: DRILL-5863
 URL: https://issues.apache.org/jira/browse/DRILL-5863
 Project: Apache Drill
  Issue Type: Bug
  Components: Web Server
Affects Versions: 1.12.0
Reporter: Kunal Khatua
Assignee: Kunal Khatua
Priority: Minor
 Fix For: 1.12.0


The fix for this is to use dataTable library's {{data-order}} attribute for the 
data elements that need to sort by an implicit value.

||Old order of Minor Fragment||New order of Minor Fragment||
|...|...|
|01-09-01  | 01-09-01|
|01-10-01  | 01-10-01|
|01-100-01 | 01-11-01|
|01-101-01 | 01-12-01|
|... | ... |

||Old order of Duration||New order of Duration|||
|...|...|
|1m15s  | 55.03s|
|55s  | 1m15s|
|...|...|




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144149536
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/test/ClientFixture.java ---
@@ -128,6 +128,21 @@ public void alterSystem(String key, Object value) {
   }
 
   /**
+   * Reset a system option
+   * @param key
+   */
+
+  public void resetSystem(String key) {
--- End diff --

These are added in PR #970, in [this 
file](https://github.com/apache/drill/pull/970/files#diff-e2de7c62cedd56fbabbf3cff0f0663a2).

But, this PR is likely to be committed before this one, so I'll deal with 
the merge conflict later.


---

Build failed in Jenkins: drill-scm #895

[Apache Jenkins Server]

See 

Changes:

[progers] DRILL-5830: Resolve regressions to MapR DB from DRILL-5546

[progers] DRILL-5775: Select * query on a maprdb binary table fails

--
[...truncated 159.25 KB...]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
drill-kudu-storage ---
[INFO] 
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ drill-kudu-storage ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-checkstyle-plugin:2.12.1:check (checkstyle-validation) @ 
drill-kudu-storage ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (avoid_bad_dependencies) @ 
drill-kudu-storage ---
[INFO] 
[INFO] --- maven-install-plugin:2.5.1:install (default-install) @ 
drill-kudu-storage ---
[INFO] Installing 

 to 
/home/jenkins/.m2/repository/org/apache/drill/contrib/drill-kudu-storage/1.12.0-SNAPSHOT/drill-kudu-storage-1.12.0-SNAPSHOT.jar
[INFO] Installing 
 to 
/home/jenkins/.m2/repository/org/apache/drill/contrib/drill-kudu-storage/1.12.0-SNAPSHOT/drill-kudu-storage-1.12.0-SNAPSHOT.pom
[INFO] Installing 

 to 
/home/jenkins/.m2/repository/org/apache/drill/contrib/drill-kudu-storage/1.12.0-SNAPSHOT/drill-kudu-storage-1.12.0-SNAPSHOT-tests.jar
[INFO] 
[INFO] 
[INFO] Building contrib/mongo-storage-plugin 1.12.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ drill-mongo-storage 
---
[INFO] Deleting 

[INFO] 
[INFO] --- apache-rat-plugin:0.12:check (rat-checks) @ drill-mongo-storage ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 61 implicit excludes (use -debug for more details).
[INFO] Exclude: **/*.log
[INFO] Exclude: **/*.css
[INFO] Exclude: **/*.js
[INFO] Exclude: **/*.md
[INFO] Exclude: **/*.eps
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.seq
[INFO] Exclude: **/*.parquet
[INFO] Exclude: **/*.sql
[INFO] Exclude: **/git.properties
[INFO] Exclude: **/*.csv
[INFO] Exclude: **/*.csvh
[INFO] Exclude: **/*.csvh-test
[INFO] Exclude: **/*.tsv
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.ssv
[INFO] Exclude: **/drill-*.conf
[INFO] Exclude: **/.buildpath
[INFO] Exclude: **/*.proto
[INFO] Exclude: **/*.fmpp
[INFO] Exclude: **/target/**
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.tdd
[INFO] Exclude: **/*.project
[INFO] Exclude: **/TAGS
[INFO] Exclude: **/*.checkstyle
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.settings/**
[INFO] Exclude: .*/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: **/*.pb.cc
[INFO] Exclude: **/*.pb.h
[INFO] Exclude: **/*.linux
[INFO] Exclude: **/client/build/**
[INFO] Exclude: **/cmake_install.cmake
[INFO] Exclude: **/*.tbl
[INFO] Exclude: **/*.httpd
[INFO] Exclude: **/*.autotools
[INFO] Exclude: **/*.cproject
[INFO] Exclude: **/*.drill
[INFO] Exclude: dependency-reduced-pom.xml
[INFO] 29 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 29 licenses.
[INFO] 
[INFO] --- git-commit-id-plugin:2.1.9:revision (for-jars) @ drill-mongo-storage 
---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
drill-mongo-storage ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
drill-mongo-storage ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
drill-mongo-storage ---
[INFO] Compiling 20 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ 
drill-mongo-storage ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
drill-mongo-storage ---
[INFO] Compiling 8 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ 

Possible submission for BI tools documentation

[Jack Ingoldsby]

Hi,
I'm a Sales Engineer with Sisense, a growing BI solution provider.

As part of a internal project, I used Drill to query s3 buckets
successfully and bring the results into our solution through ODBC/JDBC.

Drill configuration didn't take long, I was able to bring in 37m records in
22 minutes in embedded mode, on a fairly modest box.

I'm currently writing a blog post on Drill for our organization, but it
occurred to me it might be nice to include a short note on connecting
Sisense to Drill for the
https://drill.apache.org/docs/using-drill-with-bi-tools-introduction/
documentation.

If I were to write short guide on connecting Sisense to Drill,  would you
have any interest in adding it to the BI section?

Regards,
Jack

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143076
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestOptionsAuthEnabled.java
 ---
@@ -115,4 +124,59 @@ private void setOptHelper() throws Exception {
   test(String.format("ALTER SYSTEM SET `%s` = %d;", 
ExecConstants.SLICE_TARGET, ExecConstants.SLICE_TARGET_DEFAULT));
 }
   }
+
+  @Test
+  public void testAdminUserOptions() throws Exception {
+FixtureBuilder builder = ClusterFixture.builder();
+
+try (ClusterFixture cluster = builder.build();
+ ClientFixture client = cluster.clientFixture()) {
+  OptionManager optionManager = 
cluster.drillbit().getContext().getOptionManager();
+
+  // Admin Users Tests
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUser =  
optionManager.getOption(ExecConstants.ADMIN_USERS_VALIDATOR);;
+  assertEquals(configAdminUser, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Option accessor should never return the 'fake' default from the 
config
+  String adminUser1 = 
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager);
+  assertNotEquals(adminUser1, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Change TEST_ADMIN_USER if necessary
+  String TEST_ADMIN_USER = "ronswanson";
+  if (adminUser1.equals(TEST_ADMIN_USER)) {
+TEST_ADMIN_USER += "thefirst";
+  }
+  // Check if the admin option accessor honors a user-supplied values
+  String sql = String.format("ALTER SYSTEM SET `%s`='%s'", 
ExecConstants.ADMIN_USERS_KEY, TEST_ADMIN_USER);
+  client.queryBuilder().sql(sql).run();
+  String adminUser2 = 
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager);
+  assertEquals(adminUser2, TEST_ADMIN_USER);
+
+  // Admin User Groups Tests
+
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUserGroups =  
optionManager.getOption(ExecConstants.ADMIN_USER_GROUPS_VALIDATOR);
+  assertEquals(configAdminUserGroups, 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.DEFAULT_ADMIN_USER_GROUPS);
+
+  // Option accessor should never return the 'fake' default from the 
config
+  String adminUserGroups1 = 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.getAdminUserGroups(optionManager);
+  assertNotEquals(adminUserGroups1, 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.DEFAULT_ADMIN_USER_GROUPS);
+
+  // Change TEST_ADMIN_USER_GROUPS if necessary
+  String TEST_ADMIN_USER_GROUPS = "yakshavers";
+  if (adminUserGroups1.equals(TEST_ADMIN_USER_GROUPS)) {
+TEST_ADMIN_USER_GROUPS += ",wormracers";
+  }
+  // Check if the admin option accessor honors a user-supplied values
+  sql = String.format("ALTER SYSTEM SET `%s`='%s'", 
ExecConstants.ADMIN_USER_GROUPS_KEY, TEST_ADMIN_USER_GROUPS);
+  client.queryBuilder().sql(sql).run();
+  String adminUserGroups2 = 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.getAdminUserGroups(optionManager);
+  assertEquals(adminUserGroups2, TEST_ADMIN_USER_GROUPS);
--- End diff --

I am pushing changes which add these tests.


---

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143072
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DrillRoot.java 
---
@@ -75,6 +80,29 @@ public ClusterInfo getClusterInfoJSON() {
 final DrillConfig config = dbContext.getConfig();
 final boolean userEncryptionEnabled = 
config.getBoolean(ExecConstants.USER_ENCRYPTION_SASL_ENABLED);
 final boolean bitEncryptionEnabled = 
config.getBoolean(ExecConstants.BIT_ENCRYPTION_SASL_ENABLED);
+// If the user is logged in and is admin user then show the admin user 
info
+// For all other cases the user info need-not or should-not be 
displayed
+OptionManager optionManager = work.getContext().getOptionManager();
+final boolean isUserLoggedIn = AuthDynamicFeature.isUserLoggedIn(sc);
+String adminUsers = isUserLoggedIn ?
+
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager) : null;
+String adminUserGroups = isUserLoggedIn ?
+
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.getAdminUserGroups(optionManager) : 
null;
+
+// separate groups by comma + space
+if (adminUsers != null) {
+  String[] groups = adminUsers.split(",");
+  adminUsers = DrillStringUtils.join(groups, ", ");
+}
+
+// separate groups by comma + space
+if (adminUserGroups != null) {
+  String[] groups = adminUserGroups.split(",");
+  adminUserGroups = DrillStringUtils.join(groups, ", ");
--- End diff --

I pushing changes that handle ill-formatted user input


---

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143097
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestOptionsAuthEnabled.java
 ---
@@ -115,4 +124,59 @@ private void setOptHelper() throws Exception {
   test(String.format("ALTER SYSTEM SET `%s` = %d;", 
ExecConstants.SLICE_TARGET, ExecConstants.SLICE_TARGET_DEFAULT));
 }
   }
+
+  @Test
+  public void testAdminUserOptions() throws Exception {
+FixtureBuilder builder = ClusterFixture.builder();
+
+try (ClusterFixture cluster = builder.build();
+ ClientFixture client = cluster.clientFixture()) {
+  OptionManager optionManager = 
cluster.drillbit().getContext().getOptionManager();
+
+  // Admin Users Tests
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUser =  
optionManager.getOption(ExecConstants.ADMIN_USERS_VALIDATOR);;
+  assertEquals(configAdminUser, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Option accessor should never return the 'fake' default from the 
config
+  String adminUser1 = 
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager);
+  assertNotEquals(adminUser1, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Change TEST_ADMIN_USER if necessary
+  String TEST_ADMIN_USER = "ronswanson";
+  if (adminUser1.equals(TEST_ADMIN_USER)) {
+TEST_ADMIN_USER += "thefirst";
+  }
+  // Check if the admin option accessor honors a user-supplied values
+  String sql = String.format("ALTER SYSTEM SET `%s`='%s'", 
ExecConstants.ADMIN_USERS_KEY, TEST_ADMIN_USER);
+  client.queryBuilder().sql(sql).run();
--- End diff --

Done.


---

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143113
  
--- Diff: 
common/src/main/java/org/apache/drill/common/util/DrillStringUtils.java ---
@@ -1,203 +1,258 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.drill.common.util;
-
-import io.netty.buffer.ByteBuf;
-
-import org.apache.commons.lang3.StringEscapeUtils;
-import org.apache.commons.lang3.StringUtils;
-
-public class DrillStringUtils {
-
-  /**
-   * Converts the long number into more human readable string.
-   */
-  public static String readable(long bytes) {
-int unit = 1024;
-long absBytes = Math.abs(bytes);
-if (absBytes < unit) {
-  return bytes + " B";
-}
-int exp = (int) (Math.log(absBytes) / Math.log(unit));
-char pre = ("KMGTPE").charAt(exp-1);
-return String.format("%s%.1f %ciB", (bytes == absBytes ? "" : "-"), 
absBytes / Math.pow(unit, exp), pre);
-  }
-
-
-  /**
-   * Unescapes any Java literals found in the {@code String}.
-   * For example, it will turn a sequence of {@code '\'} and
-   * {@code 'n'} into a newline character, unless the {@code '\'}
-   * is preceded by another {@code '\'}.
-   *
-   * @param input  the {@code String} to unescape, may be null
-   * @return a new unescaped {@code String}, {@code null} if null string 
input
-   */
-  public static final String unescapeJava(String input) {
-return StringEscapeUtils.unescapeJava(input);
-  }
-
-  /**
-   * Escapes the characters in a {@code String} according to Java string 
literal
-   * rules.
-   *
-   * Deals correctly with quotes and control-chars (tab, backslash, cr, ff,
-   * etc.) so, for example, a tab becomes the characters {@code '\\'} and
-   * {@code 't'}.
-   *
-   * Example:
-   * 
-   * input string: He didn't say, "Stop!"
-   * output string: He didn't say, \"Stop!\"
-   * 
-   *
-   * @param input  String to escape values in, may be null
-   * @return String with escaped values, {@code null} if null string input
-   */
-  public static final String escapeJava(String input) {
-return StringEscapeUtils.escapeJava(input);
-  }
-
-  public static final String escapeNewLines(String input) {
-if (input == null) {
-  return null;
-}
-StringBuilder result = new StringBuilder();
-boolean sawNewline = false;
-for (int i = 0; i < input.length(); i++) {
-  char curChar = input.charAt(i);
-  if (curChar == '\r' || curChar == '\n') {
-if (sawNewline) {
-  continue;
-}
-sawNewline = true;
-result.append("\\n");
-  } else {
-sawNewline = false;
-result.append(curChar);
-  }
-}
-return result.toString();
-  }
-
-  /**
-   * Copied form commons-lang 2.x code as common-lang 3.x has this API 
removed.
-   * 
(http://commons.apache.org/proper/commons-lang/article3_0.html#StringEscapeUtils.escapeSql)
-   * @param str
-   * @return
-   */
-  public static String escapeSql(String str) {
-return (str == null) ? null : StringUtils.replace(str, "'", "''");
-  }
-
-  /**
-   * Return a printable representation of a byte buffer, escaping the 
non-printable
-   * bytes as '\\xNN' where NN is the hexadecimal representation of such 
bytes.
-   *
-   * This function does not modify  the {@code readerIndex} and {@code 
writerIndex}
-   * of the byte buffer.
-   */
-  public static String toBinaryString(ByteBuf buf, int strStart, int 
strEnd) {
-StringBuilder result = new StringBuilder();
-for (int i = strStart; i < strEnd ; ++i) {
-  appendByte(result, buf.getByte(i));
-}
-return result.toString();
-  }
-
-  /**
-   * Return a printable representation of a byte array, 

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143087
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestOptionsAuthEnabled.java
 ---
@@ -115,4 +124,59 @@ private void setOptHelper() throws Exception {
   test(String.format("ALTER SYSTEM SET `%s` = %d;", 
ExecConstants.SLICE_TARGET, ExecConstants.SLICE_TARGET_DEFAULT));
 }
   }
+
+  @Test
+  public void testAdminUserOptions() throws Exception {
+FixtureBuilder builder = ClusterFixture.builder();
+
+try (ClusterFixture cluster = builder.build();
+ ClientFixture client = cluster.clientFixture()) {
+  OptionManager optionManager = 
cluster.drillbit().getContext().getOptionManager();
+
+  // Admin Users Tests
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUser =  
optionManager.getOption(ExecConstants.ADMIN_USERS_VALIDATOR);;
+  assertEquals(configAdminUser, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Option accessor should never return the 'fake' default from the 
config
+  String adminUser1 = 
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager);
+  assertNotEquals(adminUser1, 
ExecConstants.ADMIN_USERS_VALIDATOR.DEFAULT_ADMIN_USERS);
+
+  // Change TEST_ADMIN_USER if necessary
+  String TEST_ADMIN_USER = "ronswanson";
+  if (adminUser1.equals(TEST_ADMIN_USER)) {
+TEST_ADMIN_USER += "thefirst";
+  }
+  // Check if the admin option accessor honors a user-supplied values
+  String sql = String.format("ALTER SYSTEM SET `%s`='%s'", 
ExecConstants.ADMIN_USERS_KEY, TEST_ADMIN_USER);
+  client.queryBuilder().sql(sql).run();
+  String adminUser2 = 
ExecConstants.ADMIN_USERS_VALIDATOR.getAdminUsers(optionManager);
+  assertEquals(adminUser2, TEST_ADMIN_USER);
+
+  // Admin User Groups Tests
+
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUserGroups =  
optionManager.getOption(ExecConstants.ADMIN_USER_GROUPS_VALIDATOR);
+  assertEquals(configAdminUserGroups, 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.DEFAULT_ADMIN_USER_GROUPS);
+
+  // Option accessor should never return the 'fake' default from the 
config
+  String adminUserGroups1 = 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.getAdminUserGroups(optionManager);
+  assertNotEquals(adminUserGroups1, 
ExecConstants.ADMIN_USER_GROUPS_VALIDATOR.DEFAULT_ADMIN_USER_GROUPS);
+
+  // Change TEST_ADMIN_USER_GROUPS if necessary
+  String TEST_ADMIN_USER_GROUPS = "yakshavers";
+  if (adminUserGroups1.equals(TEST_ADMIN_USER_GROUPS)) {
+TEST_ADMIN_USER_GROUPS += ",wormracers";
--- End diff --

Done. It was a constant before I changed the code to handle the corner 
case. 


---

[GitHub] drill pull request #983: DRILL-5819: Default value of security.admin.user_gr...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/983#discussion_r144143103
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/server/TestOptionsAuthEnabled.java
 ---
@@ -115,4 +124,59 @@ private void setOptHelper() throws Exception {
   test(String.format("ALTER SYSTEM SET `%s` = %d;", 
ExecConstants.SLICE_TARGET, ExecConstants.SLICE_TARGET_DEFAULT));
 }
   }
+
+  @Test
+  public void testAdminUserOptions() throws Exception {
+FixtureBuilder builder = ClusterFixture.builder();
+
+try (ClusterFixture cluster = builder.build();
+ ClientFixture client = cluster.clientFixture()) {
+  OptionManager optionManager = 
cluster.drillbit().getContext().getOptionManager();
+
+  // Admin Users Tests
+  // config file should have the 'fake' default admin user and it 
should be returned
+  // by the option manager if the option has not been set by the user
+  String configAdminUser =  
optionManager.getOption(ExecConstants.ADMIN_USERS_VALIDATOR);;
--- End diff --

Done


---

[GitHub] drill pull request #968: DRILL-5830: Resolve regressions to MapR DB from DRI...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/968


---

[GitHub] drill pull request #973: DRILL-5775: Select * query on a maprdb binary table...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/973


---

[GitHub] drill issue #941: DRILL-5775: Expand * into list of column/column family for...

[prasadns14]

Github user prasadns14 commented on the issue:

https://github.com/apache/drill/pull/941
  
Yes, this PR is now obsolete. All the cases are covered as part of 
DRILL-5830 and DRILL-5775


---

[GitHub] drill issue #941: DRILL-5775: Expand * into list of column/column family for...

[paul-rogers]

Github user paul-rogers commented on the issue:

https://github.com/apache/drill/pull/941
  
Is this PR obsolete after DRILL-5830 and the other DRILL-5775? If so can we 
close this one? Otherwise, if this one addresses a case not covered by the 
others, can we revise this PR to cover just that other case, on top of latest 
master? Thanks!


---

[GitHub] drill pull request #973: DRILL-5775: Select * query on a maprdb binary table...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/973#discussion_r144124764
  
--- Diff: 
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseUtils.java
 ---
@@ -139,4 +142,26 @@ public static Filter orFilterAtIndex(Filter 
currentFilter, int index, Filter new
 return Bytes.compareTo(left, right) < 0 ? left : right;
   }
 
+
+  /**
+   * Verify the presence of a column family in the schema path of the 
hbase table or whether the schema path is
+   * the row key column.
+   *
+   * @param columns List of the selected columns
+   * @param hTableDesc HTableDescriptor of HBase/MapR-DB_binary table 
(consists the details about that table)
+   * @throws DrillRuntimeException if column family does not exist, or is 
not row_key column.
+   */
+  public static void verifyColumns(List columns, 
HTableDescriptor hTableDesc) {
--- End diff --

As it turns out, this PR is introducing the very ambiguity that DRILL-5830 
tried to remove from DRILL-5546. That is, we now have to separate ways to 
expand the wildcard: this way and the project push down way. We need this 
commit, so we'll accept this for now. But, moving forward, we should clear up 
this ambiguity.


---

[GitHub] drill pull request #985: DRILL-5862 Update project parent pom xml to the lat...

[vrozov]

GitHub user vrozov opened a pull request:

https://github.com/apache/drill/pull/985

DRILL-5862 Update project parent pom xml to the latest ASF version



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vrozov/drill DRILL-5862

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/985.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #985


commit cdcfa4d991f7fa5598bb9f74291c37bd6204209f
Author: Vlad Rozov 
Date:   2017-10-11T15:05:25Z

DRILL-5862 Update project parent pom xml to the latest ASF version




---

[jira] [Created] (DRILL-5862) Update project parent pom xml to the latest ASF version

[Vlad Rozov (JIRA)]

Vlad Rozov created DRILL-5862:
-

 Summary: Update project parent pom xml to the latest ASF version
 Key: DRILL-5862
 URL: https://issues.apache.org/jira/browse/DRILL-5862
 Project: Apache Drill
  Issue Type: Improvement
  Components: Tools, Build & Test
Reporter: Vlad Rozov
Assignee: Vlad Rozov
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #968: DRILL-5830: Resolve regressions to MapR DB from DRI...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/968#discussion_r144111856
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java
 ---
@@ -150,6 +162,40 @@ public boolean load(RecordBatchDef def, DrillBuf buf) 
throws SchemaChangeExcepti
 return schemaChanged;
   }
 
+  private boolean checkMapSchema(Collection 
currentChildren,
--- End diff --

Renamed and added Javadoc comments to better explain the purpose.


---

[GitHub] drill pull request #968: DRILL-5830: Resolve regressions to MapR DB from DRI...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/968#discussion_r144114556
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestLoad.java 
---
@@ -119,7 +122,283 @@ public void testLoadValueVector() throws Exception {
   }
 }
 assertEquals(100, recordCount);
+
+// Free the original vectors
+
+writableBatch.clear();
+
+// Free the deserialized vectors
+
 batchLoader.clear();
+
+// The allocator will verify that the frees were done correctly.
+
+allocator.close();
+  }
+
+  // TODO: Replace this low-level code with RowSet usage once
+  // DRILL-5657 is committed to master.
+
+  private static List createVectors(BufferAllocator 
allocator, BatchSchema schema, int i) {
+final List vectors = new ArrayList<>();
+for (MaterializedField field : schema) {
+  @SuppressWarnings("resource")
+  ValueVector v = TypeHelper.getNewVector(field, allocator);
+  AllocationHelper.allocate(v, 100, 50);
+  v.getMutator().generateTestData(100);
+  vectors.add(v);
+}
+return vectors;
+  }
+
+  private static DrillBuf serializeBatch(BufferAllocator allocator, 
WritableBatch writableBatch) {
+final ByteBuf[] byteBufs = writableBatch.getBuffers();
+int bytes = 0;
+for (ByteBuf buf : byteBufs) {
+  bytes += buf.writerIndex();
+}
+final DrillBuf byteBuf = allocator.buffer(bytes);
+int index = 0;
+for (ByteBuf buf : byteBufs) {
+  buf.readBytes(byteBuf, index, buf.writerIndex());
+  index += buf.writerIndex();
+}
+byteBuf.writerIndex(bytes);
+return byteBuf;
+  }
+
+  @SuppressWarnings("resource")
+  private boolean loadBatch(BufferAllocator allocator,
+  final RecordBatchLoader batchLoader,
+  BatchSchema schema) throws SchemaChangeException {
+final List vectors = createVectors(allocator, schema, 
100);
+final WritableBatch writableBatch = WritableBatch.getBatchNoHV(100, 
vectors, false);
+final DrillBuf byteBuf = serializeBatch(allocator, writableBatch);
+boolean result = batchLoader.load(writableBatch.getDef(), byteBuf);
+byteBuf.release();
 writableBatch.clear();
+return result;
+  }
+
+  @Test
+  public void testSchemaChange() throws SchemaChangeException {
+final BufferAllocator allocator = 
RootAllocatorFactory.newRoot(drillConfig);
+final RecordBatchLoader batchLoader = new RecordBatchLoader(allocator);
+
+// Initial schema: a: INT, b: VARCHAR
+// Schema change: N/A
+
+BatchSchema schema1 = new SchemaBuilder()
+.add("a", MinorType.INT)
+.add("b", MinorType.VARCHAR)
+.build();
+{
+  assertTrue(loadBatch(allocator, batchLoader, schema1));
+  assertTrue(schema1.isEquivalent(batchLoader.getSchema()));
+  batchLoader.getContainer().zeroVectors();
+}
+
+// Same schema
+// Schema change: No
+
+{
+  assertFalse(loadBatch(allocator, batchLoader, schema1));
+  assertTrue(schema1.isEquivalent(batchLoader.getSchema()));
+  batchLoader.getContainer().zeroVectors();
+}
+
+// Reverse columns: b: VARCHAR, a: INT
+// Schema change: ?
+
+{
+  BatchSchema schema = new SchemaBuilder()
+  .add("b", MinorType.VARCHAR)
+  .add("a", MinorType.INT)
+  .build();
+  assertFalse(loadBatch(allocator, batchLoader, schema));
+
+  // Potential bug: see DRILL-5828
+
+  assertTrue(schema.isEquivalent(batchLoader.getSchema()));
+  batchLoader.getContainer().zeroVectors();
+}
+
+// Drop a column: a: INT
+// Schema change: ?
--- End diff --

Fixed.


---

[GitHub] drill pull request #968: DRILL-5830: Resolve regressions to MapR DB from DRI...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/968#discussion_r144113545
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/util/VectorUtil.java ---
@@ -42,7 +43,12 @@ public static void 
showVectorAccessibleContent(VectorAccessible va, final String
 System.out.println(rows + " row(s):");
 List columns = Lists.newArrayList();
 for (VectorWrapper vw : va) {
-  columns.add(vw.getValueVector().getField().getName());
+  MaterializedField field = vw.getValueVector().getField();
--- End diff --

Fixed.


---

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/958


---

[jira] [Resolved] (DRILL-5854) IllegalStateException when empty batch is received.

[Paul Rogers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Rogers resolved DRILL-5854.

Resolution: Fixed
  Reviewer: Paul Rogers

> IllegalStateException when empty batch is received.
> ---
>
> Key: DRILL-5854
> URL: https://issues.apache.org/jira/browse/DRILL-5854
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.11.0
>Reporter: Padma Penumarthy
>Assignee: Padma Penumarthy
>  Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> When we get a batch with record count 0 and empty schema from one of the 
> senders in the merge receiver, 
> the following exception is thrown sometimes.
> Problem is that merge receiver is reading from the wrong sender when first 
> batch is empty from one of the senders.
> When first batch is empty, we are continuing without moving to the next 
> sender.
> Error: SYSTEM ERROR: IllegalStateException: Cleanup before finished. 0 out of 
> 1 streams have finished
> Fragment 0:0
> [Error Id: 0dec4617-eb06-48fd-80c9-75e9c1e74ce2 on sidrill1:31010]
>   (java.lang.IllegalStateException) Cleanup before finished. 0 out of 1 
> streams have finished
> org.apache.drill.exec.work.batch.BaseRawBatchBuffer.close():107
> org.apache.drill.common.AutoCloseables.close():76
> org.apache.drill.common.AutoCloseables.close():64
> org.apache.drill.exec.work.batch.AbstractDataCollector.close():119
> org.apache.drill.common.AutoCloseables.close():76
> org.apache.drill.exec.work.batch.IncomingBuffers.close():140
> org.apache.drill.exec.ops.FragmentContext.suppressingClose():436
> org.apache.drill.exec.ops.FragmentContext.close():429
> 
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources():320
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup():155
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():264
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624
> java.lang.Thread.run():748 (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-5859) Time for query queuing timeout not display correctly in WebUI

[Paul Rogers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Rogers resolved DRILL-5859.

Resolution: Fixed

> Time for query queuing timeout not display correctly in WebUI
> -
>
> Key: DRILL-5859
> URL: https://issues.apache.org/jira/browse/DRILL-5859
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Client - HTTP
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Prasad Nagaraj Subramanya
>Priority: Minor
> Fix For: 1.12.0
>
>
> With exec.queue.enable=true and exec.queue.timeout_millis=30 (ms), the 
> data for timed out queries due to exceeding the exec.queue.timeout_millis 
> limit in the WebUI shows as:
> {code}
> State:FAILED  Duration:   05 min 0.559 sec
> Foreman:  qa-node114.qa.lab   Planning:   0.529 sec
> Total Fragments:  0   Queued: Not Available
> Total Cost:   575,415,266.982 Execution:  05 min 0.030 sec
> {code}
> The "05 min 0.030 sec" value should be for the "Queued" key instead of the 
> "Execution" key.
> Also, the error in the log file for the query is shown as:
> "RESOURCE ERROR: Query timed out of the large queue after 300 ms."
> It should say "... 300 seconds"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #980: DRILL-5857: Fix NumberFormatException in Hive unit ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/980


---

[GitHub] drill pull request #892: DRILL-5645: negation of expression causes null poin...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/892


---

[GitHub] drill pull request #981: DRILL-5854: IllegalStateException when empty batch ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/981


---

[GitHub] drill pull request #977: DRILL-5849: Add freemarker lib to dependencyManagem...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/977


---

[GitHub] drill pull request #982: DRILL-5859: Fixed queued time calculation

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/982


---

Re: Commons-logging

[Paul Rogers]

Hi Charles,

Drill uses Logback logging in place of commons logging. There is a bridge of 
some kind that allows code written for one to work with the other.

Yes, Maven does have dependency management. Maven is the most difficult 
software I’ve ever used; the documentation is worse than awful. Maven is a 
black art unless you’ve actually read the source code. The structure of the pom 
files appears not meant for human consumption. So, if you are confused, that is 
only natural.

That said, the quick & dirty solution is simply to search the pom files for 
“commons-logging” to see how dependency exclusions are done elsewhere; there 
are many to choose from.

- Paul


> On Oct 11, 2017, at 7:58 AM, Charles Givre  wrote:
> 
> Hello Drill Devs, 
> I am working on submitting some UDFs, and in so doing discovered that one UDF 
> has a dependency (commons-validator) which itself has a dependency of 
> commons-logging which is banned. Does anyone  know why commons-logging is 
> banned?   I’m not a maven expert but is it possible to exclude the 
> commons-logging dependency somehow so that we could use commons-validator in 
> Drill?
> Thanks,
> — C

[GitHub] drill issue #971: Drill-5834 Add Networking Functions

[cgivre]

Github user cgivre commented on the issue:

https://github.com/apache/drill/pull/971
  
Thanks!  I’ll resubmit this weekend using commons-validator


> On Oct 11, 2017, at 11:26, Arina Ielchiieva  
wrote:
> 
> @arina-ielchiieva commented on this pull request.
> 
> In 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 :
> 
> > +}
> +
> +out.value = result;
> +  }
> +}
> +  }
> +
> +  /**
> +   * Returns true if the input string is a valid IP address
> +   */
> +  @FunctionTemplate(
> +name = "is_valid_IP",
> +scope = FunctionTemplate.FunctionScope.SIMPLE,
> +nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
> +  )
> +  public static class IsValidIPFunction implements DrillSimpleFunc {
> We exclude commons-logging because Drill uses logback for logging.
> Excluding it from commons-validator should work perfectly fine (I have 
tried).
> 
> 
>   commons-validator
>   commons-validator
>   1.4.0
>   
>   
> commons-logging
> commons-logging
>   
>   
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub 
, or mute the 
thread 
.
> 




---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r144047807
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

Commons-logging

[Charles Givre]

Hello Drill Devs, 
I am working on submitting some UDFs, and in so doing discovered that one UDF 
has a dependency (commons-validator) which itself has a dependency of 
commons-logging which is banned. Does anyone  know why commons-logging is 
banned?   I’m not a maven expert but is it possible to exclude the 
commons-logging dependency somehow so that we could use commons-validator in 
Drill?
Thanks,
— C

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r144037246
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r144032079
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[cgivre]

Github user cgivre commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r144030560
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+

[GitHub] drill issue #973: DRILL-5775: Select * query on a maprdb binary table fails

[vdiravka]

Github user vdiravka commented on the issue:

https://github.com/apache/drill/pull/973
  
The branch is rebased onto the master version and commits are squashed into 
one (except one Paul's commit from 
[PR-968](https://github.com/apache/drill/pull/968).


---

[GitHub] drill issue #973: DRILL-5775: Select * query on a maprdb binary table fails

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/973
  
+1, LGTM.


---

[GitHub] drill pull request #973: DRILL-5775: Select * query on a maprdb binary table...

[vdiravka]

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/973#discussion_r143980497
  
--- Diff: 
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/binary/MapRDBBinaryTable.java
 ---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.mapr.db.binary;
+
+import java.io.IOException;
+
+import org.apache.drill.common.exceptions.UserException;
+import org.apache.drill.exec.store.dfs.FileSystemPlugin;
+import org.apache.drill.exec.store.dfs.FormatSelection;
+import org.apache.drill.exec.store.hbase.AbstractHBaseDrillTable;
+import org.apache.drill.exec.store.mapr.TableFormatPlugin;
+import org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+
+public class MapRDBBinaryTable extends AbstractHBaseDrillTable {
+  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(MapRDBBinaryTable.class);
+
+  public MapRDBBinaryTable(String storageEngineName, FileSystemPlugin 
storagePlugin, TableFormatPlugin formatPlugin,
+  FormatSelection selection) {
+super(storageEngineName, storagePlugin, selection);
+String tableName = selection.getSelection().getFiles().get(0);
--- End diff --

Very good suggestions! Done. Thanks.


---

[GitHub] drill pull request #973: DRILL-5775: Select * query on a maprdb binary table...

[vdiravka]

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/973#discussion_r143971466
  
--- Diff: 
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseGroupScan.java
 ---
@@ -143,9 +143,9 @@ private HBaseGroupScan(HBaseGroupScan that) {
 
   @Override
   public GroupScan clone(List columns) {
+HBaseUtils.verifyColumns(columns, hTableDesc);
--- End diff --

Agree. I've added the same to `BinaryTableGroupScan`.


---

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/958#discussion_r143976358
  
--- Diff: 
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java ---
@@ -22,16 +22,67 @@
 import javax.annotation.concurrent.ThreadSafe;
 
 import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.util.AssertionUtil;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Preconditions;
 
 /**
  * Provides a concurrent way to manage account for memory usage without 
locking. Used as basis for Allocators. All
  * operations are threadsafe (except for close).
  */
 @ThreadSafe
-class Accountant implements AutoCloseable {
-  // private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(Accountant.class);
+@VisibleForTesting
+public class Accountant implements AutoCloseable {
+  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(Accountant.class);
+
+  // See DRILL-5808
+  // Allow a "grace margin" above the allocator limit for those operators
+  // that are trying to stay within the limit, but are hindered by the
+  // current power-of-two allocations and random vector doublings. Instead
+  // of failing the query, the accountant allows an excess allocation 
within
+  // the grace, but issues a warning to the log.
+  //
+  // Normally, the "grace margin" (lenient) allocation is allowed only for
+  // operators that request leniency (because they attempt to manage 
memory)
+  // and only in production mode. Leniency is usually denied in a debug
+  // run (assertions enabled.) Use the following system option to force
+  // allowing lenient allocation in a debug run (or, to disable lenient
+  // allocation in a production run.)
+  //
+  // Force allowing lenient allocation:
+  //
+  // -Ddrill.memory.lenient.allocator=false
+  //
+  // Force strict allocation;
+  //
+  // -Ddrill.memory.lenient.allocator=false
+
+  public static final String ALLOW_LENIENT_ALLOCATION = 
"drill.memory.lenient.allocator";
--- End diff --

`drill.memory.lenient.allocation`?


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r14395
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNetworkFunctions.java
 ---
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.fn.impl;
+
+import org.apache.drill.BaseTestQuery;
+import org.junit.Test;
+
+public class TestNetworkFunctions extends BaseTestQuery {
+
+  @Test
+  public void testInetAton() throws Exception {
+final String query = "select inet_aton( '192.168.0.1') as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues( 
Long.parseLong("3232235521") ).go();
+  }
+
+  @Test
+  public void testInetNtoa() throws Exception {
+final String query = "select inet_ntoa( 3232235521 ) as inet from 
(values(1))";
--- End diff --

spaces


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948692
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143955475
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNetworkFunctions.java
 ---
@@ -0,0 +1,93 @@
+/**
--- End diff --

Apache header should be in a form of comment.


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948412
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
--- End diff --

`BroadcastAddressFunction`


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143949784
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143955360
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143953962
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948382
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143947620
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143947986
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
--- End diff --

`address_count`?


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948743
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948906
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143947900
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
--- End diff --

`broadcast_address`?


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948455
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
--- End diff --

`AddressCountFunction`


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143947835
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143945768
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
--- End diff --

Why you need else if you have already defined result as 0?


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948213
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948634
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143950087
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNetworkFunctions.java
 ---
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.fn.impl;
+
+import org.apache.drill.BaseTestQuery;
+import org.junit.Test;
+
+public class TestNetworkFunctions extends BaseTestQuery {
+
+  @Test
+  public void testInetAton() throws Exception {
+final String query = "select inet_aton( '192.168.0.1') as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues( 
Long.parseLong("3232235521") ).go();
+  }
+
+  @Test
+  public void testInetNtoa() throws Exception {
+final String query = "select inet_ntoa( 3232235521 ) as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues("192.168.0.1").go();
+  }
+
+  @Test
+  public void testInNetwork() throws Exception {
+final String query = "select in_network( '192.168.0.1', 
'192.168.0.0/28' ) as in_net FROM (values(1))";
--- End diff --

What about negative test case?


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143950296
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNetworkFunctions.java
 ---
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.fn.impl;
+
+import org.apache.drill.BaseTestQuery;
+import org.junit.Test;
+
+public class TestNetworkFunctions extends BaseTestQuery {
+
+  @Test
+  public void testInetAton() throws Exception {
+final String query = "select inet_aton( '192.168.0.1') as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues( 
Long.parseLong("3232235521") ).go();
+  }
+
+  @Test
+  public void testInetNtoa() throws Exception {
+final String query = "select inet_ntoa( 3232235521 ) as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues("192.168.0.1").go();
+  }
+
+  @Test
+  public void testInNetwork() throws Exception {
+final String query = "select in_network( '192.168.0.1', 
'192.168.0.0/28' ) as in_net FROM (values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("in_net").baselineValues(true).go();
+  }
+
+  @Test
+  public void testBroadcastAddress() throws Exception {
+final String query = "select getBroadcastAddress( '192.168.0.0/28' ) 
AS broadcastAddress FROM (values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("broadcastAddress").baselineValues("192.168.0.15").go();
+  }
+  @Test
+  public void testNetmask() throws Exception {
+final String query = "select getNetmask( '192.168.0.0/28' ) AS netmask 
FROM (values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("netmask").baselineValues("255.255.255.240").go();
+  }
+  @Test
+  public void testFunctions() throws Exception {
--- End diff --

Well, it is still good to split test into multiple for each function.


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143949930
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNetworkFunctions.java
 ---
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.fn.impl;
+
+import org.apache.drill.BaseTestQuery;
+import org.junit.Test;
+
+public class TestNetworkFunctions extends BaseTestQuery {
+
+  @Test
+  public void testInetAton() throws Exception {
+final String query = "select inet_aton( '192.168.0.1') as inet from 
(values(1))";
+
testBuilder().sqlQuery(query).ordered().baselineColumns("inet").baselineValues( 
Long.parseLong("3232235521") ).go();
--- End diff --

Please remove spaces...


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143946222
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143946390
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143945471
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
--- End diff --

Please mind code style: `if (utils.getInfo().isInRange( ipString)) {`


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143944750
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
--- End diff --

Apache header should be in a form of comment, not Java doc.


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143947725
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143944633
  
--- Diff: exec/java-exec/pom.xml ---
@@ -434,7 +434,11 @@
   joda-time
   2.9
 
-
+
+  commons-net
+  commons-net
+  LATEST
--- End diff --

Please use exact lib version instead of `LATEST`.


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143945974
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
--- End diff --

Can be defined inside of method instead of as workspace variable.


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143944908
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
--- End diff --

space -> `NetworkFunctions {`


---

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143946313
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143948135
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+
+public void eval() {
+
+  String ipString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputIP.start,
 inputIP.end, inputIP.buffer);
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+
+  int result = 0;
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  if( utils.getInfo().isInRange( ipString ) ){
+result = 1;
+  }
+  else{
+result = 0;
+  }
+  out.value = result;
+}
+  }
+
+
+  /**
+   * This function retunrs the number of IP addresses in the input CIDR 
block.
+   */
+  @FunctionTemplate(
+name = "getAddressCount",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getAddressCountFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BigIntHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
+
+public void setup() {
+}
+
+public void eval() {
+
+  String cidrString = 
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(inputCIDR.start,
 inputCIDR.end, inputCIDR.buffer);
+  utils = new org.apache.commons.net.util.SubnetUtils(cidrString);
+
+  out.value = utils.getInfo().getAddressCount();
+
+}
+
+  }
+
+  /**
+   * This function returns the broadcast address of a given CIDR block.
+   */
+  @FunctionTemplate(
+name = "getBroadcastAddress",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class getBroadcastAddressFunction implements 
DrillSimpleFunc {
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+VarCharHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
 

[GitHub] drill pull request #971: Drill-5834 Add Networking Functions

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/971#discussion_r143945315
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/NetworkFunctions.java
 ---
@@ -0,0 +1,668 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import io.netty.buffer.DrillBuf;
+import org.apache.commons.net.util.SubnetUtils;
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.annotations.Workspace;
+import org.apache.drill.exec.expr.holders.BigIntHolder;
+import org.apache.drill.exec.expr.holders.BitHolder;
+import org.apache.drill.exec.expr.holders.VarCharHolder;
+
+import javax.inject.Inject;
+
+public class NetworkFunctions{
+  static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(NetworkFunctions.class);
+
+  private NetworkFunctions() {}
+
+  /**
+   * This function takes two arguments, an input IPv4 and a CIDR, and 
returns true if the IP is in the given CIDR block
+   *
+   */
+  @FunctionTemplate(
+name = "in_network",
+scope = FunctionTemplate.FunctionScope.SIMPLE,
+nulls = FunctionTemplate.NullHandling.NULL_IF_NULL
+  )
+  public static class InNetworkFunction implements DrillSimpleFunc {
+
+@Param
+VarCharHolder inputIP;
+
+@Param
+VarCharHolder inputCIDR;
+
+@Output
+BitHolder out;
+
+@Inject
+DrillBuf buffer;
+
+@Workspace
+SubnetUtils utils;
--- End diff --

There is not need to define `utils` as `Workspace`, it can be defined 
inside of class.


---