[jira] [Commented] (DRILL-3474) Add implicit file columns support

2017-03-31 Thread Bridget Bevens (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951778#comment-15951778
 ] 

Bridget Bevens commented on DRILL-3474:
---

Link to doc: 
http://drill.apache.org/docs/querying-a-file-system-introduction/#implicit-columns
 

> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-06-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350706#comment-15350706
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/491
  
Changes merged into master with commit id 
1ddd0ce7186afd164052d6aa8f63a75fd01148f4


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-06-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350707#comment-15350707
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user arina-ielchiieva closed the pull request at:

https://github.com/apache/drill/pull/491


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-06-27 Thread Arina Ielchiieva (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350688#comment-15350688
 ] 

Arina Ielchiieva commented on DRILL-3474:
-

Merged  with commit id 1ddd0ce7186afd164052d6aa8f63a75fd01148f4.
Bug fixed in Jira DRILL-4733 with commit id 
19cdcd95808ddb6f9e19bbccb753017993ad78d6.

> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
>  Labels: doc-impacting
> Fix For: 1.7.0
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280365#comment-15280365
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62879671
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
 ---
@@ -324,6 +324,11 @@ protected boolean setupNewSchema() throws 
SchemaChangeException {
   if (name == EMPTY_STRING) {
 continue;
   }
+
--- End diff --

In current implementation dirN columns will be returned in select all 
clause during querying directory as before. 


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280367#comment-15280367
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62879684
  
--- Diff: 
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveDrillNativeScanBatchCreator.java
 ---
@@ -67,17 +67,15 @@ public ScanBatch getBatch(FragmentContext context, 
HiveDrillNativeParquetSubScan
 final List columns = config.getColumns();
 final String partitionDesignator = context.getOptions()
 
.getOption(ExecConstants.FILESYSTEM_PARTITION_COLUMN_LABEL).string_val;
+List> virtualColumns = Lists.newLinkedList();
--- End diff --

Renamed.


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280360#comment-15280360
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62879209
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
@@ -148,6 +148,18 @@
   String FILESYSTEM_PARTITION_COLUMN_LABEL = 
"drill.exec.storage.file.partition.column.label";
   OptionValidator FILESYSTEM_PARTITION_COLUMN_LABEL_VALIDATOR = new 
StringValidator(FILESYSTEM_PARTITION_COLUMN_LABEL, "dir");
 
+  /**
+   * Implicit file columns
+   */
+  String IMPLICIT_FILENAME_COLUMN_LABEL = 
"drill.exec.storage.implicit.filename.column.label";
+  OptionValidator IMPLICIT_FILENAME_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_FILENAME_COLUMN_LABEL, "filename");
+  String IMPLICIT_SUFFIX_COLUMN_LABEL = 
"drill.exec.storage.implicit.suffix.column.label";
+  OptionValidator IMPLICIT_SUFFIX_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_SUFFIX_COLUMN_LABEL, "suffix");
+  String IMPLICIT_FQN_COLUMN_LABEL = 
"drill.exec.storage.implicit.fqn.column.label";
+  OptionValidator IMPLICIT_FQN_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_FQN_COLUMN_LABEL, "fqn");
+  String IMPLICIT_DIRNAME_COLUMN_LABEL = 
"drill.exec.storage.implicit.dirname.column.label";
--- End diff --

Renamed to FILEPATH.


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279568#comment-15279568
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user parthchandra commented on the pull request:

https://github.com/apache/drill/pull/491#issuecomment-218364635
  
Minor comments. Otherwise looks good. +1
Note the doc impact.


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279564#comment-15279564
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62793098
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
 ---
@@ -324,6 +324,11 @@ protected boolean setupNewSchema() throws 
SchemaChangeException {
   if (name == EMPTY_STRING) {
 continue;
   }
+
--- End diff --

This has a major documentation impact since there is a change in the DIR 
columns returned in a star query. Please make a note of this in the JIRA.


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279563#comment-15279563
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62793024
  
--- Diff: 
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveDrillNativeScanBatchCreator.java
 ---
@@ -67,17 +67,15 @@ public ScanBatch getBatch(FragmentContext context, 
HiveDrillNativeParquetSubScan
 final List columns = config.getColumns();
 final String partitionDesignator = context.getOptions()
 
.getOption(ExecConstants.FILESYSTEM_PARTITION_COLUMN_LABEL).string_val;
+List> virtualColumns = Lists.newLinkedList();
--- End diff --

Name these implicitColumns instead of virtualColumns to keep the naming 
consistent? 


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279562#comment-15279562
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/491#discussion_r62792998
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
@@ -148,6 +148,18 @@
   String FILESYSTEM_PARTITION_COLUMN_LABEL = 
"drill.exec.storage.file.partition.column.label";
   OptionValidator FILESYSTEM_PARTITION_COLUMN_LABEL_VALIDATOR = new 
StringValidator(FILESYSTEM_PARTITION_COLUMN_LABEL, "dir");
 
+  /**
+   * Implicit file columns
+   */
+  String IMPLICIT_FILENAME_COLUMN_LABEL = 
"drill.exec.storage.implicit.filename.column.label";
+  OptionValidator IMPLICIT_FILENAME_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_FILENAME_COLUMN_LABEL, "filename");
+  String IMPLICIT_SUFFIX_COLUMN_LABEL = 
"drill.exec.storage.implicit.suffix.column.label";
+  OptionValidator IMPLICIT_SUFFIX_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_SUFFIX_COLUMN_LABEL, "suffix");
+  String IMPLICIT_FQN_COLUMN_LABEL = 
"drill.exec.storage.implicit.fqn.column.label";
+  OptionValidator IMPLICIT_FQN_COLUMN_LABEL_VALIDATOR = new 
StringValidator(IMPLICIT_FQN_COLUMN_LABEL, "fqn");
+  String IMPLICIT_DIRNAME_COLUMN_LABEL = 
"drill.exec.storage.implicit.dirname.column.label";
--- End diff --

Probably better to name this PATH or FILEPATH instead of DIRNAME to prevent 
confusion with DIR[0-9]


> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-05-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266717#comment-15266717
 ] 

ASF GitHub Bot commented on DRILL-3474:
---

GitHub user arina-ielchiieva opened a pull request:

https://github.com/apache/drill/pull/491

DRILL-3474: Add implicit file columns support



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arina-ielchiieva/drill DRILL-3474

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/491.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #491


commit aff44f3fafa35d946115899f9e9941e2b2af22d6
Author: Arina Ielchiieva 
Date:   2016-04-18T16:36:52Z

DRILL-3474: Add implicit file columns support




> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3474) Add implicit file columns support

2016-04-29 Thread Arina Ielchiieva (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264048#comment-15264048
 ] 

Arina Ielchiieva commented on DRILL-3474:
-

Implementation will add support to four implicit file columns: filename, 
suffix, dfqn, dirname.
They will be available during querying file or directory if called explicitly.

> Add implicit file columns support
> -
>
> Key: DRILL-3474
> URL: https://issues.apache.org/jira/browse/DRILL-3474
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: 1.1.0
>Reporter: Jim Scott
>Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> I could not find another ticket which talks about this ...
> The file name should be a column which can be selected or filtered when 
> querying a directory just like dir0, dir1 are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)