[jira] [Created] (DRILL-4794) Regression: Wrong result for query with disjunctive partition filters

[Aman Sinha (JIRA)]

Aman Sinha created DRILL-4794:
-

 Summary: Regression: Wrong result for query with disjunctive 
partition filters
 Key: DRILL-4794
 URL: https://issues.apache.org/jira/browse/DRILL-4794
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 1.7.0
Reporter: Aman Sinha
Assignee: Aman Sinha


For a query that contains certain types of disjunctive filter conditions such 
as 'dir0=x OR dir1=y'  we get wrong result when metadata caching is used.  This 
is a regression due to DRILL-4530.  

Note that the filter involves OR of 2 different directory levels. For the 
normal case of OR condition at the same level the problem does not occur. 

Correct result (without metadata cache) 
{noformat}
0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or 
dir1='Q3' ;
+-+
| EXPR$0  |
+-+
| 60  |
+-+
{noformat}

Wrong result (with metadata cache):
{noformat}
0: jdbc:drill:zk=local> select count(*) from dfs.`orders` where dir0=1994 or 
dir1='Q3' ;
+-+
| EXPR$0  |
+-+
| 50  |
+-+
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386952#comment-15386952
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user chunhui-shi commented on a diff in the pull request:

https://github.com/apache/drill/pull/546#discussion_r71634838
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java
 ---
@@ -129,11 +129,21 @@ private void setFlattenVector() {
 try {
   final TypedFieldId typedFieldId = 
incoming.getValueVectorId(popConfig.getColumn());
   final MaterializedField field = 
incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]);
-  final RepeatedValueVector vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
-  field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  RepeatedValueVector vector = null;
+  try {
+vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
+field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  } catch(ClassCastException ex) {
+if(incoming.getRecordCount() != 0) {
+  throw UserException.unsupportedError(ex).message("Flatten does 
not support inputs of non-list values.").build(logger);
+}
+//when incoming recordCount is 0, don't throw exception since the 
type being seen here is not solid
+logger.error("setFlattenVector cast failed and recordcount is 0, 
create empty vector anyway.");
+vector = new RepeatedMapVector(field, oContext.getAllocator(), 
null);
+  }
   flattener.setFlattenField(vector);
 } catch (Exception ex) {
-  throw UserException.unsupportedError(ex).message("Trying to flatten 
a non-repeated field.").build(logger);
+  throw UserException.unsupportedError(ex).message("Flatten does not 
support inputs of non-list values.").build(logger);
--- End diff --

The two catch blocks are parallel. The UserException in line 138 wont be 
caught and thrown again in line 146.


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386889#comment-15386889
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/546#discussion_r71630375
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java
 ---
@@ -129,11 +129,21 @@ private void setFlattenVector() {
 try {
   final TypedFieldId typedFieldId = 
incoming.getValueVectorId(popConfig.getColumn());
   final MaterializedField field = 
incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]);
-  final RepeatedValueVector vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
-  field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  RepeatedValueVector vector = null;
+  try {
+vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
+field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  } catch(ClassCastException ex) {
+if(incoming.getRecordCount() != 0) {
+  throw UserException.unsupportedError(ex).message("Flatten does 
not support inputs of non-list values.").build(logger);
+}
+//when incoming recordCount is 0, don't throw exception since the 
type being seen here is not solid
+logger.error("setFlattenVector cast failed and recordcount is 0, 
create empty vector anyway.");
+vector = new RepeatedMapVector(field, oContext.getAllocator(), 
null);
+  }
   flattener.setFlattenField(vector);
 } catch (Exception ex) {
-  throw UserException.unsupportedError(ex).message("Trying to flatten 
a non-repeated field.").build(logger);
+  throw UserException.unsupportedError(ex).message("Flatten does not 
support inputs of non-list values.").build(logger);
--- End diff --

UserException may be thrown twice? Line 138 & Line 146. 


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (DRILL-2792) Killing the drillbit which is the foreman results in direct memory being held on

[Padma Penumarthy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Padma Penumarthy reassigned DRILL-2792:
---

Assignee: Padma Penumarthy

> Killing the drillbit which is the foreman results in direct memory being held 
> on
> 
>
> Key: DRILL-2792
> URL: https://issues.apache.org/jira/browse/DRILL-2792
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 0.8.0
>Reporter: Ramana Inukonda Nagaraj
>Assignee: Padma Penumarthy
> Fix For: Future
>
>
> Killed one of the drillbits which is the foreman for the query- 
> Profiles page reports that query has cancelled.
> Due to bug Drill-2778 sqlline hangs. However after killing sqlline the 
> current direct memory used does not go down to pre query levels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386886#comment-15386886
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/546#discussion_r71630181
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java
 ---
@@ -129,11 +129,21 @@ private void setFlattenVector() {
 try {
   final TypedFieldId typedFieldId = 
incoming.getValueVectorId(popConfig.getColumn());
   final MaterializedField field = 
incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]);
-  final RepeatedValueVector vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
-  field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  RepeatedValueVector vector = null;
+  try {
+vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
+field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  } catch(ClassCastException ex) {
--- End diff --

In stead of catch ClassCastException, you may check if the input value 
vector is an instance of RepeatedValueVector.  Using try{} catch{} seems a bit 
over-kill for this case. 



> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386875#comment-15386875
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user jinfengni commented on a diff in the pull request:

https://github.com/apache/drill/pull/546#discussion_r71629104
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/nested/TestFastComplexSchema.java
 ---
@@ -80,4 +80,33 @@ public void test4() throws Exception {
 "   AND r.r_regionkey = 4)) t \n" +
 "ORDER  BY t.f.name");
   }
+
+  @Test //DRILL-4783 when resultset is empty, don't throw exception.
+  public void test5() throws Exception {
+test("alter session set `planner.enable_hashjoin` = false");
--- End diff --

Any reason why you need set these two options? 


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386786#comment-15386786
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user chunhui-shi commented on the issue:

https://github.com/apache/drill/pull/546
  
Thanks @jaltekruse for the review. I updated exception text and added test 
case following your suggestion. 


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386454#comment-15386454
 ] 

ASF GitHub Bot commented on DRILL-4673:
---

Github user vdiravka commented on the issue:

https://github.com/apache/drill/pull/541
  
Agree with class/method names. I was confused with 
`SqlCreateOrReplaceView()`. Thanks Julian.
@sudheeshkatkam Could you review please?


> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command return
> -
>
> Key: DRILL-4673
> URL: https://issues.apache.org/jira/browse/DRILL-4673
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Functions - Drill
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: drill
>
> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command "DROP TABLE" return if table doesn't exist.
> The same for "DROP VIEW IF EXISTS"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386437#comment-15386437
 ] 

ASF GitHub Bot commented on DRILL-4673:
---

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/541#discussion_r71585214
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/ViewHandler.java
 ---
@@ -106,36 +106,43 @@ public PhysicalPlan getPlan(SqlNode sqlNode) throws 
ValidationException, RelConv
 }
   }
 
-  /** Handler for Drop View DDL command. */
-  public static class DropView extends ViewHandler {
-public DropView(SqlHandlerConfig config) {
+  /** Handler for Drop View [If Exists] DDL command. */
+  public static class DropViewIfExists extends ViewHandler {
+public DropViewIfExists(SqlHandlerConfig config) {
   super(config);
 }
 
 @Override
 public PhysicalPlan getPlan(SqlNode sqlNode) throws 
ValidationException, RelConversionException, IOException, ForemanSetupException 
{
-  SqlDropView dropView = unwrap(sqlNode, SqlDropView.class);
-  final String viewToDrop = dropView.getName();
+  SqlDropViewIfExists dropView = unwrap(sqlNode, 
SqlDropViewIfExists.class);
+  final String viewName = dropView.getName();
   final AbstractSchema drillSchema =
   
SchemaUtilites.resolveToMutableDrillSchema(context.getNewDefaultSchema(), 
dropView.getSchemaPath());
 
   final String schemaPath = drillSchema.getFullSchemaName();
 
-  final Table existingTable = 
SqlHandlerUtil.getTableFromSchema(drillSchema, viewToDrop);
-  if (existingTable != null && existingTable.getJdbcTableType() != 
Schema.TableType.VIEW) {
-throw UserException.validationError()
-.message("[%s] is not a VIEW in schema [%s]", viewToDrop, 
schemaPath)
-.build(logger);
-  } else if (existingTable == null) {
-throw UserException.validationError()
-.message("Unknown view [%s] in schema [%s].", viewToDrop, 
schemaPath)
-.build(logger);
+  final Table viewToDrop = 
SqlHandlerUtil.getTableFromSchema(drillSchema, viewName);
+  if (dropView.checkViewExistence()) {
+if (viewToDrop == null || viewToDrop.getJdbcTableType() != 
Schema.TableType.VIEW){
--- End diff --

Answered above.


> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command return
> -
>
> Key: DRILL-4673
> URL: https://issues.apache.org/jira/browse/DRILL-4673
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Functions - Drill
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: drill
>
> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command "DROP TABLE" return if table doesn't exist.
> The same for "DROP VIEW IF EXISTS"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386433#comment-15386433
 ] 

ASF GitHub Bot commented on DRILL-4673:
---

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/541#discussion_r71585003
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableIfExistsHandler.java
 ---
@@ -28,47 +30,57 @@
 import org.apache.drill.exec.physical.PhysicalPlan;
 import org.apache.drill.exec.planner.sql.DirectPlan;
 import org.apache.drill.exec.planner.sql.SchemaUtilites;
-import org.apache.drill.exec.planner.sql.parser.SqlDropTable;
+import org.apache.drill.exec.planner.sql.parser.SqlDropTableIfExists;
 import org.apache.drill.exec.store.AbstractSchema;
 
 // SqlHandler for dropping a table.
-public class DropTableHandler extends DefaultSqlHandler {
+public class DropTableIfExistsHandler extends DefaultSqlHandler {
 
-  private static org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(DropTableHandler.class);
+  private static org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(DropTableIfExistsHandler.class);
 
-  public DropTableHandler(SqlHandlerConfig config) {
+  public DropTableIfExistsHandler(SqlHandlerConfig config) {
 super(config);
   }
 
   /**
-   * Function resolves the schema and invokes the drop method. Raises an 
exception if the schema is
-   * immutable.
-   * @param sqlNode - Table name identifier
-   * @return - Single row indicating drop succeeded, raise exception 
otherwise
+   * Function resolves the schema and invokes the drop method
+   * (while IF EXISTS statement is used function invokes the drop method 
only if table exists).
+   * Raises an exception if the schema is immutable.
+   * @param sqlNode - SqlDropTableIfExists (SQL parse tree of drop table 
[if exists] query)
+   * @return - Single row indicating drop succeeded or table is not found 
while IF EXISTS statement is used,
+   * raise exception otherwise
* @throws ValidationException
* @throws RelConversionException
* @throws IOException
*/
   @Override
   public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, 
RelConversionException, IOException {
 
-SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-SqlIdentifier tableIdentifier = dropTableNode.getTableIdentifier();
+SqlDropTableIfExists dropTableIfExistsNode = ((SqlDropTableIfExists) 
sqlNode);
+SqlIdentifier tableIdentifier = 
dropTableIfExistsNode.getTableIdentifier();
 
 SchemaPlus defaultSchema = config.getConverter().getDefaultSchema();
 AbstractSchema drillSchema = null;
 
 if (tableIdentifier != null) {
-  drillSchema = 
SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, 
dropTableNode.getSchema());
+  drillSchema = 
SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, 
dropTableIfExistsNode.getSchema());
 }
 
-String tableName = ((SqlDropTable) sqlNode).getName();
+String tableName = dropTableIfExistsNode.getName();
 if (drillSchema == null) {
   throw UserException.validationError()
   .message("Invalid table_name [%s]", tableName)
   .build(logger);
 }
 
+if (dropTableIfExistsNode.checkTableExistence()) {
+  final Table tableToDrop = 
SqlHandlerUtil.getTableFromSchema(drillSchema, tableName);
+  if (tableToDrop == null || tableToDrop.getJdbcTableType() != 
Schema.TableType.TABLE) {
--- End diff --

I checked such query `drop table if exists hbase.TestTableNullStr` and 
found that the message is proper and same like for `DROP TABLE` statement: 
`PARSE ERROR: Unable to create or drop tables/views. Schema [hbase] is 
immutable.`.


> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command return
> -
>
> Key: DRILL-4673
> URL: https://issues.apache.org/jira/browse/DRILL-4673
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Functions - Drill
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: drill
>
> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command "DROP TABLE" return if table doesn't exist.
> The same for "DROP VIEW IF EXISTS"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4673) Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386413#comment-15386413
 ] 

ASF GitHub Bot commented on DRILL-4673:
---

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/541#discussion_r71583345
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableIfExistsHandler.java
 ---
@@ -28,47 +30,57 @@
 import org.apache.drill.exec.physical.PhysicalPlan;
 import org.apache.drill.exec.planner.sql.DirectPlan;
 import org.apache.drill.exec.planner.sql.SchemaUtilites;
-import org.apache.drill.exec.planner.sql.parser.SqlDropTable;
+import org.apache.drill.exec.planner.sql.parser.SqlDropTableIfExists;
 import org.apache.drill.exec.store.AbstractSchema;
 
 // SqlHandler for dropping a table.
-public class DropTableHandler extends DefaultSqlHandler {
+public class DropTableIfExistsHandler extends DefaultSqlHandler {
 
-  private static org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(DropTableHandler.class);
+  private static org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(DropTableIfExistsHandler.class);
 
-  public DropTableHandler(SqlHandlerConfig config) {
+  public DropTableIfExistsHandler(SqlHandlerConfig config) {
 super(config);
   }
 
   /**
-   * Function resolves the schema and invokes the drop method. Raises an 
exception if the schema is
-   * immutable.
-   * @param sqlNode - Table name identifier
-   * @return - Single row indicating drop succeeded, raise exception 
otherwise
+   * Function resolves the schema and invokes the drop method
+   * (while IF EXISTS statement is used function invokes the drop method 
only if table exists).
+   * Raises an exception if the schema is immutable.
+   * @param sqlNode - SqlDropTableIfExists (SQL parse tree of drop table 
[if exists] query)
+   * @return - Single row indicating drop succeeded or table is not found 
while IF EXISTS statement is used,
+   * raise exception otherwise
* @throws ValidationException
* @throws RelConversionException
* @throws IOException
*/
   @Override
   public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, 
RelConversionException, IOException {
 
-SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-SqlIdentifier tableIdentifier = dropTableNode.getTableIdentifier();
+SqlDropTableIfExists dropTableIfExistsNode = ((SqlDropTableIfExists) 
sqlNode);
+SqlIdentifier tableIdentifier = 
dropTableIfExistsNode.getTableIdentifier();
 
 SchemaPlus defaultSchema = config.getConverter().getDefaultSchema();
 AbstractSchema drillSchema = null;
 
 if (tableIdentifier != null) {
-  drillSchema = 
SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, 
dropTableNode.getSchema());
+  drillSchema = 
SchemaUtilites.resolveToMutableDrillSchema(defaultSchema, 
dropTableIfExistsNode.getSchema());
 }
 
-String tableName = ((SqlDropTable) sqlNode).getName();
+String tableName = dropTableIfExistsNode.getName();
 if (drillSchema == null) {
   throw UserException.validationError()
   .message("Invalid table_name [%s]", tableName)
   .build(logger);
 }
 
+if (dropTableIfExistsNode.checkTableExistence()) {
+  final Table tableToDrop = 
SqlHandlerUtil.getTableFromSchema(drillSchema, tableName);
+  if (tableToDrop == null || tableToDrop.getJdbcTableType() != 
Schema.TableType.TABLE) {
--- End diff --

I checked every `schema#getTable` and found that every method can throw 
exception only if something go wrong (permissionError, dataReadError, 
IOException etc). 
`getTableFromSchema `can not throw any exception, because in this method 
`catch (Exception e)` is used.


> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command return
> -
>
> Key: DRILL-4673
> URL: https://issues.apache.org/jira/browse/DRILL-4673
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Functions - Drill
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Minor
>  Labels: drill
>
> Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on 
> command "DROP TABLE" return if table doesn't exist.
> The same for "DROP VIEW IF EXISTS"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386238#comment-15386238
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user jaltekruse commented on the issue:

https://github.com/apache/drill/pull/546
  
The original behavior was not by design, it was a bug. I would just detect 
the case where a scalar is being flattened and throw an appropriate user facing 
message like "Flatten does not support inputs of non-list values".

The tests you have included here don't check their results. I would also 
suggest adding a test for the new negative case, checking for the appropriate 
message.

If you want to more precisely control the input to your test, I would 
recommend taking a look at the operator unit test system I checked in and 
demonstrated at the hangout last month. This will allow you to write more 
targeted tests for this case. Feel free to ask questions if the usage is not 
clear from the examples in this class.


https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/physical/unit/BasicPhysicalOpUnitTest.java


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4793) integration oozie with drill--using oozie to support the activity for executing the drill queries

[Kathleen Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kathleen Li updated DRILL-4793:
---
Issue Type: New Feature  (was: Bug)

> integration oozie with drill--using oozie to support the activity for 
> executing the drill queries
> -
>
> Key: DRILL-4793
> URL: https://issues.apache.org/jira/browse/DRILL-4793
> Project: Apache Drill
>  Issue Type: New Feature
>  Components: Functions - Drill
>Affects Versions: 1.6.0
>Reporter: Kathleen Li
>Priority: Minor
>
>  Using Oozie for chaining the ETL jobs and want to know if  we can make Oozie 
> support an activity for executing the drill queries like hive queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4793) integration oozie with drill--using oozie to support the activity for executing the drill queries

[Kathleen Li (JIRA)]

Kathleen Li created DRILL-4793:
--

 Summary: integration oozie with drill--using oozie to support the 
activity for executing the drill queries
 Key: DRILL-4793
 URL: https://issues.apache.org/jira/browse/DRILL-4793
 Project: Apache Drill
  Issue Type: Bug
  Components: Functions - Drill
Affects Versions: 1.6.0
Reporter: Kathleen Li
Priority: Minor


 Using Oozie for chaining the ETL jobs and want to know if  we can make Oozie 
support an activity for executing the drill queries like hive queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (DRILL-4785) Limit 0 queries regressed in Drill 1.7.0

[Dechang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dechang Gu closed DRILL-4785.
-

verified with limit 0 queries.  LGTM.

> Limit 0 queries regressed in Drill 1.7.0 
> -
>
> Key: DRILL-4785
> URL: https://issues.apache.org/jira/browse/DRILL-4785
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.7.0
> Environment: Redhat EL6
>Reporter: Dechang Gu
>Assignee: Venki Korukanti
> Fix For: 1.8.0
>
>
> We noticed a bunch of limit 0 queries regressed quite a bit: +2500ms, while 
> the same queries took  ~400ms in Apache Drill 1.6.0. 5-6X regression. Further 
> investigation indicates that most likely the root cause of the regression is 
> in the commit: 
> vkorukanti committed with vkorukanti DRILL-4446: Support mandatory work 
> assignment to endpoint requirement…
> commit id:  10afc708600ea9f4cb0e7c2cd981b5b1001fea0d
> With drill build on this commit, query takes 3095ms
> and in the drillbit.log:
> 2016-07-15 17:27:55,048 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO  
> o.a.drill.exec.work.foreman.Foreman - Query text for query id 
> 28768074-4ed6-a70a-2e6a-add3201ab801: SELECT * FROM (SELECT 
> CAST(EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) AS 
> INTEGER) AS `mn_business_date_ok`,AVG((CASE WHEN ((CAST(EXTRACT(YEAR FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND 
> (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 
> + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS 
> `avg_Calculation_CIDBACJBCCCBHDGB_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND 
> (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 
> + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS 
> `sum_Calculation_CIDBACJBCCCBHDGB_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND 
> (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 
> + 1 AS INTEGER) <= 4)) THEN 1 ELSE NULL END)) AS 
> `sum_Calculation_CJEBBAEBBFADBDFJ_ok`,SUM((CASE WHEN ((CAST(EXTRACT(YEAR FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND 
> (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 
> + 1 AS INTEGER) <= 4)) THEN (`rfm_sales`.`pos_comps` + 
> `rfm_sales`.`pos_promos`) ELSE NULL END)) AS `sum_Net_Sales__YTD___copy__ok` 
> FROM `dfs.xxx`.`views/rfm_sales` `rfm_sales` GROUP BY CAST(EXTRACT(MONTH FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER)) T LIMIT 0
> 2016-07-15 17:27:55,664 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO  
> o.a.d.exec.store.parquet.Metadata - Took 208 ms to read metadata from cache 
> file
> 2016-07-15 17:27:56,783 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:foreman] INFO  
> o.a.d.exec.store.parquet.Metadata - Took 129 ms to read metadata from cache 
> file
> 2016-07-15 17:27:57,960 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO  
> o.a.d.e.w.fragment.FragmentExecutor - 
> 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State change requested 
> AWAITING_ALLOCATION --> RUNNING
> 2016-07-15 17:27:57,961 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO  
> o.a.d.e.w.f.FragmentStatusReporter - 
> 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State to report: RUNNING
> 2016-07-15 17:27:57,989 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO  
> o.a.d.e.w.fragment.FragmentExecutor - 
> 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State change requested RUNNING --> 
> FINISHED
> 2016-07-15 17:27:57,989 ucs-node2.perf.lab 
> [28768074-4ed6-a70a-2e6a-add3201ab801:frag:0:0] INFO  
> o.a.d.e.w.f.FragmentStatusReporter - 
> 28768074-4ed6-a70a-2e6a-add3201ab801:0:0: State to report: FINISHED
> while running the same query on the parent commit (commit id 
> 9f4fff800d128878094ae70b454201f79976135d), it only takes  492ms.
> and in the drillbit.log:
> 2016-07-15 17:19:27,309 ucs-node7.perf.lab 
> [2876826f-ee19-9466-0c0c-869f47c409f8:foreman] INFO  
> o.a.drill.exec.work.foreman.Foreman - Query text for query id 
> 2876826f-ee19-9466-0c0c-869f47c409f8: SELECT * FROM (SELECT 
> CAST(EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) AS 
> INTEGER) AS `mn_business_date_ok`,AVG((CASE WHEN ((CAST(EXTRACT(YEAR FROM 
> CAST(`rfm_sales`.`business_date` AS DATE)) AS INTEGER) = 2014) AND 
> (CAST((EXTRACT(MONTH FROM CAST(`rfm_sales`.`business_date` AS DATE)) - 1) / 3 
> + 1 AS INTEGER) <= 4)) THEN `rfm_sales`.`pos_netsales` ELSE NULL END)) AS 
> 

[jira] [Commented] (DRILL-4682) Allow full schema identifier in SELECT clause

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386021#comment-15386021
 ] 

ASF GitHub Bot commented on DRILL-4682:
---

GitHub user vdiravka opened a pull request:

https://github.com/apache/drill/pull/549

DRILL-4682: Allow full schema identifier in SELECT clause

 - changed SqlBracketlessSyntax logic in the DrillCompoundIdentifier;
 - added checking of using full schema name in column names;
 - added unit test testColumnNamesWithSchemaName;

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vdiravka/drill DRILL-4682

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/549.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #549


commit 0e5123c80e0b7d2dca36503dab3917bbae827702
Author: Vitalii Diravka 
Date:   2016-06-13T11:44:33Z

DRILL-4682: Allow full schema identifier in SELECT clause
 - changed SqlBracketlessSyntax logic in the DrillCompoundIdentifier;
 - added checking of using full schema name in column names;
 - added unit test testColumnNamesWithSchemaName;




> Allow full schema identifier in SELECT clause
> -
>
> Key: DRILL-4682
> URL: https://issues.apache.org/jira/browse/DRILL-4682
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: SQL Parser
>Reporter: Andries Engelbrecht
>
> Currently Drill requires aliases to identify columns in the SELECT clause 
> when working with multiple tables/workspaces.
> Many BI/Analytical and other tools by default will use the full schema 
> identifier in the select clause when generating SQL statements for execution 
> for generic JDBC or ODBC sources. Not supporting this feature causes issues 
> and a slower adoption of utilizing Drill as an execution engine within the 
> larger Analytical SQL community.
> Propose to support 
> SELECT ... FROM 
> ..
> Also see DRILL-3510 for double quote support as per ANSI_QUOTES
> SELECT ""."".""."" FROM 
> ""."".""
> Which is very common generic SQL being generated by most tools when dealing 
> with a generic SQL data source.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4792) Include session options used for a query as part of the profile

[Arina Ielchiieva (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385976#comment-15385976
 ] 

Arina Ielchiieva commented on DRILL-4792:
-

Session options effective at the time query was run will be displayed in query 
profile.
Appropriate block will appear after Query Profile information (screenshot - 
session_options_block.JPG).
User may collapse Session Options block (screenshot - 
session_options_collapsed.JPG).
Also appropriate information can be seen in json profile (screenshot - 
session_options_json.JPG).
If there were no session options set at the time query was run, Session Options 
block won't appear (screenshot - no_session_options.JPG).

> Include session options used for a query as part of the profile
> ---
>
> Key: DRILL-4792
> URL: https://issues.apache.org/jira/browse/DRILL-4792
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.7.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.8.0
>
>
> Include session options used for a query as part of the profile.
> This will be very useful for debugging/diagnostics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-4792) Include session options used for a query as part of the profile

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva updated DRILL-4792:

Attachment: session_options_json.JPG
session_options_collapsed.JPG
session_options_block.JPG
no_session_options.JPG

> Include session options used for a query as part of the profile
> ---
>
> Key: DRILL-4792
> URL: https://issues.apache.org/jira/browse/DRILL-4792
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.7.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
>Priority: Minor
> Fix For: 1.8.0
>
> Attachments: no_session_options.JPG, session_options_block.JPG, 
> session_options_collapsed.JPG, session_options_json.JPG
>
>
> Include session options used for a query as part of the profile.
> This will be very useful for debugging/diagnostics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4792) Include session options used for a query as part of the profile

[Arina Ielchiieva (JIRA)]

Arina Ielchiieva created DRILL-4792:
---

 Summary: Include session options used for a query as part of the 
profile
 Key: DRILL-4792
 URL: https://issues.apache.org/jira/browse/DRILL-4792
 Project: Apache Drill
  Issue Type: Improvement
Affects Versions: 1.7.0
Reporter: Arina Ielchiieva
Assignee: Arina Ielchiieva
Priority: Minor
 Fix For: 1.8.0


Include session options used for a query as part of the profile.
This will be very useful for debugging/diagnostics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4783) Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385426#comment-15385426
 ] 

ASF GitHub Bot commented on DRILL-4783:
---

Github user chunhui-shi commented on a diff in the pull request:

https://github.com/apache/drill/pull/546#discussion_r71471032
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java
 ---
@@ -129,8 +129,14 @@ private void setFlattenVector() {
 try {
   final TypedFieldId typedFieldId = 
incoming.getValueVectorId(popConfig.getColumn());
   final MaterializedField field = 
incoming.getSchema().getColumn(typedFieldId.getFieldIds()[0]);
-  final RepeatedValueVector vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
-  field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  RepeatedValueVector vector = null;
+  try {
+vector = 
RepeatedValueVector.class.cast(incoming.getValueAccessorById(
+field.getValueClass(), 
typedFieldId.getFieldIds()).getValueVector());
+  } catch(ClassCastException ex) {
+//if cast fail
--- End diff --

Thanks Jinfeng for the review. The fix will return empty result if it is 
flatten(100).

Since this fix is to address the issue when there is no incoming record, 
flatten above convert_from should not throw exception. Meanwhile it is better 
to keep other behavior unchanged unless we have strong reason to change it. 
Based on this understanding, I am updating the change to still throw the same 
cast exception if flatten is not getting a RepeatedValueVector to work on.


> Flatten on CONVERT_FROM fails with ClassCastException if resultset is empty
> ---
>
> Key: DRILL-4783
> URL: https://issues.apache.org/jira/browse/DRILL-4783
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Flatten failed to work on top of convert_from when the resultset is empty. 
> For a HBase table like this:
> 0: jdbc:drill:zk=localhost:5181> select convert_from(t.address.cities,'json') 
> from hbase.`/tmp/flattentest` t;
> +--+
> |  EXPR$0 
>  |
> +--+
> | {"list":[{"city":"SunnyVale"},{"city":"Palo Alto"},{"city":"Mountain 
> View"}]}|
> | {"list":[{"city":"Seattle"},{"city":"Bellevue"},{"city":"Renton"}]} 
>  |
> | {"list":[{"city":"Minneapolis"},{"city":"Falcon Heights"},{"city":"San 
> Paul"}]}  |
> +--+
> Flatten works when row_key is in (1,2,3)
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=1) t1;
> +---+
> |  EXPR$0   |
> +---+
> | {"city":"SunnyVale"}  |
> | {"city":"Palo Alto"}  |
> | {"city":"Mountain View"}  |
> +---+
> But Flatten throws exception if the resultset is empty
> 0: jdbc:drill:zk=localhost:5181> select flatten(t1.json.list) from (select 
> convert_from(t.address.cities,'json') json from hbase.`/tmp/flattentest` t 
> where row_key=4) t1;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast 
> org.apache.drill.exec.vector.NullableIntVector to 
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 07fd0cab-d1e6-4259-bfec-ad80f02d93a2 on atsqa4-127.qa.lab:31010] 
> (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4791) Provide a light-weight, versioned client API

[Paul Rogers (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385335#comment-15385335
 ] 

Paul Rogers edited comment on DRILL-4791 at 7/20/16 6:02 AM:
-

Absolutely. Nevertheless, please see the attached proposal for why a "light 
weight" and "versioned" API may be helpful for client apps. The proposal 
suggests a revised JDBC interface on top of the light-weight client. By 
light-weight, here I mean a new JDBC that is 1% the size of the current one, 
and does not depend on ZK.

Plus, a new low-level protocol that is easy to implement in C, Python or 
anything that supports plain old sockets.


was (Author: paul-rogers):
Absolutely. Nevertheless, please see the attached proposal for why a "light 
weight" and "versioned" API may be helpful for client apps. The proposal 
suggests a revised JDBC interface on top of the light-weight client.

> Provide a light-weight, versioned client API
> 
>
> Key: DRILL-4791
> URL: https://issues.apache.org/jira/browse/DRILL-4791
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>
> Drill's existing client APIs are "industrial strength" - they provide full 
> access to the sophisticated distributed, columnar RPCs which Drill uses 
> internall. However, they are too complex for most client needs. Provide a 
> simpler API optimized for clients: row-based result sets, synchronous, etc.
> At the same time, Drill clients must currently link with the same version of 
> Drill code as is running on the Drill cluster. This forces clients to upgrade 
> in lock-step with the cluster. Allow Drill clients to be upgraded after (or 
> even before) the Drill cluster to simplify management of desktop apps that 
> use Drill.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)