[GitHub] drill issue #1059: DRILL-5851: Empty table during a join operation with a no...

[HanumathRao]

Github user HanumathRao commented on the issue:

https://github.com/apache/drill/pull/1059
  
@paul-rogers  Thanks for the review. I have moved the testcase to one test 
file and called it from multiple places. I have also made changes to move the 
code in checkForEarlyExit to base class. Please let me know if any other 
changes are required. thanks.


---

[GitHub] drill pull request #1078: DRILL-6054: don't try to split the filter when it ...

[chunhui-shi]

GitHub user chunhui-shi opened a pull request:

https://github.com/apache/drill/pull/1078

DRILL-6054: don't try to split the filter when it is not AND



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chunhui-shi/drill work1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1078.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1078


commit e44ed46471317f43c494a497551e6546016f3a10
Author: chunhui-shi 
Date:   2017-12-22T23:42:27Z

DRILL-6054: don't try to split the filter when it is not AND




---

Re: Implementing inner joins for mongo

[Chunhui Shi]

Hi Dennis,


I would suggest you to look at the logical plan of this query will be 
generating by running this sql statement 'explain plan without implementation 
for https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/JoinPushThroughJoinRule.java


Hope this helps,


Chunhui


From: Dennis Knochenwefel 
Sent: Thursday, December 28, 2017 4:48:29 AM
To: dev@drill.apache.org
Subject: Implementing inner joins for mongo

Hello Drill Dev Pros,

I have found the drill mongo store and would like to extend it to push
down INNER JOINs. Therefore I would like to rewrite INNER JOINs into the
mongo aggregation pipeline. Here is a SQL example:

SELECT *
FROM `mymongo.db`.`facts` `facts`
   INNER JOIN `mymongo.db`.`set` `set` ON (`facts`.`group` = `set`.`group`)
WHERE ((`set`.`date` = '09.05.2017') AND (`set`.`id` = '1'))

Could you give me a hint how to do that? I am familiar with the
aggregation pipeline of mongo, but am not sure how to implement the
rewrite. I have found the push down of WHERE clauses for mongo [1]

But I am still struggling to do the same for inner joins. If I implement
"public class MongoPushDownInnerJoinScan extends
StoragePluginOptimizerRule" then how would the constructor look like.
Which equivalent of MongoGroupScan (AbstractGroupScan) [2] would I have
to implement? Any help would be very much appreciated.

Thank you and kind regards,

Dennis


[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_blob_master_contrib_storage-2Dmongo_src_main_java_org_apache_drill_exec_store_mongo_MongoPushDownFilterForScan.java=DwIDaQ=cskdkSMqhcnjZxdQVpwTXg=FCGQb-L4gJ1XbsL1WU2sugDtPvzIxWFzAi5u4TTtxaI=AgtqT-QgqWrLpsLSl6rmMe7wlrIS_ZxLsgbN53Dw9UE=deXYtgUqGSZXm13dZpenqfHCMpb_H6fL6DVcSHXTMQs=

[2]
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_blob_master_contrib_storage-2Dmongo_src_main_java_org_apache_drill_exec_store_mongo_MongoGroupScan.java=DwIDaQ=cskdkSMqhcnjZxdQVpwTXg=FCGQb-L4gJ1XbsL1WU2sugDtPvzIxWFzAi5u4TTtxaI=AgtqT-QgqWrLpsLSl6rmMe7wlrIS_ZxLsgbN53Dw9UE=zuccYRJCUAMkxZwbmaDUWZoUBdvXf8jvkQGQcS3qfZE=

--
Dennis Knochenwefel
Founder
Reportix
Germany

[GitHub] drill issue #1076: DRILL-6036: Create sys.connections table

[sohami]

Github user sohami commented on the issue:

https://github.com/apache/drill/pull/1076
  
Filtering should be done based on impersonated user which will be stored in 
UserSession as well. So your check 
[here](https://github.com/kkhatua/drill/blob/66ab15df7e0df6804b1938adc048728c7436450b/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java#L81)
 is fine.

But we shouldn't display the [target 
User](https://github.com/kkhatua/drill/blob/66ab15df7e0df6804b1938adc048728c7436450b/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java#L123)
 separately in the table. Just displaying the session user is fine which will 
be same as querying user. Because when we do `session.getTargetUserName()`  it 
get's the target name which was passed in Connection URL during connection 
time. But during query time the actual targeted user or the session user can 
change.




---

[jira] [Created] (DRILL-6063) Set correct ThreadContext ClassLoader before using Hadoop Configuration class in DrillClient

[Sorabh Hamirwasia (JIRA)]

Sorabh Hamirwasia created DRILL-6063:


 Summary: Set correct ThreadContext ClassLoader before using Hadoop 
Configuration class in DrillClient
 Key: DRILL-6063
 URL: https://issues.apache.org/jira/browse/DRILL-6063
 Project: Apache Drill
  Issue Type: Improvement
  Components: Client - Java
Affects Versions: 1.11.0
Reporter: Sorabh Hamirwasia
Assignee: Sorabh Hamirwasia


Drill Client uses Hadoop Configuration class in security mechanism factories. 
Configuration class internally uses ThreadContext class loader to get the 
default resources for configurations. When Drill JDBC driver is used with 3rd 
party tool like Squirrel then it creates a class loader with all Drill Driver 
related dependencies. This class loader is used to instantiate an object of 
Driver class in a separate thread. But the thread context class loader has 
dependencies related to Squirrel only. As a result of this when Hadoop 
Configuration class tries to load the resources with default values it doesn't 
find them in class path.

If we specify hadoop dependencies in Squirrel's classpath instead of Driver 
dependencies class path then it will work since Thread Context class loader 
will have Hadoop libraries loaded too and will find the resources. But this is 
not good as it is tying up one Squirrel instance with one version of Hadoop 
which might not be true in all the cases like here when same Squirrel instance 
can be used to load multiple versions of Drill Driver having different hadoop 
version dependencies.

To deal with this issue a simple fix is to set the Thread Context class loader 
to be same as Drill Driver dependencies class loader before accessing Hadoop 
Configuration class in Drill Client and later restore it back.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill issue #1066: DRILL-3993: Changes to support Calcite 1.15

[vvysotskyi]

Github user vvysotskyi commented on the issue:

https://github.com/apache/drill/pull/1066
  
@chunhui-shi, this branch is not ready completely. There is opened a pull 
request on Apache Calcite 
[CALCITE-2018](https://issues.apache.org/jira/browse/CALCITE-2018) which should 
be cherry-picked into this branch. When it is merged into Apache master, we 
will update both branches and create a pull request on mapr/incubator-calcite.


---

[GitHub] drill issue #1066: DRILL-3993: Changes to support Calcite 1.15

[chunhui-shi]

Github user chunhui-shi commented on the issue:

https://github.com/apache/drill/pull/1066
  
Can you file a pull request to mapr/incubator-calcite with your changes in 
https://github.com/KulykRoman/incubator-calcite/commits/DrillCalcite1.15.0_rc0?
So we can review and get these changes into official branch to publish.


---

[GitHub] drill issue #1076: DRILL-6036: Create sys.connections table

[kkhatua]

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1076
  
@parthchandra I've added the same security check being done for DRILL-5068. 
But I'm not sure if the match for connections should be in the original user or 
the impersonated user, or both. Suggestions? 


---

[GitHub] drill issue #1077: DRILL-5068: Create sys.profiles and sys.profiles_json tab...

[kkhatua]

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1077
  
@sohami 
Applied the checks for security to deny unauthorized access.
This is a screenshot with an admin user (`mapr`) querying the table.
https://user-images.githubusercontent.com/4335237/3695-cbe7efd4-ec84-11e7-9bf1-e4b007c8a062.png;>

This is a screenshot with a non-admin user (`kuser1`) querying the table.
https://user-images.githubusercontent.com/4335237/3715-f7f0b85e-ec84-11e7-8fe1-bbed71e2d5ea.png;>



---

[GitHub] drill issue #1067: DRILL-3958: Return a valid error message when storage plu...

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/1067
  
+1


---