Re: Drill plugin for Kafka

[Neeraja Rentachintala]

Great Anil. Thanks.

On Wed, Jul 13, 2016 at 9:02 PM, AnilKumar B  wrote:

> Thanks Parth. Created the https://issues.apache.org/jira/browse/DRILL-4779
>
> Like Mongo storage plugin contribution, Kamesh and me would like to start
> working on this feature.
>
> Thanks & Regards,
> B Anil Kumar.
>
> On Wed, Jul 13, 2016 at 7:53 PM, Parth Chandra  wrote:
>
> > That would be great.
> >
> > I don't think anyone is working on one. So please go ahead and create a
> > JIRA and assign it to yourself.
> >
> > And thank you for contributing.
> >
> >
> >
> > On Wed, Jul 13, 2016 at 3:22 PM, AnilKumar B 
> > wrote:
> >
> > > Hi All,
> > >
> > > Is it useful to have Drill plugin for Kafka? And if so, is there
> anyone,
> > > working on it already? Else planning to start working on it.
> > >
> > >
> > > Thanks & Regards,
> > > B Anil Kumar.
> > >
> >
>

Re: Drill plugin for Kafka

[AnilKumar B]

Thanks Parth. Created the https://issues.apache.org/jira/browse/DRILL-4779

Like Mongo storage plugin contribution, Kamesh and me would like to start
working on this feature.

Thanks & Regards,
B Anil Kumar.

On Wed, Jul 13, 2016 at 7:53 PM, Parth Chandra  wrote:

> That would be great.
>
> I don't think anyone is working on one. So please go ahead and create a
> JIRA and assign it to yourself.
>
> And thank you for contributing.
>
>
>
> On Wed, Jul 13, 2016 at 3:22 PM, AnilKumar B 
> wrote:
>
> > Hi All,
> >
> > Is it useful to have Drill plugin for Kafka? And if so, is there anyone,
> > working on it already? Else planning to start working on it.
> >
> >
> > Thanks & Regards,
> > B Anil Kumar.
> >
>

[jira] [Created] (DRILL-4779) Kafka storage plugin support

[B Anil Kumar (JIRA)]

B Anil Kumar created DRILL-4779:
---

 Summary: Kafka storage plugin support
 Key: DRILL-4779
 URL: https://issues.apache.org/jira/browse/DRILL-4779
 Project: Apache Drill
  Issue Type: New Feature
  Components: Storage - Other
Reporter: B Anil Kumar


Implement Kafka storage plugin will enable the strong SQL support for Kafka.

Initially implementation can target for supporting text, json and avro message 
types



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill issue #540: Fix for DRILL-4759: Drill throwing array index out of boun...

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/540
  
Can you log a new JIRA to investigate and address issues with any other 
column types?
+1 for this change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Drill plugin for Kafka

[Parth Chandra]

That would be great.

I don't think anyone is working on one. So please go ahead and create a
JIRA and assign it to yourself.

And thank you for contributing.



On Wed, Jul 13, 2016 at 3:22 PM, AnilKumar B  wrote:

> Hi All,
>
> Is it useful to have Drill plugin for Kafka? And if so, is there anyone,
> working on it already? Else planning to start working on it.
>
>
> Thanks & Regards,
> B Anil Kumar.
>

Drill plugin for Kafka

[AnilKumar B]

Hi All,

Is it useful to have Drill plugin for Kafka? And if so, is there anyone,
working on it already? Else planning to start working on it.


Thanks & Regards,
B Anil Kumar.

[GitHub] drill pull request #519: DRILL-4530: Optimize partition pruning with metadat...

[amansinha100]

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/519#discussion_r70706252
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java
 ---
@@ -143,6 +144,11 @@ public static final RelOptRule 
getDirFilterOnScan(OptimizerRulesContext optimize
   }
 
   protected void doOnMatch(RelOptRuleCall call, Filter filterRel, Project 
projectRel, TableScan scanRel) {
+if (wasAllPartitionsPruned) {
--- End diff --

Agree that this should not be an internal state.  I had added it to 
overcome an issue where the prunescan rule gets re-applied even when there are 
not qualifying partitions (this happens because we always populate the 
newPartitions list with at least 1 entry to produce a valid schema during 
execution).   

I tried removing this flag but will need some associated changes to prevent 
the rule from being reapplied without termination.  The old code (without my 
patch) terminates the pruning because FileSelection.supportDirPruning() returns 
FALSE when the selection has directories.   Whereas, for this optimization I 
want to do the pruning on directories first.   So, let me think of a way to 
address that. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #517: DRILL-4704 fix

[daveoshinsky]

Github user daveoshinsky commented on the issue:

https://github.com/apache/drill/pull/517
  
Regarding Aman's last comment, please add the following... In the scenario 
addressed by this fix, there is a const integer that came from the SQL select 
where clause, and that is what is being casted into a decimal, for comparison 
with each a decimal value in each row.   Since the integer is constant, a 
thoroughly optimized piece of code would cast it just once to a decimal, and 
use the stored decimal to compare with the decimal value in each row.  It 
appears that that is not what this code is doing. It is casting repeatedly, for 
each row.  So, rather than be concerned with just a few added integer divisions 
by 10 (a small number around log10(the const integer) + 1 divides), this code 
should ideally be (more) extensively modified to do the cast operation just 
once.  That's too big a change for me to make for this fix.  Besides, which is 
preferable?  Code that does not function properly (DRILL-4704), and is not well 
optimized?  Or code that behaves correctly, and is only slightly 
 less well optimized?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #540: Fix for DRILL-4759: Drill throwing array index out of boun...

[ppadma]

Github user ppadma commented on the issue:

https://github.com/apache/drill/pull/540
  
updated the pull request with test case added.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #440: Elasticsearch storage plugin

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/440
  
@hamdanuk and @xwinie please ask questions on the user list. Details [are 
here](http://drill.apache.org/mailinglists/).

@hamdanuk can you please close this pull request? Our project uses pull 
requests for code review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #482: DRILL-4604: Generate warning on Web UI if drillbits...

[sudheeshkatkam]

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/482#discussion_r70672079
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java ---
@@ -207,6 +210,21 @@ private void javaPropertiesToSystemOptions() {
   }
 
   /**
+   * If cluster version is the same as default, update option to current 
drillbit version.
+   */
+  private void checkAndUpdateClusterVersionOption(OptionManager 
optionManager, DrillbitEndpoint drillbitEndpoint) {
+OptionValue versionOption = 
optionManager.getOption(ExecConstants.CLUSTER_VERSION);
+OptionValidator validator = 
SystemOptionManager.getValidator(ExecConstants.CLUSTER_VERSION);
+if (versionOption.equals(validator.getDefault())) {
+  optionManager.setOption(OptionValue.createOption(
--- End diff --

I don't think using a system option is the right way to detect version 
conflicts; cluster wide upgrade and restart will cause all drillbits to 
generate a warning because system options are persisted.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #493: DRILL-4647: C++ client fails to propagate a dead connectio...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/493
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #:

[xwinie]

Github user xwinie commented on the pull request:


https://github.com/apache/drill/commit/2ca9c907bff639e08a561eac32e0acab3a0b3304#commitcomment-18225081
  
drill query mongodb show  Chinese garbled;
drill query:

![image](https://cloud.githubusercontent.com/assets/3609288/16795720/846f1a96-4911-11e6-83f3-ec76005edd57.png)

mongo query  results：

![image](https://cloud.githubusercontent.com/assets/3609288/16795720/846f1a96-4911-11e6-83f3-ec76005edd57.png)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #440: Elasticsearch storage plugin

[xwinie]

Github user xwinie commented on the issue:

https://github.com/apache/drill/pull/440
  
drill query mongodb plugin show Chinese garbled!

![image](https://cloud.githubusercontent.com/assets/3609288/16794123/653533d0-4908-11e6-82df-6dd0c98c47bb.png)
  

![image](https://cloud.githubusercontent.com/assets/3609288/16794132/6e495a6e-4908-11e6-8123-356c7a2a76fb.png)
  所有中文显示都是乱码！




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---