[GitHub] storm pull request #1728: STORM-2142 ReportErrorAndDie runs suicide function...

2016-10-17 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1728


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1728: STORM-2142 ReportErrorAndDie runs suicide function only w...

2016-10-17 Thread HeartSaVioR 
Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1728
  
@knusbaum Thanks for the review. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1735: STORM-203 Adding paths to default java library path

2016-10-17 Thread picadoh 
Github user picadoh commented on the issue:

https://github.com/apache/storm/pull/1735
  
yeah, makes total sense 👍 removed it from the path. thanks @knusbaum 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1738: STORM-2127: Storm-eventhubs should use latest amqp...

2016-10-17 Thread raviperi 
GitHub user raviperi opened a pull request:

https://github.com/apache/storm/pull/1738

STORM-2127: Storm-eventhubs should use latest amqp and eventhubs-client 
versions

STORM-2127: Storm-eventhubs should use latest amqp and eventhubs-client 
versions

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/raviperi/storm 1.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1738.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1738


commit 60e6eb9dd685f1ee434065f0de80bd4b1e45bef6
Author: Ravi Peri 
Date:   2016-10-17T22:59:56Z

STORM-2127: Storm-eventhubs should use latest amqp and eventhubs-client 
versions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1736: STORM-1446 Compile the Calcite logical plan to Storm Trid...

2016-10-17 Thread HeartSaVioR 
Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1736
  
@manuzhang Yes issue description just links Julian's comment and I think 
it's not sufficient. I'll also write up motivation and rationale to issue 
description. Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1603: STORM-2012: Upgrade Kafka to 0.8.2.2

2016-10-17 Thread knusbaum 
Github user knusbaum commented on the issue:

https://github.com/apache/storm/pull/1603
  
Builds fine locally.
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1728: STORM-2142 ReportErrorAndDie runs suicide function only w...

2016-10-17 Thread knusbaum 
Github user knusbaum commented on the issue:

https://github.com/apache/storm/pull/1728
  
(Nice catch, BTW)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1728: STORM-2142 ReportErrorAndDie runs suicide function only w...

2016-10-17 Thread knusbaum 
Github user knusbaum commented on the issue:

https://github.com/apache/storm/pull/1728
  
+1
Looks like mistranslation. This restores the behavior of the old Clojure 
code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1735: STORM-203 Adding paths to default java library path

2016-10-17 Thread knusbaum 
Github user knusbaum commented on the issue:

https://github.com/apache/storm/pull/1735
  
/usr/lib64 is FHS standard. /usr/lib/x86_64-linux-gnu is Debian-specific. I 
think we should avoid setting distro-specific defaults.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1737: STORM-2151: Update dependency on hadoop version to...

2016-10-17 Thread kishorvpatil 
GitHub user kishorvpatil opened a pull request:

https://github.com/apache/storm/pull/1737

STORM-2151: Update dependency on hadoop version to 2.7.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kishorvpatil/incubator-storm changeHDPVersion

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1737.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1737


commit ff0bac2f926dcce767d33d6c79ac0fbd233ca277
Author: Kishor Patil 
Date:   2016-10-17T19:49:32Z

Update dependency on hadoop version to 2.7.1




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1702: STORM-2127: Storm-eventhubs should use latest amqp and ev...

2016-10-17 Thread raviperi 
Github user raviperi commented on the issue:

https://github.com/apache/storm/pull/1702
  
@harshach  
Updated title as per your recommendation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1717: STORM-2127: Storm-eventhubs should use latest amqp and ev...

2016-10-17 Thread raviperi 
Github user raviperi commented on the issue:

https://github.com/apache/storm/pull/1717
  
@harshach  
Updated title as per your recommendation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1702: Jira 2127- Storm-eventhubs should use latest amqp and eve...

2016-10-17 Thread harshach 
Github user harshach commented on the issue:

https://github.com/apache/storm/pull/1702
  
@raviperi sorry for nitpick. we need the format of the title to be
" STORM-2127: Storm-eventhubs should use latest amqp and eventhubs-client 
versions"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1717: Update eventhub client dependency. Move Storm-Eventhubs d...

2016-10-17 Thread harshach 
Github user harshach commented on the issue:

https://github.com/apache/storm/pull/1717
  
@raviperi  can you add the STORM jira number to the commit message and 
squash commits in both the PRs. Use the same title as the JIRA title 
https://issues.apache.org/jira/browse/STORM-2127


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1702: Update Eventhub-Client jar dependency to 1.0.1

2016-10-17 Thread harshach 
Github user harshach commented on the issue:

https://github.com/apache/storm/pull/1702
  
@raviperi do you have JIRA filed here for this 
https://issues.apache.org/jira/browse/STORM/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1696: STORM-2104: More graceful handling of acked/failed tuples...

2016-10-17 Thread srdo 
Github user srdo commented on the issue:

https://github.com/apache/storm/pull/1696
  
@hmcl ping. Had a chance to look at this? It would be nice to get merged 
soon.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Different way used to create file path in hdfs state

2016-10-17 Thread Sachin Pasalkar 
Can anyone let me know if its done purposefully?


On 13/10/16, 11:08 PM, "Sachin Pasalkar" 
wrote:

>Hi,
>
>I was looking HdfsState class where createOutputFile api present at line
>182 is as follows
>
>Path path = new Path(this.fileNameFormat.getPath(),
>this.fileNameFormat.getName(this.rotation, System.currentTimeMillis()));
>
>whereas there is another createOutputFile at line 272 line
>
>Path p = new Path(this.fsUrl + this.fileNameFormat.getPath(),
>this.fileNameFormat.getName(this.rotation, System.currentTimeMillis()));
>
>In line number 182 user has to provide the hdfs uri in path itself, its
>handled properly in 272
>
>Is there any reason for above difference or its just missed?
>
>Thanks,
>Sachin

[GitHub] storm issue #1736: STORM-1446 Compile the Calcite logical plan to Storm Trid...

2016-10-17 Thread manuzhang 
Github user manuzhang commented on the issue:

https://github.com/apache/storm/pull/1736
  
@HeartSaVioR Thanks for the thorough explanation. I'd suggest record the 
motive and rationals somewhere.  It's hard to follow why something exists in 
the first place. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1736: STORM-1446 Compile the Calcite logical plan to Storm Trid...

2016-10-17 Thread HeartSaVioR 
Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1736
  
@manuzhang 
Thanks for showing interest on this. Let me rearrange your questions:

> Has this changed the workflow of StormSQL?

No, we still rely on Trident, and nothing changed in point of workflow.

> Motive behind this big change and benefits

This is started from [Julian's 
comment](https://issues.apache.org/jira/browse/STORM-1040?focusedCommentId=15034472=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15034472)
 and also [Milinda's 
comment](https://issues.apache.org/jira/browse/STORM-1040?focusedCommentId=15035182=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15035182).

For me having own relational algebras (rel) has several advantages, 

- We can push operator handling logic to rel itself. Before that we should 
traverse Calcite logical rel  tree with PostOrderRelNodeVisitor, and visitor 
needs to handle Calcite's rel directly. Now the logic how to configure Trident 
topology is all handled from separate rels.

- We sometimes want to have more derived rels compared to Calcite logical 
operators. One of example is `Join`.  There's only one logical rel regarding 
join in Calcite - LogicalJoin - but we're now converting LogicalJoin to 
EquiJoin if conditions are met. If we have various types of join it will make 
the difference. We're not prepared yet, but streaming scan vs table scan, and 
streaming insert vs table insert are the other cases.

```
TridentStormAggregateRel(group=[{0}], EXPR$1=[COUNT()])
  TridentStormCalcRel(expr#0..4=[{inputs}], expr#5=[0], expr#6=[>($t0, 
$t5)], DEPTID=[$t3], EMPID=[$t0], $condition=[$t6])
TridentStormEquiJoinRel(condition=[=($2, $3)], joinType=[inner])
  TridentStormStreamScanRel(table=[[EMP]])
  TridentStormStreamScanRel(table=[[DEPT]])
```

We can even override the methods how to represent the rel in explain string 
if we think Calcite's explain is less informational. For example, showing 
initial parallelism (when we support) for Scan.

- This patch starts addressing query optimizations. One of example is Calc, 
which is for merging multiple calculations into one. No need to filter and 
projection separately. They're still not minimized (as projection is), but it 
can be addressed after STORM-2072. Defining derived rels helps further query 
optimizations, like filter pushdown. Calcite rels is not aware of data source 
characteristic, and we can include it to our own rels.

Please don't hesitate to ask questions. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---