[DISCUSS] Release Storm 1.1.0

[Jungtaek Lim]

Hi devs,

It's been 5 months after releasing Storm 1.0.0, and now 1.x branch has lots
of CHANGELOG and also pending reviews.
It's also been a long time after 1.1.0 RC1 is canceled.

I think it may be good to put some efforts to review and merge pending pull
requests (except things which takes time to review and test), and release
1.1.0 soon.

What do you think?

I'm also open to volunteer release manager for 1.1.0 after we document the
process of official release.

Thanks,
Jungtaek Lim (HeartSaVioR)

[GitHub] storm issue #1591: STORM-1038: Upgrade netty to 4.x in 1.x-branch

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1591
  
Please rebase and force push accordingly. Your PR now has all of the 
commits from other side.

You can give it an another try: 1) create new branch based on recent master 
or 1.x 2) cherry-pick your commits 3) checkout to your PR branch 4) reset your 
PR branch to the branch which you just worked 4) force push.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1591: STORM-1038: Upgrade netty to 4.x in 1.x-branch

[hsun-cnnxty]

GitHub user hsun-cnnxty reopened a pull request:

https://github.com/apache/storm/pull/1591

STORM-1038: Upgrade netty to 4.x in 1.x-branch

This is to add the feature to 1.x-branch.  The original PR for master 
branch is #728.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hsun-cnnxty/storm 1.x-branch-netty4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1591.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1591


commit 57fbccc7e0d710ccaf04a5a828f0ff4cf29ec855
Author: Hang Sun 
Date:   2016-07-25T22:30:54Z

STORM-1038: Upgraded netty to 4.x

commit 144b6eddf314ce9a0b5beb8c99500c56ff0b8ec0
Author: Hang Sun 
Date:   2016-07-30T23:42:52Z

STORM-1038: removed unused imports

commit 286287fb20968f8bb27e0b596ddd8008c71bc593
Author: Jungtaek Lim 
Date:   2016-07-31T14:01:28Z

STORM-1995: close input stream and other tweaks to downloadChunk

* manually applying f38982c (via @abellina)

commit d16ce38f99077012af3a77b5b46a42733845f140
Author: Jungtaek Lim 
Date:   2016-07-31T14:02:47Z

Merge branch 'STORM-1996-1.x' into 1.x-branch

commit 40e215249065e799a7539f95b0f6f6fac3138c29
Author: Jungtaek Lim 
Date:   2016-07-31T14:14:47Z

add STORM-1995 to CHANGELOG

commit fe5157bf35ed21e4ecf2edb7c5bc9d1402713384
Author: lujinhong 
Date:   2016-06-27T02:33:28Z

STORM-1594 org.apache.storm.tuple.Fields can throw NPE if given invalid 
field in selector

* Closes #1522

commit d48e97fbfdde4f3be03ce1bbb357b3b7c8f19e2f
Author: Jungtaek Lim 
Date:   2016-07-31T14:33:16Z

Merge branch 'STORM-1594-1.x' into 1.x-branch

commit 14924e3ea54bd8204b6b8a5bdde444db81a2ad09
Author: Jungtaek Lim 
Date:   2016-07-31T14:33:41Z

add STORM-1594 to CHANGELOG

commit eed4b42358faf0eef2585a402fbe342b10a2edc3
Author: Sriharsha Chintalapani 
Date:   2016-08-03T04:35:10Z

Merge branch 'STORM-1839' of https://github.com/priyank5485/storm into 
STORM-1839

commit f4480ade9aa124152b4946b9c6b9d90103128300
Author: Sriharsha Chintalapani 
Date:   2016-08-03T04:59:42Z

Added STORM-1839 to CHANGELOG.

commit 8355c9ee269efa3f61ee25b4f8515ca5ad196a24
Author: Jungtaek Lim 
Date:   2016-08-05T07:16:33Z

fix parent version of storm-kinesis module

commit 636b780eb30a6ec7a88bdc87af87001dcbe2456a
Author: Alessandro Bellina 
Date:   2016-07-28T03:37:40Z

STORM-1890: looks like too many calls were moved to getStatic. It doesnt 
break things, but it is sloppy, so reverting some to $.get

commit f38a38e8260ace2147bde4f80e103e7a2e542a4f
Author: Robert (Bobby) Evans 
Date:   2016-08-04T16:09:59Z

STORM-2021: Fix License

commit 0253657f6ea4997cc9af95c428ad2b171fa13390
Author: Jungtaek Lim 
Date:   2016-08-08T03:35:24Z

Merge branch 'storm-kinesis-version-fix-1.x' into 1.x-branch

commit d51e696b7d538a5dda97a71960b3d6d04ea4e311
Author: Manikumar Reddy O 
Date:   2016-07-21T11:57:03Z

STORM-1979: Storm Druid Connector implementation. This uses Druid's 
tranquility library.

commit 7e90b93f7c8831cb31f9bac1df7db22d1b3fe7c4
Author: Satish Duggana 
Date:   2016-08-09T05:28:19Z

Added STORM-1979 to CHANGELOG

commit 70383db6a16b8b904d81aaf68518db6cc15398c4
Author: Jungtaek Lim 
Date:   2016-08-09T08:07:59Z

fix parent version of storm-druid pom in 1.x branch

commit fe25aa6019c16968cfcd35adb7a72b0295417879
Author: P. Taylor Goetz 
Date:   2016-08-10T19:46:52Z

Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 1d0b50302c1287c9567fa17eede9e0a598381651.

commit 9e9ec4347db572ba93021b05ea8dd0e3aeadccd6
Author: P. Taylor Goetz 
Date:   2016-08-10T19:46:57Z

Revert "[maven-release-plugin] prepare release v1.1.0"

This reverts commit 4d80461a62e6f739dccff79af176bc109bc491e7.

commit b5458b8ee4668d31a67629fcf0eb4aa3978ef521
Author: Jungtaek Lim 
Date:   2016-08-11T14:43:55Z

Fix parent version of storm-druid and storm-kinesis

* during the rollback they still keep versions since it's manually fixed 
after the release process

commit c5cf73b9078176482d273907c79ca8e9a1e85498
Author: Kyle Nusbaum 
Date:   2016-06-17T17:59:06Z

Improvements for Trident RAS API.

commit d50a2437923f2919fbee130b8e9e86a62a2d9f48
Author: Boyang Jerry Peng 
Date:   2016-05-04T22:08:57Z

[STORM-1766] - A better algorithm server rack selection for RAS

[GitHub] storm pull request #1591: STORM-1038: Upgrade netty to 4.x in 1.x-branch

[hsun-cnnxty]

Github user hsun-cnnxty closed the pull request at:

https://github.com/apache/storm/pull/1591


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1687: Apache master storm 1694 top storm 2097

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1687
  
Since it does load the two modules - storm-kafka and storm-kafka-client - 
but only load kafka-client with `${storm.kafka.client.version}`. Does 
storm-starter work with both storm-kafka and storm-kafka-client? If not, I 
guess we need to move out storm-kafka and storm-kafka-client to separate 
modules like recent examples modules.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1685: STORM-2097: Improve logging in trident core and examples

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1685
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1714
  
Another head up: my patch got merged to Calcite master, and Calcite dev@ is 
discussing about releasing new minor version soon.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1685: STORM-2097: Improve logging in trident core and examples

[harshach]

Github user harshach commented on the issue:

https://github.com/apache/storm/pull/1685
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1668: STORM-2040 Fix bug on assert-can-serialize

[harshach]

Github user harshach commented on the issue:

https://github.com/apache/storm/pull/1668
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1720: STORM-1546: Adding Read and Write Aggregations for...

[knusbaum]

GitHub user knusbaum opened a pull request:

https://github.com/apache/storm/pull/1720

STORM-1546: Adding Read and Write Aggregations for Pacemaker to make it HA 
compatible (1.x-branch)

1.x-branch version of Pacemaker-HA

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knusbaum/incubator-storm 
Pacemaker-HA-1.x-branch2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1720.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1720


commit 7b34582463857eba8bc08ff8abab338f4e83fbd6
Author: Kyle Nusbaum 
Date:   2016-09-29T18:58:54Z

Moving files into old position for merge.

commit dd7ff1e0a0cecfe581c7441f941e3b9bbcc90c9d
Author: Sanket 
Date:   2016-03-04T22:41:02Z

Porting Pacemaker-HA to community.

commit a6171bd6770676d4d0bdb9c9d05a4785df35b671
Author: Kyle Nusbaum 
Date:   2016-09-29T20:43:09Z

Ported successfully.

commit 20180cca1de9b25dc391722a9088e10107a31443
Author: Kyle Nusbaum 
Date:   2016-09-29T20:45:01Z

Minor cleanup.

commit 9c5860498a71c04fda1e201283a63ec2c6352062
Author: Kyle Nusbaum 
Date:   2016-09-29T22:06:16Z

Making HA backwards-compatible




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1710: STORM-1546: Adding Read and Write Aggregations for...

[knusbaum]

Github user knusbaum commented on a diff in the pull request:

https://github.com/apache/storm/pull/1710#discussion_r81219539
  
--- Diff: storm-core/src/jvm/org/apache/storm/Config.java ---
@@ -926,10 +926,10 @@
 public static final String UI_HTTPS_NEED_CLIENT_AUTH = 
"ui.https.need.client.auth";
 
 /**
- * The host that Pacemaker is running on.
+ * The list of servers that Pacemaker is running on.
  */
-@isString
-public static final String PACEMAKER_HOST = "pacemaker.host";
+@isStringList
+public static final String PACEMAKER_SERVERS = "pacemaker.servers";
--- End diff --

@HeartSaVioR Yep, the 1.x-branch PR will leave pacemaker.host in, but have 
it deprecated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Plan for merge SQE and Storm SQL

[P. Taylor Goetz]

FYI, I’ve merged the SQE code and documentation into the sqe_import branch:

https://github.com/apache/storm/tree/sqe_import 


Note the build will fail on the last component (storm-sqe) due to the 
compilation issues mentioned earlier.

-Taylor


> On Sep 29, 2016, at 11:13 AM, Bobby Evans  wrote:
> 
> Agreed, or if we can find a way to not break compatibility (like perhaps 
> having a common base class for most of the logic and one subclass that uses a 
> string while another that uses the byte array).   - Bobby
> 
>On Thursday, September 29, 2016 10:05 AM, Jungtaek Lim  
> wrote:
> 
> 
> The change on storm-redis seems to require breaking backward compatibility,
> so I would love to see another pull request to integrate it via general
> review process.
> If it can be integrated to SQE without changing storm-redis, that would be
> nice.
> 
> Does it make sense?
> 
> - Jungtaek Lim (HeartSaVioR)
> 
> 2016년 9월 29일 (목) 오전 6:08, P. Taylor Goetz 님이 작성:
> 
>> The changes are available in GitHub, I just overlooked it. And they’re
>> actually neatly contained in two commits:
>> 
>> The changes to storm-kafka can be found here:
>> 
>> 
>> https://github.com/jwplayer/storm/commit/2069c76695a225e4bb8f402c89e572836104755a
>> 
>> 
>> The changes to storm-redis are here:
>> 
>> 
>> https://github.com/jwplayer/storm/commit/30d000d3ff673efa8b927d23e554a705fb2928b8
>> 
>> 
>> 
>> The changes to storm-kafka are straightforward and implemented in such a
>> way that they would be useful for use cases outside of SQE. As the commit
>> message states, it adds a new kafka deserialization scheme (FullScheme)
>> that includes the key, value, topic, partition and offset when reading from
>> kafka, which is a feature I can see as being valuable for some use cases. I
>> would be +1 for merging that code.
>> 
>> The changes to storm-redis are a little different, as Morrigan pointed
>> out, because it only addresses the Trident API, but IMHO it looks like a
>> good direction.
>> 
>> @HeartSavior — Would you have some time to take a look at the storm-redis
>> changes and provide your opinion, since you’re one of the original authors
>> of that code?
>> 
>> -Taylor
>> 
>> 
>> On Sep 26, 2016, at 6:28 PM, Jungtaek Lim  wrote:
>> 
>> Great!
>> 
>> For storm-redis, we might need to modify key/value mapper to use byte[]
>> rather than String.
>> When I co-authored storm-redis, I forgot considering binary format of
>> serde. If we want to address that part, we can also address it.
>> 
>> 2016년 9월 27일 (화) 오전 7:19, Morrigan Jones 님이 작성:
>> 
>> Sure, when I can. Storm-kafka should be pretty easy. The storm-redis one
>> will require more work to make it more complete.
>> 
>> On Mon, Sep 26, 2016 at 6:09 PM, P. Taylor Goetz 
>> wrote:
>> 
>> Thanks for the explanation Morrigan!
>> 
>> Would you be willing to provide a pull request or patch so the community
>> can review?
>> 
>> It sounds like at least some of the changes you mention could be useful
>> 
>> to
>> 
>> the broader community (beyond the SQL effort).
>> 
>> Thanks again,
>> 
>> -Taylor
>> 
>> On Sep 26, 2016, at 4:40 PM, Morrigan Jones 
>> 
>> wrote:
>> 
>> 
>> storm-kafka - This is needed because storm-kafka does not provide a
>> 
>> scheme
>> 
>> class that gives you the key, value (payload), partition, and offset.
>> MessageMetadataScheme.java comes comes closest, but is missing the key.
>> This was a pretty simple change on my part.
>> 
>> storm-redis - This is needed for proper support of Redis hashes. The
>> existing storm-redis uses a static string (additionalKey in
>> the RedisDataTypeDescription class) for the field name in hash types. I
>> updated it to use a configurable KeyFactory for both the hash name and
>> 
>> the
>> 
>> field name. We also added some limited support for set types. This is
>> admittedly the messiest between the two jars since we only cared about
>> 
>> the
>> 
>> trident states and would require a lot more changes to get storm-redis
>> 
>> more
>> 
>> "feature complete" overall.
>> 
>> 
>> 
>> 
>> On Mon, Sep 26, 2016 at 4:03 PM, P. Taylor Goetz 
>> 
>> wrote:
>> 
>> 
>> Sounds good. I’ll find out if it builds against 2.x. If so I’ll go
>> 
>> that
>> 
>> direction. Otherwise I’ll come back with my findings and we can
>> 
>> discuss
>> 
>> it
>> 
>> further.
>> 
>> I notice there are jars in the git repo that we obviously can’t
>> 
>> import.
>> 
>> They look like they might be custom JWPlayer builds of storm-kafka and
>> storm-redis.
>> 
>> Morrigan — Do you know if there is any differences there that required
>> custom builds of those components?
>> 
>> -Taylor
>> 
>> On Sep 26, 2016, at 3:31 PM, Bobby Evans
>> 
>> > 
>> 

Re: Error while building locally

[Sonu B]

Following is the result of git status command

> git status
--
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add ..." to include in what will be committed)

storm-core/.gitignore

nothing added to commit but untracked files present (use "git add" to track)
-

git diff doesn't show anything either.




On Thu, Sep 29, 2016 at 2:15 PM, Jungtaek Lim  wrote:

> Hi Sonu,
>
> if you clone the code, could you check git status and / or git diff to see
> the change if any?
> I'm not an Windows user, but showing version as 2.0.0 is strange and I
> haven't heard similar issues.
>
> - Jungtaek Lim
>

[GitHub] storm issue #1647: Get rid of compiler plugin setting for external modules

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1647
  
@HeartSaVioR Thanks I pulled this into master, 1.x and 1.0.x


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1647: Get rid of compiler plugin setting for external mo...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1647


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1647: Get rid of compiler plugin setting for external modules

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1647
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1684: [STORM-2093] Fix permissions in multi-tenant, secure mode

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1684
  
I agree with @HeartSaVioR could you please change the commit message to 
something more like the pull request title?

```
git commit --amend
```

should do it for you.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1714
  
Just a head up: my patch for Calcite 
(https://github.com/apache/calcite/pull/283) passes the review and in progress 
of merging. 
(handling some comment fixes.)
So I'm expecting that we can get runtime safety with next version of 
Calcite.

Do we want to hold on waiting next version of Calcite? Or just ship this as 
it is for now, and I will submit another pull request following up the change 
of Calcite?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1716: STORM-2126: fix NPE due to race condition in compu...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1716


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Plan for merge SQE and Storm SQL

[Bobby Evans]

Agreed, or if we can find a way to not break compatibility (like perhaps having 
a common base class for most of the logic and one subclass that uses a string 
while another that uses the byte array).   - Bobby 

On Thursday, September 29, 2016 10:05 AM, Jungtaek Lim  
wrote:
 

 The change on storm-redis seems to require breaking backward compatibility,
so I would love to see another pull request to integrate it via general
review process.
If it can be integrated to SQE without changing storm-redis, that would be
nice.

Does it make sense?

- Jungtaek Lim (HeartSaVioR)

2016년 9월 29일 (목) 오전 6:08, P. Taylor Goetz 님이 작성:

> The changes are available in GitHub, I just overlooked it. And they’re
> actually neatly contained in two commits:
>
> The changes to storm-kafka can be found here:
>
>
> https://github.com/jwplayer/storm/commit/2069c76695a225e4bb8f402c89e572836104755a
>
>
> The changes to storm-redis are here:
>
>
> https://github.com/jwplayer/storm/commit/30d000d3ff673efa8b927d23e554a705fb2928b8
> 
>
>
> The changes to storm-kafka are straightforward and implemented in such a
> way that they would be useful for use cases outside of SQE. As the commit
> message states, it adds a new kafka deserialization scheme (FullScheme)
> that includes the key, value, topic, partition and offset when reading from
> kafka, which is a feature I can see as being valuable for some use cases. I
> would be +1 for merging that code.
>
> The changes to storm-redis are a little different, as Morrigan pointed
> out, because it only addresses the Trident API, but IMHO it looks like a
> good direction.
>
> @HeartSavior — Would you have some time to take a look at the storm-redis
> changes and provide your opinion, since you’re one of the original authors
> of that code?
>
> -Taylor
>
>
> On Sep 26, 2016, at 6:28 PM, Jungtaek Lim  wrote:
>
> Great!
>
> For storm-redis, we might need to modify key/value mapper to use byte[]
> rather than String.
> When I co-authored storm-redis, I forgot considering binary format of
> serde. If we want to address that part, we can also address it.
>
> 2016년 9월 27일 (화) 오전 7:19, Morrigan Jones 님이 작성:
>
> Sure, when I can. Storm-kafka should be pretty easy. The storm-redis one
> will require more work to make it more complete.
>
> On Mon, Sep 26, 2016 at 6:09 PM, P. Taylor Goetz 
> wrote:
>
> Thanks for the explanation Morrigan!
>
> Would you be willing to provide a pull request or patch so the community
> can review?
>
> It sounds like at least some of the changes you mention could be useful
>
> to
>
> the broader community (beyond the SQL effort).
>
> Thanks again,
>
> -Taylor
>
> On Sep 26, 2016, at 4:40 PM, Morrigan Jones 
>
> wrote:
>
>
> storm-kafka - This is needed because storm-kafka does not provide a
>
> scheme
>
> class that gives you the key, value (payload), partition, and offset.
> MessageMetadataScheme.java comes comes closest, but is missing the key.
> This was a pretty simple change on my part.
>
> storm-redis - This is needed for proper support of Redis hashes. The
> existing storm-redis uses a static string (additionalKey in
> the RedisDataTypeDescription class) for the field name in hash types. I
> updated it to use a configurable KeyFactory for both the hash name and
>
> the
>
> field name. We also added some limited support for set types. This is
> admittedly the messiest between the two jars since we only cared about
>
> the
>
> trident states and would require a lot more changes to get storm-redis
>
> more
>
> "feature complete" overall.
>
>
>
>
> On Mon, Sep 26, 2016 at 4:03 PM, P. Taylor Goetz 
>
> wrote:
>
>
> Sounds good. I’ll find out if it builds against 2.x. If so I’ll go
>
> that
>
> direction. Otherwise I’ll come back with my findings and we can
>
> discuss
>
> it
>
> further.
>
> I notice there are jars in the git repo that we obviously can’t
>
> import.
>
> They look like they might be custom JWPlayer builds of storm-kafka and
> storm-redis.
>
> Morrigan — Do you know if there is any differences there that required
> custom builds of those components?
>
> -Taylor
>
> On Sep 26, 2016, at 3:31 PM, Bobby Evans
>
> 
>
> wrote:
>
> Does it compile against 2.X?  If so I would prefer to have it go
>
> there,
>
> and then possibly 1.x if people what it there too. - Bobby
>
>
>  On Monday, September 26, 2016 12:47 PM, P. Taylor Goetz <
>
> ptgo...@gmail.com> wrote:
>
>
> The IP Clearance vote has passed and we are now able to import the
>
> SQE
>
> code.
>
>
> The question now is to where do we want to import the code?
>
> My inclination is to import it to “external” in the 1.x branch. It
>
> can
>
> be ported to other branches as necessary/if desired. An alternative
>
> would
>
> be to treat it as a feature branch, but 

Re: [DISCUSS] Plan for merge SQE and Storm SQL

[Jungtaek Lim]

The change on storm-redis seems to require breaking backward compatibility,
so I would love to see another pull request to integrate it via general
review process.
If it can be integrated to SQE without changing storm-redis, that would be
nice.

Does it make sense?

- Jungtaek Lim (HeartSaVioR)

2016년 9월 29일 (목) 오전 6:08, P. Taylor Goetz 님이 작성:

> The changes are available in GitHub, I just overlooked it. And they’re
> actually neatly contained in two commits:
>
> The changes to storm-kafka can be found here:
>
>
> https://github.com/jwplayer/storm/commit/2069c76695a225e4bb8f402c89e572836104755a
>
>
> The changes to storm-redis are here:
>
>
> https://github.com/jwplayer/storm/commit/30d000d3ff673efa8b927d23e554a705fb2928b8
> 
>
>
> The changes to storm-kafka are straightforward and implemented in such a
> way that they would be useful for use cases outside of SQE. As the commit
> message states, it adds a new kafka deserialization scheme (FullScheme)
> that includes the key, value, topic, partition and offset when reading from
> kafka, which is a feature I can see as being valuable for some use cases. I
> would be +1 for merging that code.
>
> The changes to storm-redis are a little different, as Morrigan pointed
> out, because it only addresses the Trident API, but IMHO it looks like a
> good direction.
>
> @HeartSavior — Would you have some time to take a look at the storm-redis
> changes and provide your opinion, since you’re one of the original authors
> of that code?
>
> -Taylor
>
>
> On Sep 26, 2016, at 6:28 PM, Jungtaek Lim  wrote:
>
> Great!
>
> For storm-redis, we might need to modify key/value mapper to use byte[]
> rather than String.
> When I co-authored storm-redis, I forgot considering binary format of
> serde. If we want to address that part, we can also address it.
>
> 2016년 9월 27일 (화) 오전 7:19, Morrigan Jones 님이 작성:
>
> Sure, when I can. Storm-kafka should be pretty easy. The storm-redis one
> will require more work to make it more complete.
>
> On Mon, Sep 26, 2016 at 6:09 PM, P. Taylor Goetz 
> wrote:
>
> Thanks for the explanation Morrigan!
>
> Would you be willing to provide a pull request or patch so the community
> can review?
>
> It sounds like at least some of the changes you mention could be useful
>
> to
>
> the broader community (beyond the SQL effort).
>
> Thanks again,
>
> -Taylor
>
> On Sep 26, 2016, at 4:40 PM, Morrigan Jones 
>
> wrote:
>
>
> storm-kafka - This is needed because storm-kafka does not provide a
>
> scheme
>
> class that gives you the key, value (payload), partition, and offset.
> MessageMetadataScheme.java comes comes closest, but is missing the key.
> This was a pretty simple change on my part.
>
> storm-redis - This is needed for proper support of Redis hashes. The
> existing storm-redis uses a static string (additionalKey in
> the RedisDataTypeDescription class) for the field name in hash types. I
> updated it to use a configurable KeyFactory for both the hash name and
>
> the
>
> field name. We also added some limited support for set types. This is
> admittedly the messiest between the two jars since we only cared about
>
> the
>
> trident states and would require a lot more changes to get storm-redis
>
> more
>
> "feature complete" overall.
>
>
>
>
> On Mon, Sep 26, 2016 at 4:03 PM, P. Taylor Goetz 
>
> wrote:
>
>
> Sounds good. I’ll find out if it builds against 2.x. If so I’ll go
>
> that
>
> direction. Otherwise I’ll come back with my findings and we can
>
> discuss
>
> it
>
> further.
>
> I notice there are jars in the git repo that we obviously can’t
>
> import.
>
> They look like they might be custom JWPlayer builds of storm-kafka and
> storm-redis.
>
> Morrigan — Do you know if there is any differences there that required
> custom builds of those components?
>
> -Taylor
>
> On Sep 26, 2016, at 3:31 PM, Bobby Evans
>
> 
>
> wrote:
>
> Does it compile against 2.X?  If so I would prefer to have it go
>
> there,
>
> and then possibly 1.x if people what it there too. - Bobby
>
>
>  On Monday, September 26, 2016 12:47 PM, P. Taylor Goetz <
>
> ptgo...@gmail.com> wrote:
>
>
> The IP Clearance vote has passed and we are now able to import the
>
> SQE
>
> code.
>
>
> The question now is to where do we want to import the code?
>
> My inclination is to import it to “external” in the 1.x branch. It
>
> can
>
> be ported to other branches as necessary/if desired. An alternative
>
> would
>
> be to treat it as a feature branch, but I’d rather take the former
>
> approach.
>
>
> Thought/opinions?
>
> -Taylor
>
> On Sep 21, 2016, at 8:39 PM, P. Taylor Goetz 
>
> wrote:
>
>
> My apologies. I meant to cc dev@, but didn't. Will forward in a
>
> bit...
>
>
> The vote (lazy consensus) is underway on general@incubator, 

[GitHub] storm pull request #1718: Storm 2124 show requested cpu mem for each compone...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1718


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1718: Storm 2124 show requested cpu mem for each component 1x

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1718
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Error while building locally

[Jungtaek Lim]

Hi Sonu,

if you clone the code, could you check git status and / or git diff to see
the change if any?
I'm not an Windows user, but showing version as 2.0.0 is strange and I
haven't heard similar issues.

- Jungtaek Lim