[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746920#comment-14746920 ] ASF GitHub Bot commented on STORM-1015: --- Github user hsun-cnnxty commented on the pu

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user hsun-cnnxty commented on the pull request: https://github.com/apache/storm/pull/705#issuecomment-140633818 Erik, I added 2 reference links in the description with more information on "Kafka's consumer offset management api". As of migration from existing o

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user hsun-cnnxty commented on a diff in the pull request: https://github.com/apache/storm/pull/705#discussion_r39595289 --- Diff: external/storm-kafka/src/jvm/storm/kafka/PartitionStateManagerFactory.java --- @@ -0,0 +1,68 @@ +package storm.kafka; + +import backt

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746912#comment-14746912 ] ASF GitHub Bot commented on STORM-1015: --- Github user hsun-cnnxty commented on a diff

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746913#comment-14746913 ] ASF GitHub Bot commented on STORM-1015: --- Github user hsun-cnnxty commented on a diff

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user hsun-cnnxty commented on a diff in the pull request: https://github.com/apache/storm/pull/705#discussion_r39595284 --- Diff: external/storm-kafka/src/jvm/storm/kafka/KafkaDataStore.java --- @@ -0,0 +1,202 @@ +package storm.kafka; + +import com.google.common.c

[jira] [Updated] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hang Sun updated STORM-1015: Description: Current Kafka spout stores the offsets (and some other states) inside ZK with its proprietary

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746690#comment-14746690 ] ASF GitHub Bot commented on STORM-1015: --- Github user erikdw commented on the pull re

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user erikdw commented on the pull request: https://github.com/apache/storm/pull/705#issuecomment-140606694 Furthermore, can you please provide a reference to "Kafka's consumer offset management api" in your description? --- If your project is set up for it, you can reply to th

[GitHub] storm pull request: configurable supervisor log filename

Github user erikdw commented on the pull request: https://github.com/apache/storm/pull/738#issuecomment-140592209 Seems the 0.9.x-branch isn't set up for travis (above failure is because we don't have a `.travis.yml`, and no `Rakefile`). I'm assuming that this means I shouldn't be

[GitHub] storm pull request: configurable supervisor log filename

Github user erikdw commented on the pull request: https://github.com/apache/storm/pull/733#issuecomment-140591396 @harshach : maybe you are willing trade me reviewing this PR for my [fixes to SECURITY.md](https://github.com/apache/storm/pull/735)? :smiling_imp: I also have a

[GitHub] storm pull request: configurable supervisor log filename

GitHub user erikdw opened a pull request: https://github.com/apache/storm/pull/738 configurable supervisor log filename For the storm-on-mesos project there are multiple Storm Supervisors on every worker host. Each of these Supervisors is dedicated to a particular topology. A

[jira] [Updated] (STORM-1047) document internals of bin/storm.py

[ https://issues.apache.org/jira/browse/STORM-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Weathers updated STORM-1047: - Description: The `python` script `bin/storm.py` is completely undocumented regarding its internal

[GitHub] storm pull request: added missing installation instructions for in...

Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140578684 +1. Thanks for contributing! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user knusbaum commented on the pull request: https://github.com/apache/storm/pull/736#issuecomment-140578252 Compiles/runs fine for me. It would be nice to have a makefile or somesuch to implement basic targets like compile, install, and test, since maven's commands ha

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746468#comment-14746468 ] ASF GitHub Bot commented on STORM-1012: --- Github user knusbaum commented on the pull

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746464#comment-14746464 ] ASF GitHub Bot commented on STORM-1012: --- Github user knusbaum commented on a diff in

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39577652 --- Diff: DEVELOPER.md --- @@ -274,7 +261,8 @@ You can create a _distribution_ (like what you can download from Apache) as foll do not use the Maven rele

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746461#comment-14746461 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on a diff

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39577362 --- Diff: DEVELOPER.md --- @@ -274,7 +261,8 @@ You can create a _distribution_ (like what you can download from Apache) as foll do not use the Maven r

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746445#comment-14746445 ] ASF GitHub Bot commented on STORM-1012: --- Github user knusbaum commented on a diff in

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39576902 --- Diff: storm-core/pom.xml --- @@ -385,168 +427,199 @@ com.googlecode.json-simple:json-simple

[GitHub] storm pull request: added missing installation instructions for in...

Github user krichter722 commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140576591 Done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746440#comment-14746440 ] ASF GitHub Bot commented on STORM-1012: --- Github user knusbaum commented on a diff in

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39576725 --- Diff: DEVELOPER.md --- @@ -274,7 +261,8 @@ You can create a _distribution_ (like what you can download from Apache) as foll do not use the Maven rele

[jira] [Updated] (STORM-1047) document internals of bin/storm.py

[ https://issues.apache.org/jira/browse/STORM-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated STORM-1047: Issue Type: Documentation (was: Bug) > document internals of bin/storm.py > ---

[jira] [Updated] (STORM-1047) document internals of bin/storm.py

[ https://issues.apache.org/jira/browse/STORM-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Richter updated STORM-1047: Description: The `python` script `bin/storm.py` is completely undocumented regarding it's internals

[jira] [Created] (STORM-1047) document internals of bin/storm.py

Karl Richter created STORM-1047: --- Summary: document internals of bin/storm.py Key: STORM-1047 URL: https://issues.apache.org/jira/browse/STORM-1047 Project: Apache Storm Issue Type: Bug Aff

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746418#comment-14746418 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on the pu

[GitHub] storm pull request: added missing installation instructions for in...

Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140570580 @krichter722 Great idea. Could you add it too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/736#issuecomment-140570127 May be a beginner questions, I have some questions regarding changeset. 1. Seems like we're excluding "clj" files from shaded clojure library dependencies whi

Location of tests for python scripts in bin/?

Hi, I was wondering where the tests for the python scripts in `bin/` (e.g. `bin/storm.py`) reside? The location doesn't seem to follow a well-known layout (e.g. `tests` subdirectory) nor is anything mentioned in `DEVELOPER.md` and there's no `HACKING` file. Best regards, Kalle Richter signature

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746400#comment-14746400 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on a diff

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39574688 --- Diff: storm-dist/binary/src/main/assembly/binary.xml --- @@ -30,38 +30,6 @@ false lib false -

[GitHub] storm pull request: added missing installation instructions for in...

Github user krichter722 commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140565438 > Btw, you can get Clojure failed whenever any clojure tests error occur, so it doesn't tell you have an issue related to multilang. You should open target/test-re

[GitHub] storm pull request: added missing installation instructions for in...

Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140564025 Great, +1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] storm pull request: added missing installation instructions for in...

Github user krichter722 commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140563170 > Btw, you can get Clojure failed whenever any clojure tests error occur, so it doesn't tell you have an issue related to multilang. You should open target/test-re

[GitHub] storm pull request: added missing installation instructions for in...

Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140561321 > Python doesn't have package manager so if you don't have python, you need to install manually. I agree with you. We don't need that sentence, just providing

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746324#comment-14746324 ] ASF GitHub Bot commented on STORM-1015: --- Github user erikdw commented on the pull re

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user erikdw commented on the pull request: https://github.com/apache/storm/pull/705#issuecomment-140556439 This seems like it might not be backwards compatible with the existing kafka-spout; i.e., the offsets in ZK are presumably not going to be stored exactly the same as they

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746319#comment-14746319 ] ASF GitHub Bot commented on STORM-1015: --- Github user erikdw commented on a diff in t

[jira] [Commented] (STORM-1015) Store Kafka offsets with Kafka's consumer offset management api

[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746314#comment-14746314 ] ASF GitHub Bot commented on STORM-1015: --- Github user erikdw commented on a diff in t

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user erikdw commented on a diff in the pull request: https://github.com/apache/storm/pull/705#discussion_r39569887 --- Diff: external/storm-kafka/src/jvm/storm/kafka/PartitionStateManagerFactory.java --- @@ -0,0 +1,68 @@ +package storm.kafka; + +import backtype.s

[GitHub] storm pull request: [STORM-1015] Allow Kafka offsets to be saved u...

Github user erikdw commented on a diff in the pull request: https://github.com/apache/storm/pull/705#discussion_r39569807 --- Diff: external/storm-kafka/src/jvm/storm/kafka/KafkaDataStore.java --- @@ -0,0 +1,202 @@ +package storm.kafka; + +import com.google.common.collec

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746312#comment-14746312 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on a diff

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39569716 --- Diff: storm-core/pom.xml --- @@ -385,168 +427,199 @@ com.googlecode.json-simple:json-simple

[GitHub] storm pull request: added missing installation instructions for in...

Github user krichter722 commented on the pull request: https://github.com/apache/storm/pull/737#issuecomment-140553259 Thanks for your feedback. I'm just getting started with (building) `storm`. > Python doesn't have package manager so if you don't have python, you need to ins

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746256#comment-14746256 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on a diff

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39567827 --- Diff: dev-tools/travis/travis-script.sh --- @@ -31,7 +31,7 @@ cd ${STORM_SRC_ROOT_DIR} export STORM_TEST_TIMEOUT_MS=10 # We now lean

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746252#comment-14746252 ] ASF GitHub Bot commented on STORM-1012: --- Github user HeartSaVioR commented on a diff

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/736#discussion_r39567664 --- Diff: dev-tools/travis/travis-install.sh --- @@ -26,12 +26,23 @@ TRAVIS_SCRIPT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) cd ${ST

[GitHub] storm pull request: added missing installation instructions for in...

Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/737#discussion_r39566345 --- Diff: DEVELOPER.md --- @@ -222,6 +222,19 @@ To pull in a merge request you should generally follow the command line instruct # Build the cod

[GitHub] storm pull request: added missing installation instructions for in...

GitHub user krichter722 opened a pull request: https://github.com/apache/storm/pull/737 added missing installation instructions for installation of rvm and n… …vm of DEVELOPER.md (without running \`mvn clean install\`) isn't possible on an average system (like Ubuntu 15.04) and

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/736#issuecomment-140513112 I am working on a version for 0.10.0 too, This is against master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746006#comment-14746006 ] ASF GitHub Bot commented on STORM-1012: --- Github user revans2 commented on the pull r

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746003#comment-14746003 ] ASF GitHub Bot commented on STORM-1012: --- GitHub user revans2 opened a pull request:

[GitHub] storm pull request: STORM-1012 STORM-967 STORM-922 STORM-1042 Shad...

GitHub user revans2 opened a pull request: https://github.com/apache/storm/pull/736 STORM-1012 STORM-967 STORM-922 STORM-1042 Shaded everything that was not already shaded, and cleaned up build s… …o the dependency-reduced pom.xml is used correctly. You can merge this pull requ

[jira] [Commented] (STORM-1012) Shade Jackson dependency

[ https://issues.apache.org/jira/browse/STORM-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745704#comment-14745704 ] ASF GitHub Bot commented on STORM-1012: --- Github user hmcl commented on the pull requ

[GitHub] storm pull request: STORM-1012: Shading jackson.

Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/702#issuecomment-140458502 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

Documentation of release restriction in bin/storm.py

Hi, I would like to hack a bit on `bin/storm.py` and was wondering where the restriction to run this script in the source (root) is documented (more exact the failure with `The storm client can only be run from within a release. You appear to be trying to run the client from a checkout of Storm's s

[CANCELED] [VOTE] Release Apache Storm 0.10.0

Canceling this vote since I missed some critical issues. Will cut a new release once the issues are resolved and merged. -Taylor > On Sep 12, 2015, at 11:32 AM, Harsha wrote: > > Taylor, > STORM-1039 & STORM-1042 are blockers without these UI won't > work. We need to includ

[jira] [Commented] (STORM-893) Resource Aware Scheduling

[ https://issues.apache.org/jira/browse/STORM-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745594#comment-14745594 ] Boyang Jerry Peng commented on STORM-893: - Initial implementation of the Resource A

Re: Storm multi-tenancy in 0.10 - is there any topology resource isolation?

Hello Eric, I am glad you are interested in my work on Resource Aware Scheduling in Storm.   I have a initial implementation of the code on my github: https://github.com/jerrypeng/storm/tree/opensource_ras I haven't made a pull request yet but I will make it soon.  We are actively improving and a

[GitHub] storm pull request: clean up much of the wording in SECURITY.md

Github user harshach commented on the pull request: https://github.com/apache/storm/pull/735#issuecomment-140409554 +1. Thanks @erikdw --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[jira] [Commented] (STORM-1045) Parse CREATE TABLE statements

[ https://issues.apache.org/jira/browse/STORM-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745446#comment-14745446 ] ASF GitHub Bot commented on STORM-1045: --- Github user d2r commented on a diff in the

[GitHub] storm pull request: STORM-1045. Parse CREATE TABLE statements.

Github user d2r commented on a diff in the pull request: https://github.com/apache/storm/pull/734#discussion_r39508506 --- Diff: external/sql/storm-sql-core/pom.xml~ --- @@ -0,0 +1,60 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchem

[jira] [Assigned] (STORM-1044) Setting dop to zero does not raise an error

[ https://issues.apache.org/jira/browse/STORM-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned STORM-1044: -- Assignee: Matthias J. Sax > Setting dop to zero does not raise an error > -

Re: Storm multi-tenancy in 0.10 - is there any topology resource isolation?

>From reading the SECURITY.md file in the storm code, it seems the answer is that the "resource limits" feature is not what I pictured. The Storm 0.10 "resource limiting" functionality provided by the MultitenantScheduler

Re: worker-launcher - there is now native code included in Storm?

Some of the docs were hard for me to unravel (e.g., the section on supervisor needing a headless user). But I tried my best! https://github.com/apache/storm/pull/735 - Erik On Mon, Sep 14, 2015 at 11:21 PM, Erik Weathers wrote: > Thanks for the response Harsha! > > Here's the direct link to t

[GitHub] storm pull request: clean up much of the wording in SECURITY.md

GitHub user erikdw opened a pull request: https://github.com/apache/storm/pull/735 clean up much of the wording in SECURITY.md You can merge this pull request into a Git repository by running: $ git pull https://github.com/erikdw/storm SECURITY.md-cleanup Alternatively you ca

[jira] [Commented] (STORM-1041) Topology with kafka spout stops processing

[ https://issues.apache.org/jira/browse/STORM-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745013#comment-14745013 ] Abhishek Agarwal commented on STORM-1041: - Can you take the stack trace after the