[GitHub] storm issue #2133: Add the option to set client.id to storm-kafka, STORM-252...
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2133 See original PR, https://github.com/apache/storm/pull/2126 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2133: Add the option to set client.id to storm-kafka, ST...
Github user carl34 closed the pull request at: https://github.com/apache/storm/pull/2133 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2126: Add the option to set client.id to storm-kafka
GitHub user carl34 reopened a pull request: https://github.com/apache/storm/pull/2126 Add the option to set client.id to storm-kafka Add the ability to set the Kafka client.id property to storm-kafka and storm-kafka-client (https://issues.apache.org/jira/browse/STORM-2524). storm-kafka example: ``` SpoutConfig spoutConfig = new SpoutConfig( brokers, topicConfs.topic, + "client_id-" + confs.consumerGroupId + "-" + topicConfs.topic, "/consumers", confs.consumerGroupId + "-" + topicConfs.topic ); ``` storm-kafka-client example: ``` KafkaSpoutConfigkafkaSpoutConfig = KafkaSpoutConfig.builder(KAFKA_LOCAL_BROKER, TOPIC) .setGroupId(confs.consumerGroupId + "-" + topicConfs.topic) + .setClientId("client_id-" + confs.consumerGroupId + "-" + topicConfs.topic) .build(); ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/carl34/storm master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2126.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2126 commit 26bcfdbef95c4aecbff8aee0a587281f59b55bcf Author: Carl Haferd Date: 2017-05-22T23:09:19Z Add the option to set client.id to storm-kafka, STORM-2524 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2133: Add the option to set client.id to storm-kafka, STORM-252...
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2133 @revans2, please let me know if I need to make any more adjustments for this to be merged in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2133: Add the option to set client.id to storm-kafka, STORM-252...
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2133 See https://github.com/apache/storm/pull/2126 for review details, re-creating second time (first attempt was https://github.com/apache/storm/pull/2132) to resolve CHANGELOG.md conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2133: Add the option to set client.id to storm-kafka, ST...
GitHub user carl34 opened a pull request: https://github.com/apache/storm/pull/2133 Add the option to set client.id to storm-kafka, STORM-2524 Add the ability to set the Kafka client.id property to storm-kafka and storm-kafka-client (https://issues.apache.org/jira/browse/STORM-2524). storm-kafka example: ``` SpoutConfig spoutConfig = new SpoutConfig( brokers, topicConfs.topic, + "client_id-" + confs.consumerGroupId + "-" + topicConfs.topic, "/consumers", confs.consumerGroupId + "-" + topicConfs.topic ); ``` storm-kafka-client example: ``` KafkaSpoutConfigkafkaSpoutConfig = KafkaSpoutConfig.builder(KAFKA_LOCAL_BROKER, TOPIC) .setGroupId(confs.consumerGroupId + "-" + topicConfs.topic) + .setClientId("client_id-" + confs.consumerGroupId + "-" + topicConfs.topic) .build(); ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/carl34/storm master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2133 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2132: Add the option to set client.id to storm-kafka
Github user carl34 closed the pull request at: https://github.com/apache/storm/pull/2132 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2050: STORM-2448: Add in Storm and JDK versions when submitting...
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/2050 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2051: STORM-2448: Add in Storm and JDK versions when submitting...
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/2051 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2132: Add the option to set client.id to storm-kafka
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2132 See https://github.com/apache/storm/pull/2126, attempting to resolve CHANGELOG.md merge conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2132: Add the option to set client.id to storm-kafka
GitHub user carl34 opened a pull request: https://github.com/apache/storm/pull/2132 Add the option to set client.id to storm-kafka Add the ability to set the Kafka client.id property to storm-kafka and storm-kafka-client (https://issues.apache.org/jira/browse/STORM-2524). storm-kafka example: ``` SpoutConfig spoutConfig = new SpoutConfig( brokers, topicConfs.topic, + "client_id-" + confs.consumerGroupId + "-" + topicConfs.topic, "/consumers", confs.consumerGroupId + "-" + topicConfs.topic ); ``` storm-kafka-client example: ``` KafkaSpoutConfigkafkaSpoutConfig = KafkaSpoutConfig.builder(KAFKA_LOCAL_BROKER, TOPIC) .setGroupId(confs.consumerGroupId + "-" + topicConfs.topic) + .setClientId("client_id-" + confs.consumerGroupId + "-" + topicConfs.topic) .build(); ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/carl34/storm master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2132.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2132 commit ca50f699a4f7b50ef2f463ed076297abeabd321a Author: Carl Haferd Date: 2017-05-17T20:04:53Z Add the option to set client.id to storm-kafka commit b26ff87e6c5dc562502a21335e96876162ee77be Author: Robert Evans Date: 2017-05-22T17:08:25Z Add the option to set client.id to storm-kafka, STORM-2524 commit 1a895950aa2604d76cd00eecf8973913f92239c4 Author: carl34 Date: 2017-05-22T22:04:32Z Merge pull request #4 from carl34/kafka_client_id Add the option to set client.id to storm-kafka, STORM-2524 commit 275d20d07930e6b5901fb082dd0b678abd53904b Author: Carl Haferd Date: 2017-05-22T22:06:58Z Merge branch 'master' of https://github.com/carl34/storm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2126: Add the option to set client.id to storm-kafka
Github user carl34 closed the pull request at: https://github.com/apache/storm/pull/2126 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2126: Add the option to set client.id to storm-kafka
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2126 I can try rebasing again for the CHANGELOG.md updates, please let me know if I should add a new entry or to that file or if there are any other actions for me to take. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2049: STORM-2448: Add in Storm and JDK versions when sub...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2049 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2049: STORM-2448: Add in Storm and JDK versions when submitting...
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/2049 +1 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2129: [STORM-2206] - initial checkin of visualization using vis...
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/2129 Thanks a lot for this. It looks really good in general. One thing I noticed that could use some improvement is the bolt-info dropdown. The info can run off the window with no option to scroll. It seems like the scroll bar doesn't show up until the info is well out of view, and when it does show up, I can't scroll enough to bring everything back in view. Two more minor things: 1. When clicking on a spout, the info window still says "Bolt Info" 2. It's not exactly clear what the things labeled "Up:" are. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2126: Add the option to set client.id to storm-kafka and storm-...
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2126 @priyank5485 sounds good, I went ahead and removed the storm-kafka-client change so that only storm-kafka is updated and we will plan to use setProp. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2126: Add the option to set client.id to storm-kafka and storm-...
Github user carl34 commented on the issue: https://github.com/apache/storm/pull/2126 Added javadocs and rebased --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2126: Add the option to set client.id to storm-kafka and...
GitHub user carl34 reopened a pull request: https://github.com/apache/storm/pull/2126 Add the option to set client.id to storm-kafka and storm-kafka-client Add the ability to set the Kafka client.id property to storm-kafka and storm-kafka-client (https://issues.apache.org/jira/browse/STORM-2524). storm-kafka example: ``` SpoutConfig spoutConfig = new SpoutConfig( brokers, topicConfs.topic, + "client_id-" + confs.consumerGroupId + "-" + topicConfs.topic, "/consumers", confs.consumerGroupId + "-" + topicConfs.topic ); ``` storm-kafka-client example: ``` KafkaSpoutConfigkafkaSpoutConfig = KafkaSpoutConfig.builder(KAFKA_LOCAL_BROKER, TOPIC) .setGroupId(confs.consumerGroupId + "-" + topicConfs.topic) + .setClientId("client_id-" + confs.consumerGroupId + "-" + topicConfs.topic) .build(); ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/carl34/storm master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2126.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2126 commit ca50f699a4f7b50ef2f463ed076297abeabd321a Author: Carl Haferd Date: 2017-05-17T20:04:53Z Add the option to set client.id to storm-kafka commit cb543c0393d23ae93c0394697d5a555037cee486 Author: carl34 Date: 2017-05-22T20:26:25Z Merge pull request #3 from carl34/kafka_client_id Add the option to set client.id to storm-kafka --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2131: Add the option to set client.id to storm-kafka
Github user carl34 closed the pull request at: https://github.com/apache/storm/pull/2131 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2131: Add the option to set client.id to storm-kafka
GitHub user carl34 opened a pull request: https://github.com/apache/storm/pull/2131 Add the option to set client.id to storm-kafka You can merge this pull request into a Git repository by running: $ git pull https://github.com/carl34/storm kafka_client_id Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2131.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2131 commit ca50f699a4f7b50ef2f463ed076297abeabd321a Author: Carl HaferdDate: 2017-05-17T20:04:53Z Add the option to set client.id to storm-kafka --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2126: Add the option to set client.id to storm-kafka and...
Github user carl34 closed the pull request at: https://github.com/apache/storm/pull/2126 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2129: [STORM-2206] - initial checkin of visualization us...
Github user Crim commented on a diff in the pull request: https://github.com/apache/storm/pull/2129#discussion_r117833868 --- Diff: storm-core/src/ui/public/js/visualization.js --- @@ -16,434 +16,330 @@ * limitations under the License. */ -// Inspired by -// https://github.com/samizdatco/arbor/blob/master/docs/sample-project/main.js - -function renderGraph(elem) { - -var canvas = $(elem).get(0); -canvas.width = $(window).width(); -canvas.height = $(window).height(); -var ctx = canvas.getContext("2d"); -var gfx = arbor.Graphics(canvas); -var psys; - -var totaltrans = 0; -var weights = {}; -var texts = {}; -var update = false; - -var myRenderer = { -init: function(system){ -psys = system; -psys.screenSize(canvas.width, canvas.height) -psys.screenPadding(20); -myRenderer.initMouseHandling(); -}, - -signal_update: function() { -update = true; -}, - -redraw: function() { - -if(!psys) -return; - -if(update) { -totaltrans = calculate_total_transmitted(psys); -weights = calculate_weights(psys, totaltrans); -texts = calculate_texts(psys, totaltrans); -update = false; +var _showSystem = false; +var _showAcker = false; +var _showMetrics = false; +var container; + +var options = { +edges:{ +arrows: { +to: {enabled: true, scaleFactor:1} +}, +hoverWidth: 1.5, +shadow:{ +enabled: true, +color: 'rgba(0,0,0,0.5)', +size:10, +x:5, +y:5 +} +}, +nodes: { +color: { +border: '#2B7CE9', +background: '#97C2FC', +highlight: { +border: '#2B7CE9', +background: '#D2E5FF' +}, +hover: { +border: '#2B7CE9', +background: '#D2E5FF' +} +}, +shadow:{ + enabled: true, + color: 'rgba(0,0,0,0.5)', + size:10, + x:5, + y:5 +}, +}, +physics:{ +enabled: false +}, +layout: { +randomSeed: 31337, +improvedLayout:true, +hierarchical: { +enabled: true, +levelSeparation: 150, +nodeSpacing: 300, +treeSpacing: 200, +blockShifting: true, +edgeMinimization: true, +parentCentralization: true, +direction: 'UD',// UD, DU, LR, RL +sortMethod: 'directed' // hubsize, directed +} +}, +interaction: { +navigationButtons: false } +}; +// Holds all stream names +var availableStreamsHash = { } +// Holds our network +var network; -ctx.fillStyle = "white"; -ctx.fillRect(0, 0, canvas.width, canvas.height); -var x = 0; - +// Holds nodes and edge definitions +var nodes = new vis.DataSet(); +var edges = new vis.DataSet(); -psys.eachEdge(function(edge, pt1, pt2) { +// Update/refresh settings +var should_update = true +var update_freq_ms = 3 -var len = Math.sqrt(Math.pow(pt2.x - pt1.x,2) + Math.pow(pt2.y - pt1.y,2)); -var sublen = len - (Math.max(50, 20 + gfx.textWidth(edge.target.name)) / 2); -var thirdlen = len/3; -var theta = Math.atan2(pt2.y - pt1.y, pt2.x - pt1.x); - -var newpt2 = { -x : pt1.x + (Math.cos(theta) * sublen), -y : pt1.y + (Math.sin(theta) * sublen) -}; +function parseResponse(json) { +console.log("Updating network");
[GitHub] storm issue #2130: STORM-2526: Revert changes mistakenly made to generated f...
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/2130 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2129: [STORM-2206] - initial checkin of visualization us...
Github user revans2 commented on a diff in the pull request: https://github.com/apache/storm/pull/2129#discussion_r117826289 --- Diff: storm-core/src/ui/public/js/visualization.js --- @@ -16,434 +16,330 @@ * limitations under the License. */ -// Inspired by -// https://github.com/samizdatco/arbor/blob/master/docs/sample-project/main.js - -function renderGraph(elem) { - -var canvas = $(elem).get(0); -canvas.width = $(window).width(); -canvas.height = $(window).height(); -var ctx = canvas.getContext("2d"); -var gfx = arbor.Graphics(canvas); -var psys; - -var totaltrans = 0; -var weights = {}; -var texts = {}; -var update = false; - -var myRenderer = { -init: function(system){ -psys = system; -psys.screenSize(canvas.width, canvas.height) -psys.screenPadding(20); -myRenderer.initMouseHandling(); -}, - -signal_update: function() { -update = true; -}, - -redraw: function() { - -if(!psys) -return; - -if(update) { -totaltrans = calculate_total_transmitted(psys); -weights = calculate_weights(psys, totaltrans); -texts = calculate_texts(psys, totaltrans); -update = false; +var _showSystem = false; +var _showAcker = false; +var _showMetrics = false; +var container; + +var options = { +edges:{ +arrows: { +to: {enabled: true, scaleFactor:1} +}, +hoverWidth: 1.5, +shadow:{ +enabled: true, +color: 'rgba(0,0,0,0.5)', +size:10, +x:5, +y:5 +} +}, +nodes: { +color: { +border: '#2B7CE9', +background: '#97C2FC', +highlight: { +border: '#2B7CE9', +background: '#D2E5FF' +}, +hover: { +border: '#2B7CE9', +background: '#D2E5FF' +} +}, +shadow:{ + enabled: true, + color: 'rgba(0,0,0,0.5)', + size:10, + x:5, + y:5 +}, +}, +physics:{ +enabled: false +}, +layout: { +randomSeed: 31337, +improvedLayout:true, +hierarchical: { +enabled: true, +levelSeparation: 150, +nodeSpacing: 300, +treeSpacing: 200, +blockShifting: true, +edgeMinimization: true, +parentCentralization: true, +direction: 'UD',// UD, DU, LR, RL +sortMethod: 'directed' // hubsize, directed +} +}, +interaction: { +navigationButtons: false } +}; +// Holds all stream names +var availableStreamsHash = { } +// Holds our network +var network; -ctx.fillStyle = "white"; -ctx.fillRect(0, 0, canvas.width, canvas.height); -var x = 0; - +// Holds nodes and edge definitions +var nodes = new vis.DataSet(); +var edges = new vis.DataSet(); -psys.eachEdge(function(edge, pt1, pt2) { +// Update/refresh settings +var should_update = true +var update_freq_ms = 3 -var len = Math.sqrt(Math.pow(pt2.x - pt1.x,2) + Math.pow(pt2.y - pt1.y,2)); -var sublen = len - (Math.max(50, 20 + gfx.textWidth(edge.target.name)) / 2); -var thirdlen = len/3; -var theta = Math.atan2(pt2.y - pt1.y, pt2.x - pt1.x); - -var newpt2 = { -x : pt1.x + (Math.cos(theta) * sublen), -y : pt1.y + (Math.sin(theta) * sublen) -}; +function parseResponse(json) { +console.log("Updating
[GitHub] storm pull request #2129: [STORM-2206] - initial checkin of visualization us...
Github user revans2 commented on a diff in the pull request: https://github.com/apache/storm/pull/2129#discussion_r117822198 --- Diff: storm-core/src/ui/public/js/vis.min.js --- @@ -0,0 +1,45 @@ +/** + * vis.js + * https://github.com/almende/vis + * + * A dynamic, browser-based visualization library. + * + * @version 4.16.1 + * @date2016-04-18 + * + * @license + * Copyright (C) 2011-2016 Almende B.V, http://almende.com + * + * Vis.js is dual licensed under both + * + * * The Apache 2.0 License + * http://www.apache.org/licenses/LICENSE-2.0 + * + * and + * + * * The MIT License + * http://opensource.org/licenses/MIT + * + * Vis.js may be distributed under either license. --- End diff -- With this we need to make sure that we update LICENSE.txt to include this file and the license that we pick for it. We probably also want to white list this and vis.min.css for rat, so it stops complaining about them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2090: STORM-2489: Overlap and data loss on WindowedBolt ...
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2090#discussion_r117826059 --- Diff: storm-client/src/jvm/org/apache/storm/windowing/TimeTriggerPolicy.java --- @@ -62,7 +62,9 @@ public void reset() { @Override public void start() { -executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), duration, duration, TimeUnit.MILLISECONDS); +// initial delay is slightly less than the duration so that the initial tuples wont't expire due to time drift +long initialDelay = duration - Math.min((long) (duration * .1), 10); +executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), initialDelay, duration, TimeUnit.MILLISECONDS); --- End diff -- Also since this is likely going to rely on in-bolt state to avoid expiring tuples before the user sees them, I think we should add to the documentation that this set of policies don't guarantee at-least-once processing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2129: [STORM-2206] - initial checkin of visualization using vis...
Github user Crim commented on the issue: https://github.com/apache/storm/pull/2129 Oh yea I should have mentioned that. I had to go thru and fix that to get my local build working off of master :p --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2090: STORM-2489: Overlap and data loss on WindowedBolt based o...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2090 I'm wondering if you guys with some experience with windowing would mind taking a look at the question posted in https://github.com/apache/storm/pull/2127? It also has to do with tuples possibly getting expired before they're passed to the user's code, and affects the same policies as this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2090: STORM-2489: Overlap and data loss on WindowedBolt ...
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2090#discussion_r117821993 --- Diff: storm-client/src/jvm/org/apache/storm/windowing/TimeTriggerPolicy.java --- @@ -62,7 +62,9 @@ public void reset() { @Override public void start() { -executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), duration, duration, TimeUnit.MILLISECONDS); +// initial delay is slightly less than the duration so that the initial tuples wont't expire due to time drift +long initialDelay = duration - Math.min((long) (duration * .1), 10); +executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), initialDelay, duration, TimeUnit.MILLISECONDS); --- End diff -- Won't this still be vulnerable to losing initial tuples if the first execution is delayed longer than expected for some reason? I'm wondering if we could make this more reliable by tracking the time we consider expired instead of/in addition to tracking what time it is now. Tuples between the prevExpired timestamp and the nowExpired timestamp are tuples we can't be sure have been processed, so they should be included in the window. Since the expired timestamps also move ~duration every onTrigger, we still get the tuples expired we expect. For the first onTrigger, we can just process all the tuples because we can tell it's the first call since the prevExpired timestamp isn't set yet. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2129: [STORM-2206] - initial checkin of visualization using vis...
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2129 @Crim Don't feel bad. When I was playing around with this I found (https://issues.apache.org/jira/browse/STORM-2526) which was my own face palm moment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2130: STORM-2526: Revert changes mistakenly made to generated f...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2130 +1, opened Storm UI and clicked around a bit. It seems to work now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2100: STORM-2503: Fix lgtm.com alerts on equality and co...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2129: [STORM-2206] - initial checkin of visualization using vis...
Github user Crim commented on the issue: https://github.com/apache/storm/pull/2129 @revans2 You're right! (facepalm) Updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2125: nothing but fix one word
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2125 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2125: nothing but fix one word
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2125 Thanks @lijiansong I merged this in keep up the good work. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2126: Add the option to set client.id to storm-kafka and storm-...
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2126 Also please upmerge there is a merge conflict. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2126: Add the option to set client.id to storm-kafka and...
Github user revans2 commented on a diff in the pull request: https://github.com/apache/storm/pull/2126#discussion_r117774461 --- Diff: external/storm-kafka/pom.xml --- @@ -57,7 +57,7 @@ maven-checkstyle-plugin -557 +558 --- End diff -- I would prefer to see the violations fixed instead of adding in new ones. In this case I suspect that it is missing javadocs for the new SpoutConfig constructor. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2129: [STORM-2206] - initial checkin of visualization using vis...
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2129 My biggest concern right now it that it looks like you missed checking in a few files. I see `vis.min.js`, `vis.min.css`, `streams.png`, and `bolt_info.png` all getting a 404 when I try to use it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #2130: STORM-2526: Revert changes mistakenly made to gene...
GitHub user revans2 opened a pull request: https://github.com/apache/storm/pull/2130 STORM-2526: Revert changes mistakenly made to generated files I regenerated all of the generated thrift files to be sure I hadn't mistakenly changed anything else. Then I updated the one call to those APIs in the java code. You can merge this pull request into a Git repository by running: $ git pull https://github.com/revans2/incubator-storm STORM-2526 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2130.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2130 commit 185101dd8cd4955e37279670d8ae3f61e43f37a6 Author: Robert (Bobby) EvansDate: 2017-05-22T15:02:48Z STORM-2526: Revert changes mistakenly made to generated files --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #2083: STORM-2421: support lists of childopts in DaemonConfig.
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2083 @hmcc looks like there is a merge conflict now. If you could resolve it I would be happy to check this in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Updating APIs to fix checkstyle violations
As this is for a change going into 2.x I totally agree lets just change the API. If we really want to do a good job we can also have a pull request in the 1.x version that adds in the new API and deprecates the old one, so users are not caught off guard as much. - Bobby On Sunday, May 21, 2017, 6:11:19 AM CDT, Jungtaek Limwrote: I prefer to not have too much exceptions from chosen style guide. So I'm OK to change the method to setSslKeystore, but just would like to hear others voices because we're breaking backward compatibility once again (we did it at 1.1.0) just because of respecting code style. - Jungtaek Lim (HeartSaVioR) 2017년 5월 21일 (일) 오후 8:07, Stig Rohde Døssing 님이 작성: > Hi, > > As part of fixing the checkstyle violations on storm-kafka-client, we'd > need to rename some methods on KafkaSpoutConfig. See > https://github.com/apache/storm/pull/2117. Does anyone have an opinion on > when/if it is acceptable to break the API for this reason? > > I'd also like opinions on whether method names like setSslKeystore are > better than setSSLKeystore? The current checkstyle rules prohibit adjacent > capital letters in non-final names due to this bit of the checkstyle > config. > > > > > >