[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

[arunmahadevan]

Github user arunmahadevan commented on the issue:

https://github.com/apache/storm/pull/2218
  
Thanks for the reviews, squashed and merged to master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2218: STORM-2614: Enhance stateful windowing to persist ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/2218


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2263: STORM-2680 : The switch to turn on-off the cgroup in the ...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2263
  
+1 again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2263: STORM-2680 : The switch to turn on-off the cgroup in the ...

[cluo512]

Github user cluo512 commented on the issue:

https://github.com/apache/storm/pull/2263
  
@HeartSaVioR Thank you.
 I rebased.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2181: [STORM-2607] Offset consumer + 1

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2181
  
@tiodollar Any updates?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2263: STORM-2680 : The switch to turn on-off the cgroup in the ...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2263
  
+1

Could you rebase into current master? I recommend you to create a new 
branch for each pull request.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2218
  
@arunmahadevan
+1 Nice work.
Could you squash the commits into one? You can merge it by yourself, or 
please mention me if you want me to merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131789023
  
--- Diff: storm-client/src/jvm/org/apache/storm/security/auth/AutoSSL.java 
---
@@ -133,8 +134,9 @@ public static void deserializeSSLFile(String credsKey, 
String directory,
 if (resultStr != null) {
 byte[] decodedData = 
DatatypeConverter.parseBase64Binary(resultStr);
 File f = new File(directory, credsKey);
-FileOutputStream fout = new FileOutputStream(f);
-fout.write(decodedData);
+try(FileOutputStream fout = new FileOutputStream(f)) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131787411
  
--- Diff: 
external/storm-jdbc/src/main/java/org/apache/storm/jdbc/common/JdbcClient.java 
---
@@ -56,25 +56,26 @@ public void executeInsertQuery(String query, 
List columnLists) {
 
 LOG.debug("Executing query {}", query);
 
-PreparedStatement preparedStatement = 
connection.prepareStatement(query);
-if(queryTimeoutSecs > 0) {
-preparedStatement.setQueryTimeout(queryTimeoutSecs);
-}
+try(PreparedStatement preparedStatement = 
connection.prepareStatement(query)) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131787582
  
--- Diff: 
external/storm-jdbc/src/main/java/org/apache/storm/jdbc/common/JdbcClient.java 
---
@@ -106,50 +107,52 @@ public String apply(Column input) {
 Connection connection = null;
 try {
 connection = connectionProvider.getConnection();
-PreparedStatement preparedStatement = 
connection.prepareStatement(sqlQuery);
-if(queryTimeoutSecs > 0) {
-preparedStatement.setQueryTimeout(queryTimeoutSecs);
-}
-setPreparedStatementParams(preparedStatement, queryParams);
-ResultSet resultSet = preparedStatement.executeQuery();
-List rows = Lists.newArrayList();
-while(resultSet.next()){
-ResultSetMetaData metaData = resultSet.getMetaData();
-int columnCount = metaData.getColumnCount();
-List row = Lists.newArrayList();
-for(int i=1 ; i <= columnCount; i++) {
-String columnLabel = metaData.getColumnLabel(i);
-int columnType = metaData.getColumnType(i);
-Class columnJavaType = Util.getJavaType(columnType);
-if (columnJavaType.equals(String.class)) {
-row.add(new Column(columnLabel, 
resultSet.getString(columnLabel), columnType));
-} else if (columnJavaType.equals(Integer.class)) {
-row.add(new Column(columnLabel, 
resultSet.getInt(columnLabel), columnType));
-} else if (columnJavaType.equals(Double.class)) {
-row.add(new Column(columnLabel, 
resultSet.getDouble(columnLabel), columnType));
-} else if (columnJavaType.equals(Float.class)) {
-row.add(new Column(columnLabel, 
resultSet.getFloat(columnLabel), columnType));
-} else if (columnJavaType.equals(Short.class)) {
-row.add(new Column(columnLabel, 
resultSet.getShort(columnLabel), columnType));
-} else if (columnJavaType.equals(Boolean.class)) {
-row.add(new Column(columnLabel, 
resultSet.getBoolean(columnLabel), columnType));
-} else if (columnJavaType.equals(byte[].class)) {
-row.add(new Column(columnLabel, 
resultSet.getBytes(columnLabel), columnType));
-} else if (columnJavaType.equals(Long.class)) {
-row.add(new Column(columnLabel, 
resultSet.getLong(columnLabel), columnType));
-} else if (columnJavaType.equals(Date.class)) {
-row.add(new Column(columnLabel, 
resultSet.getDate(columnLabel), columnType));
-} else if (columnJavaType.equals(Time.class)) {
-row.add(new Column(columnLabel, 
resultSet.getTime(columnLabel), columnType));
-} else if (columnJavaType.equals(Timestamp.class)) {
-row.add(new Column(columnLabel, 
resultSet.getTimestamp(columnLabel), columnType));
-} else {
-throw new RuntimeException("type =  " + columnType 
+ " for column " + columnLabel + " not supported.");
+try(PreparedStatement preparedStatement = 
connection.prepareStatement(sqlQuery)) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131788092
  
--- Diff: 
external/storm-jdbc/src/main/java/org/apache/storm/jdbc/common/JdbcClient.java 
---
@@ -179,8 +182,9 @@ public void executeSql(String sql) {
 Connection connection = null;
 try {
 connection = connectionProvider.getConnection();
-Statement statement = connection.createStatement();
-statement.execute(sql);
+try(Statement statement = connection.createStatement()) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131788989
  
--- Diff: storm-client/src/jvm/org/apache/storm/security/auth/AutoSSL.java 
---
@@ -100,19 +100,20 @@ public void populateCredentials(Map 
credentials) {
 // the key.
 public static void serializeSSLFile(String readFile, Map credentials) {
 try {
-FileInputStream in = new FileInputStream(readFile);
-LOG.debug("serializing ssl file: {}", readFile);
-ByteArrayOutputStream result = new ByteArrayOutputStream();
-byte[] buffer = new byte[4096];
-int length;
-while ((length = in.read(buffer)) != -1) {
-result.write(buffer, 0, length);
-}
-String resultStr = 
DatatypeConverter.printBase64Binary(result.toByteArray());
+try(FileInputStream in = new FileInputStream(readFile)) {
--- End diff --

space between `try` and `(`, and let's not make it unnecessary nested. It 
can be flattened.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131788473
  
--- Diff: 
flux/flux-core/src/main/java/org/apache/storm/flux/parser/FluxParser.java ---
@@ -96,11 +96,12 @@ private static TopologyDef loadYaml(Yaml yaml, 
InputStream in, String propsFile,
 // properties file substitution
 if(propsFile != null){
 LOG.info("Performing property substitution.");
-InputStream propsIn = new FileInputStream(propsFile);
-Properties props = new Properties();
-props.load(propsIn);
-for(Object key : props.keySet()){
-str = str.replace("${" + key + "}", 
props.getProperty((String)key));
+try(InputStream propsIn = new FileInputStream(propsFile)) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2262: STORM-2679: Need to close resources and streams af...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2262#discussion_r131787609
  
--- Diff: 
external/storm-jdbc/src/main/java/org/apache/storm/jdbc/common/JdbcClient.java 
---
@@ -106,50 +107,52 @@ public String apply(Column input) {
 Connection connection = null;
 try {
 connection = connectionProvider.getConnection();
-PreparedStatement preparedStatement = 
connection.prepareStatement(sqlQuery);
-if(queryTimeoutSecs > 0) {
-preparedStatement.setQueryTimeout(queryTimeoutSecs);
-}
-setPreparedStatementParams(preparedStatement, queryParams);
-ResultSet resultSet = preparedStatement.executeQuery();
-List rows = Lists.newArrayList();
-while(resultSet.next()){
-ResultSetMetaData metaData = resultSet.getMetaData();
-int columnCount = metaData.getColumnCount();
-List row = Lists.newArrayList();
-for(int i=1 ; i <= columnCount; i++) {
-String columnLabel = metaData.getColumnLabel(i);
-int columnType = metaData.getColumnType(i);
-Class columnJavaType = Util.getJavaType(columnType);
-if (columnJavaType.equals(String.class)) {
-row.add(new Column(columnLabel, 
resultSet.getString(columnLabel), columnType));
-} else if (columnJavaType.equals(Integer.class)) {
-row.add(new Column(columnLabel, 
resultSet.getInt(columnLabel), columnType));
-} else if (columnJavaType.equals(Double.class)) {
-row.add(new Column(columnLabel, 
resultSet.getDouble(columnLabel), columnType));
-} else if (columnJavaType.equals(Float.class)) {
-row.add(new Column(columnLabel, 
resultSet.getFloat(columnLabel), columnType));
-} else if (columnJavaType.equals(Short.class)) {
-row.add(new Column(columnLabel, 
resultSet.getShort(columnLabel), columnType));
-} else if (columnJavaType.equals(Boolean.class)) {
-row.add(new Column(columnLabel, 
resultSet.getBoolean(columnLabel), columnType));
-} else if (columnJavaType.equals(byte[].class)) {
-row.add(new Column(columnLabel, 
resultSet.getBytes(columnLabel), columnType));
-} else if (columnJavaType.equals(Long.class)) {
-row.add(new Column(columnLabel, 
resultSet.getLong(columnLabel), columnType));
-} else if (columnJavaType.equals(Date.class)) {
-row.add(new Column(columnLabel, 
resultSet.getDate(columnLabel), columnType));
-} else if (columnJavaType.equals(Time.class)) {
-row.add(new Column(columnLabel, 
resultSet.getTime(columnLabel), columnType));
-} else if (columnJavaType.equals(Timestamp.class)) {
-row.add(new Column(columnLabel, 
resultSet.getTimestamp(columnLabel), columnType));
-} else {
-throw new RuntimeException("type =  " + columnType 
+ " for column " + columnLabel + " not supported.");
+try(PreparedStatement preparedStatement = 
connection.prepareStatement(sqlQuery)) {
+if (queryTimeoutSecs > 0) {
+preparedStatement.setQueryTimeout(queryTimeoutSecs);
+}
+setPreparedStatementParams(preparedStatement, queryParams);
+try(ResultSet resultSet = 
preparedStatement.executeQuery()) {
--- End diff --

space between `try` and `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2265: [STORM-2674] catch NoNodeException when IStormClusterStat...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2265
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2267: STORM-2682: Fix issues with null owner on rolling ...

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2267

STORM-2682: Fix issues with null owner on rolling upgrade (1.0.x)

This is the 1.0.x version of  #2266 and should probably apply cleanly to 
all of the 1.x line of releases.

This prevents doing a clean rolling upgrade from 1.0.3 to 1.0.4 and 1.1.0 
to 1.1.1.  We should look at getting this in and possibly doing another release 
ASAP.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm STORM-2682-1.0.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2267.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2267


commit 457bdf4457df241ebf5bb5a0cdfdd0d3ba85491d
Author: Robert (Bobby) Evans 
Date:   2017-08-07T20:44:08Z

STORM-2682: Fix issues with null owner on rolling upgrade




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2266: STORM-2682: Fixed issues with null owner

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2266

STORM-2682: Fixed issues with null owner



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm STORM-2682

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2266.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2266


commit 0d59f7cd326a3913a43aed45af5f73fd4f1994a4
Author: Robert (Bobby) Evans 
Date:   2017-08-07T20:47:00Z

STORM-2682: Fixed issues with null owner




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #:

[roshannaik]

Github user roshannaik commented on the pull request:


https://github.com/apache/storm/commit/96ad079354eba2c68a4e9854e14c584043e49e36#commitcomment-23523208
  
@Ethanlm @revans2  Is it OK to disable this for SystemBolt ? As far as I 
can tell, SystemBolt doesn't appear to invoke reportError().Asking since 
this is causing some issues with STORM-2306.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2262: STORM-2679: Need to close resources and streams after use

[srdo]

Github user srdo commented on the issue:

https://github.com/apache/storm/pull/2262
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2233: Storm 2258: Streams api - support CoGroupByKey

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/2233#discussion_r131659700
  
--- Diff: storm-client/src/jvm/org/apache/storm/streams/PairStream.java ---
@@ -380,6 +383,22 @@
 return partitionBy(KEY).updateStateByKeyPartition(stateUpdater);
 }
 
+/**
+ * group the values of this stream with the values having the same key 
from the other stream.
--- End diff --

group -> Groups


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2233: Storm 2258: Streams api - support CoGroupByKey

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/2233#discussion_r131680183
  
--- Diff: 
storm-client/src/jvm/org/apache/storm/streams/processors/CoGroupByKeyProcessor.java
 ---
@@ -0,0 +1,75 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.storm.streams.processors;
+
+import com.google.common.collect.ArrayListMultimap;
+import com.google.common.collect.Multimap;
+import org.apache.storm.streams.Pair;
+
+/**
+ * co-group by key implementation
+ */
+public class CoGroupByKeyProcessor extends BaseProcessor> implements BatchProcessor {
+private final String firstStream;
+private final String secondStream;
+private final Multimap firstMap = ArrayListMultimap.create();
+private final Multimap secondMap = ArrayListMultimap.create();
+
+
+public CoGroupByKeyProcessor(String firstStream, String secondStream) {
+this.firstStream = firstStream;
+this.secondStream = secondStream;
+}
+
+@Override
+public void execute(Pair input, String sourceStream) {
+K key = input.getFirst();
+if (sourceStream.equals(firstStream)) {
+V1 val = (V1) input.getSecond();
+firstMap.put(key, val);
+} else if (sourceStream.equals(secondStream)) {
+V2 val = (V2) input.getSecond();
+secondMap.put(key, val);
+}
+if (!context.isWindowed()) {
+forwardValues();
+}
+
+}
+
+@Override
+public void finish() {
+forwardValues();
+firstMap.clear();
+secondMap.clear();
+}
+
+private void forwardValues() {
+firstMap.asMap().forEach((key, values) -> {
+context.forward(Pair.of(key, Pair.of(values, 
secondMap.removeAll(key;
+});
+
+secondMap.asMap().forEach((key, values) -> {
+context.forward(Pair.of(key, Pair.of(firstMap.removeAll(key), 
values)));
--- End diff --

`firstMap.get(key)` doesn't work ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2233: Storm 2258: Streams api - support CoGroupByKey

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/2233#discussion_r131682043
  
--- Diff: 
storm-client/test/jvm/org/apache/storm/streams/processors/CoGroupByKeyProcessorTest.java
 ---
@@ -0,0 +1,107 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.streams.processors;
+
+import org.apache.storm.streams.Pair;
+import org.apache.storm.streams.operations.PairValueJoiner;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.Collection;
+import java.util.Arrays;
+
+
+import static org.junit.Assert.assertEquals;
+
+public class CoGroupByKeyProcessorTest {
+private CoGroupByKeyProcessor 
coGroupByKeyProcessor;
+private String firstStream = "first";
+private String secondStream = "second";
+private List>> res = 
new ArrayList<>();
+
+private ProcessorContext context = new ProcessorContext() {
+@Override
+public  void forward(T input) {
+res.add((Pair>)input);
+}
+
+@Override
+public  void forward(T input, String stream) {
+}
+
+@Override
+public boolean isWindowed() {
+return true;
+}
+
+@Override
+public Set getWindowedParentStreams() {
+return null;
+}
+};
+
+private List> firstKeyValeus = Arrays.asList(
+Pair.of(2, 4),
+Pair.of(5, 25),
+Pair.of(7, 49),
+Pair.of(7, 87)
+);
+
+private List> secondKeyValues = Arrays.asList(
+Pair.of(1, 1),
+Pair.of(2, 8),
+Pair.of(5, 125),
+Pair.of(5,50),
+Pair.of(6, 216)
+
+);
+
+@Test
+public void testCoGroupByKey() throws Exception {
+coGroupByKeyProcessor = new CoGroupByKeyProcessor<>(firstStream, 
secondStream);
+processValues();
+List>> result = new ArrayList<>();
+Collection list1 = new ArrayList<>();
+list1.add(25);
+Collection list2 = new ArrayList<>();
+list2.add(125);
+list2.add(50);
+result.add(Pair.of(5, Pair.of(list1, list2)));
+list1.clear();
+list2.clear();
+list1.add(49);
+list1.add(87);
+result.add(Pair.of(7, Pair.of(list1, list2)));
+}
--- End diff --

I cant find any assertions here. Are you checking if the result is as 
expected?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2233: Storm 2258: Streams api - support CoGroupByKey

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/2233#discussion_r131690416
  
--- Diff: storm-client/src/jvm/org/apache/storm/streams/PairStream.java ---
@@ -380,6 +383,22 @@
 return partitionBy(KEY).updateStateByKeyPartition(stateUpdater);
 }
 
+/**
+ * group the values of this stream with the values having the same key 
from the other stream.
--- End diff --

May be here and in the Streams API doc add a snippet to make it clear:

```
If stream1 has values - (k1, v1), (k2, v2), (k2, v3) 
and stream2 has values - (k1, x1), (k1, x2), (k3, x3)

The the co-grouped stream would contain -
(k1, ([v1], [x1, x2]), (k2, ([v2, v3], [])), (k3, ([], [x3]))
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2241: STORM-2306 : Messaging subsystem redesign.

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2241
  
FYI: Regarding improving LoadAwareShuffleGrouping, I spent some days and 
published my improvement on #2261. From my latest update, compared to current, 
#2261 is about 4x faster with single-thread benchmark, and 2x faster with 
two-threads benchmark.
I need to run same benchmarks against ShuffleGrouping but I guess #2261 
should be close to current ShuffleGrouping, based on fact that chooseTasks() 
now does the same thing which ShuffleGrouping is doing.
(ShuffleGrouping in current PR is optimized for single-thread and drops 
thread-safety, so can't compare them directly.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2261: STORM-2678 Improve performance of LoadAwareShuffleGroupin...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2261
  
I just take opposite approach, pushing updated load mapping when load 
updater timer is activated. We no longer need any tricks or optimizations to 
reduce checking, and even no need to check updating duration.

This is based on fact that `LoadAwareShuffleGrouping.updateRing()` doesn't 
break other threads calling `LoadAwareShuffleGrouping.chooseTasks()` 
concurrently.

I guess we couldn't optimize better easily unless we change some 
specifications like allowing non-thread-safety or so.

> testBenchmarkLoadAwareShuffleGroupingEvenLoad
Duration: 27596 ms
Duration: 27772 ms

> testBenchmarkLoadAwareShuffleGroupingEvenLoadAndMultiThreaded
Max duration among threads is : 89274 ms
Max duration among threads is : 86466 ms

Given that it changes some interfaces, I would like to see many reviewers 
reviewing and providing opinions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2265: [STORM-2674] catch NoNodeException when IStormClusterStat...

[Ethanlm]

Github user Ethanlm commented on the issue:

https://github.com/apache/storm/pull/2265
  
Error in storm-submit-tools: DependencyResolverTest

Should not be related to this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [Propose] move website repository from svn to git

[Bobby Evans]

Thanks for clarifying.
We have not run into an issue were we actually have needed to fix the docs for 
an already released version, not saying that it will not happen in the future 
though as the docs are not perfect.  But updating the docs to describe a 
feature that someone is working on is a much more common situation in my 
opinion, so if we do need to update docs for a release that we already did, 
then it probably would involve a lot of applying the same changes to a number 
of different places.


- Bobby


On Monday, August 7, 2017, 9:09:23 AM CDT, Jungtaek Lim  
wrote:

Maybe I was not clear. I meant removing 'releases' directory from
storm-site repo, not storm repo, given that we have duplicated docs between
twos. If we can get rid of that we can reduce the size greatly, but as I
stated from another thread, it may need to decouple main docs and release
docs.

One thing I'm considering is fixing doc for version which is already
released. Assuming we had to fix doc for 1.1.1 (already released), which
doc we need to fix? Only 1.x-branch and 1.1.x-branch in storm repo? Or we
need to store docs in releases directory as it is and fix it as well?

2017년 8월 7일 (월) 오후 10:52, Bobby Evans 님이 작성:

> Please don't remove the .md docs from the storm repo.  Those docs document
> the current build.  Things that are common and almost never change we can
> move to a separate repo.  But things that change from one release to
> another should stay, because they show what the current build is like, and
> having the docs in a separate repo from the code means that the code is
> going to change all the time and the docs are going to get out of date.
>
>
> - Bobby
>
>
> On Monday, August 7, 2017, 6:14:40 AM CDT, Jungtaek Lim 
> wrote:
>
> FYI: I've exported SVN repo of website and pushed to 'asf-site' branch of
> GIT repo.
>
> https://github.com/apache/storm-site
>
> Please note that 'content' directory will be used for serving website.
> (INFRA guided me to use this directory)
> Need to update README.md for the new instruction.
>
> Btw, this is still too way heavy, lots of files occupying huge space (2.3g)
> making git too way slow as same as svn. Better to reduce some more
> contents, maybe removing source docs (md files) which are also available in
> storm repo.
>
> - Jungtaek Lim (HeartSaVioR)
>
> 2017년 8월 7일 (월) 오전 11:55, Jungtaek Lim 님이 작성:
>
> > FYI: storm-site git repository is created, and according to notification
> > mail, github mirror will be created in a day.
> >
> > I filed another INFRA issues to associate website to the new repository.
> > https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-14810
> >
> > - Jungtaek Lim (HeartSaVioR)
> >
> > 2017년 8월 2일 (수) 오전 8:09, Jungtaek Lim 님이 작성:
> >
> >> FYI: I just take a step to this, but blocked at creating git repository
> >> in reporeq.apache.org.
> >>
> >> Just filed https://issues.apache.org/jira/browse/INFRA-14765. In that
> >> issue I also asked how to serve website with non-main project
> repository.
> >>
> >> 2017년 7월 31일 (월) 오후 10:56, Bobby Evans 님이
> >> 작성:
> >>
> >>> +1
> >>> I am fine with moving to git, but I would like it to be a different
> repo.
> >>> Our current repo is at least 160MB already (which is a lot to download)
> >>> but nothing compared the the web site that has lots and lots of things
> >>> checked in (I estimate it at about 1.5GB on an older version I have
> locally)
> >>>
> >>>
> >>> - Bobby
> >>>
> >>>
> >>> On Monday, July 31, 2017, 1:58:03 AM CDT, Xin Wang <
> >>> data.xinw...@gmail.com> wrote:
> >>>
> >>> +1 for moving to git.  - Xin
> >>>
> >>>
> >>>
> >>> 2017-07-31 14:54 GMT+08:00 Jungtaek Lim :
> >>>
> >>> > Bump. I think this is worth to address soon, since some contributors
> >>> > occasionally submit patches regarding documentations.
> >>> > Personally SVN is no longer feel convenient to use. If we all feel
> the
> >>> > same, let's change then.
> >>> >
> >>> > -Jungtaek Lim (HeartSaVioR)
> >>> >
> >>> > 2017년 7월 13일 (목) 오전 9:16, Jungtaek Lim 님이 작성:
> >>> >
> >>> > > Maybe we could try out Gitbox, though every committers should join
> >>> their
> >>> > > Github accounts to 'apache' group and enable 2FA.
> >>> > >
> >>> > > 2017년 7월 13일 (목) 오전 8:38, Jungtaek Lim 님이 작성:
> >>> > >
> >>> > >> Did we render webpage with asf-site branch? I didn't recognize it.
> >>> > >>
> >>> > >> Yes I meant separate git repository, like 'storm-site'. I'm happy
> >>> I'm
> >>> > not
> >>> > >> the only one who feels inconvenient with SVN repo.
> >>> > >> Would it better to initiate VOTE for this?
> >>> > >>
> >>> > >> Thanks,
> >>> > >> Jungtaek Lim (HeartSaVioR)
> >>> > >>
> >>> > >> 2017년 7월 13일 (목) 오전 4:30, P. Taylor Goetz 님이
> 작성:
> >>> > >>
> >>> > >>> We were using git before, then a year ago moved back to
> 

Re: [Propose] move website repository from svn to git

[Jungtaek Lim]

Maybe I was not clear. I meant removing 'releases' directory from
storm-site repo, not storm repo, given that we have duplicated docs between
twos. If we can get rid of that we can reduce the size greatly, but as I
stated from another thread, it may need to decouple main docs and release
docs.

One thing I'm considering is fixing doc for version which is already
released. Assuming we had to fix doc for 1.1.1 (already released), which
doc we need to fix? Only 1.x-branch and 1.1.x-branch in storm repo? Or we
need to store docs in releases directory as it is and fix it as well?

2017년 8월 7일 (월) 오후 10:52, Bobby Evans 님이 작성:

> Please don't remove the .md docs from the storm repo.  Those docs document
> the current build.  Things that are common and almost never change we can
> move to a separate repo.  But things that change from one release to
> another should stay, because they show what the current build is like, and
> having the docs in a separate repo from the code means that the code is
> going to change all the time and the docs are going to get out of date.
>
>
> - Bobby
>
>
> On Monday, August 7, 2017, 6:14:40 AM CDT, Jungtaek Lim 
> wrote:
>
> FYI: I've exported SVN repo of website and pushed to 'asf-site' branch of
> GIT repo.
>
> https://github.com/apache/storm-site
>
> Please note that 'content' directory will be used for serving website.
> (INFRA guided me to use this directory)
> Need to update README.md for the new instruction.
>
> Btw, this is still too way heavy, lots of files occupying huge space (2.3g)
> making git too way slow as same as svn. Better to reduce some more
> contents, maybe removing source docs (md files) which are also available in
> storm repo.
>
> - Jungtaek Lim (HeartSaVioR)
>
> 2017년 8월 7일 (월) 오전 11:55, Jungtaek Lim 님이 작성:
>
> > FYI: storm-site git repository is created, and according to notification
> > mail, github mirror will be created in a day.
> >
> > I filed another INFRA issues to associate website to the new repository.
> > https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-14810
> >
> > - Jungtaek Lim (HeartSaVioR)
> >
> > 2017년 8월 2일 (수) 오전 8:09, Jungtaek Lim 님이 작성:
> >
> >> FYI: I just take a step to this, but blocked at creating git repository
> >> in reporeq.apache.org.
> >>
> >> Just filed https://issues.apache.org/jira/browse/INFRA-14765. In that
> >> issue I also asked how to serve website with non-main project
> repository.
> >>
> >> 2017년 7월 31일 (월) 오후 10:56, Bobby Evans 님이
> >> 작성:
> >>
> >>> +1
> >>> I am fine with moving to git, but I would like it to be a different
> repo.
> >>> Our current repo is at least 160MB already (which is a lot to download)
> >>> but nothing compared the the web site that has lots and lots of things
> >>> checked in (I estimate it at about 1.5GB on an older version I have
> locally)
> >>>
> >>>
> >>> - Bobby
> >>>
> >>>
> >>> On Monday, July 31, 2017, 1:58:03 AM CDT, Xin Wang <
> >>> data.xinw...@gmail.com> wrote:
> >>>
> >>> +1 for moving to git.  - Xin
> >>>
> >>>
> >>>
> >>> 2017-07-31 14:54 GMT+08:00 Jungtaek Lim :
> >>>
> >>> > Bump. I think this is worth to address soon, since some contributors
> >>> > occasionally submit patches regarding documentations.
> >>> > Personally SVN is no longer feel convenient to use. If we all feel
> the
> >>> > same, let's change then.
> >>> >
> >>> > -Jungtaek Lim (HeartSaVioR)
> >>> >
> >>> > 2017년 7월 13일 (목) 오전 9:16, Jungtaek Lim 님이 작성:
> >>> >
> >>> > > Maybe we could try out Gitbox, though every committers should join
> >>> their
> >>> > > Github accounts to 'apache' group and enable 2FA.
> >>> > >
> >>> > > 2017년 7월 13일 (목) 오전 8:38, Jungtaek Lim 님이 작성:
> >>> > >
> >>> > >> Did we render webpage with asf-site branch? I didn't recognize it.
> >>> > >>
> >>> > >> Yes I meant separate git repository, like 'storm-site'. I'm happy
> >>> I'm
> >>> > not
> >>> > >> the only one who feels inconvenient with SVN repo.
> >>> > >> Would it better to initiate VOTE for this?
> >>> > >>
> >>> > >> Thanks,
> >>> > >> Jungtaek Lim (HeartSaVioR)
> >>> > >>
> >>> > >> 2017년 7월 13일 (목) 오전 4:30, P. Taylor Goetz 님이
> 작성:
> >>> > >>
> >>> > >>> We were using git before, then a year ago moved back to
> subversion
> >>> to
> >>> > >>> implement versioned documentation [1].
> >>> > >>>
> >>> > >>> If we do decide to move back to git for this, I would recommend
> >>> using a
> >>> > >>> separate git repository so it doesn’t bloat our main code
> >>> repository.
> >>> > When
> >>> > >>> generating javadoc for a new version, the svn commit to publish
> the
> >>> > site
> >>> > >>> can take around 20 minutes.
> >>> > >>>
> >>> > >>> -Taylor
> >>> > >>>
> >>> > >>> > On Jul 12, 2017, at 10:33 AM, Jungtaek Lim 
> >>> > wrote:
> >>> > >>> >
> >>> > >>> > Hi devs,
> >>> > >>> >
> >>> > >>> > 

Re: [Propose] move website repository from svn to git

[Bobby Evans]

Please don't remove the .md docs from the storm repo.  Those docs document the 
current build.  Things that are common and almost never change we can move to a 
separate repo.  But things that change from one release to another should stay, 
because they show what the current build is like, and having the docs in a 
separate repo from the code means that the code is going to change all the time 
and the docs are going to get out of date. 


- Bobby


On Monday, August 7, 2017, 6:14:40 AM CDT, Jungtaek Lim  
wrote:

FYI: I've exported SVN repo of website and pushed to 'asf-site' branch of
GIT repo.

https://github.com/apache/storm-site

Please note that 'content' directory will be used for serving website.
(INFRA guided me to use this directory)
Need to update README.md for the new instruction.

Btw, this is still too way heavy, lots of files occupying huge space (2.3g)
making git too way slow as same as svn. Better to reduce some more
contents, maybe removing source docs (md files) which are also available in
storm repo.

- Jungtaek Lim (HeartSaVioR)

2017년 8월 7일 (월) 오전 11:55, Jungtaek Lim 님이 작성:

> FYI: storm-site git repository is created, and according to notification
> mail, github mirror will be created in a day.
>
> I filed another INFRA issues to associate website to the new repository.
> https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-14810
>
> - Jungtaek Lim (HeartSaVioR)
>
> 2017년 8월 2일 (수) 오전 8:09, Jungtaek Lim 님이 작성:
>
>> FYI: I just take a step to this, but blocked at creating git repository
>> in reporeq.apache.org.
>>
>> Just filed https://issues.apache.org/jira/browse/INFRA-14765. In that
>> issue I also asked how to serve website with non-main project repository.
>>
>> 2017년 7월 31일 (월) 오후 10:56, Bobby Evans 님이
>> 작성:
>>
>>> +1
>>> I am fine with moving to git, but I would like it to be a different repo.
>>> Our current repo is at least 160MB already (which is a lot to download)
>>> but nothing compared the the web site that has lots and lots of things
>>> checked in (I estimate it at about 1.5GB on an older version I have locally)
>>>
>>>
>>> - Bobby
>>>
>>>
>>> On Monday, July 31, 2017, 1:58:03 AM CDT, Xin Wang <
>>> data.xinw...@gmail.com> wrote:
>>>
>>> +1 for moving to git.  - Xin
>>>
>>>
>>>
>>> 2017-07-31 14:54 GMT+08:00 Jungtaek Lim :
>>>
>>> > Bump. I think this is worth to address soon, since some contributors
>>> > occasionally submit patches regarding documentations.
>>> > Personally SVN is no longer feel convenient to use. If we all feel the
>>> > same, let's change then.
>>> >
>>> > -Jungtaek Lim (HeartSaVioR)
>>> >
>>> > 2017년 7월 13일 (목) 오전 9:16, Jungtaek Lim 님이 작성:
>>> >
>>> > > Maybe we could try out Gitbox, though every committers should join
>>> their
>>> > > Github accounts to 'apache' group and enable 2FA.
>>> > >
>>> > > 2017년 7월 13일 (목) 오전 8:38, Jungtaek Lim 님이 작성:
>>> > >
>>> > >> Did we render webpage with asf-site branch? I didn't recognize it.
>>> > >>
>>> > >> Yes I meant separate git repository, like 'storm-site'. I'm happy
>>> I'm
>>> > not
>>> > >> the only one who feels inconvenient with SVN repo.
>>> > >> Would it better to initiate VOTE for this?
>>> > >>
>>> > >> Thanks,
>>> > >> Jungtaek Lim (HeartSaVioR)
>>> > >>
>>> > >> 2017년 7월 13일 (목) 오전 4:30, P. Taylor Goetz 님이 작성:
>>> > >>
>>> > >>> We were using git before, then a year ago moved back to subversion
>>> to
>>> > >>> implement versioned documentation [1].
>>> > >>>
>>> > >>> If we do decide to move back to git for this, I would recommend
>>> using a
>>> > >>> separate git repository so it doesn’t bloat our main code
>>> repository.
>>> > When
>>> > >>> generating javadoc for a new version, the svn commit to publish the
>>> > site
>>> > >>> can take around 20 minutes.
>>> > >>>
>>> > >>> -Taylor
>>> > >>>
>>> > >>> > On Jul 12, 2017, at 10:33 AM, Jungtaek Lim 
>>> > wrote:
>>> > >>> >
>>> > >>> > Hi devs,
>>> > >>> >
>>> > >>> > I think we discussed moving website repository from SVN to GIT
>>> from a
>>> > >>> long
>>> > >>> > time ago, and we were OK on that, but action was not taken.
>>> > >>> >
>>> > >>> > Now I can see number of projects (Spark, Kafka, Beam, maybe
>>> more) are
>>> > >>> using
>>> > >>> > separate GIT repository for website.
>>> > >>> > Although we may still need to have version specific document (doc
>>> > >>> > directory) from code repository and copy Jekyll build result to
>>> > website
>>> > >>> > repo, anyone can look at the whole website code and craft pull
>>> > >>> requests to
>>> > >>> > help us. Git would be more convenient for ourselves than SVN
>>> (since
>>> > >>> we're
>>> > >>> > maintaining Storm from GIT).
>>> > >>> >
>>> > >>> > So I'd like to propose having a new repository 'storm-website' or
>>> > >>> > 'storm-site' with 'asf-site' as default branch, and move SVN

[GitHub] storm pull request #2265: [STORM-2674] catch NoNodeException when IStormClus...

[Ethanlm]

GitHub user Ethanlm opened a pull request:

https://github.com/apache/storm/pull/2265

[STORM-2674] catch NoNodeException when IStormClusterState tries to delete 
znodes in reportError function

Fix in 1.x-branch.

see https://github.com/apache/storm/pull/2264

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Ethanlm/storm STORM-2674-1.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2265.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2265


commit 93dc79557bc210b9239c6ae571c6103e5850b2ad
Author: Ethan Li 
Date:   2017-08-07T13:27:49Z

[STORM-2674] catch NoNodeException when IStormClusterState tries to delete 
znodes in reportError function




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2264: [STORM-2674] catch NoNodeException when IStormClusterStat...

[Ethanlm]

Github user Ethanlm commented on the issue:

https://github.com/apache/storm/pull/2264
  
Sorry, from wrong branch.. Will file a new PR..


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2264: [STORM-2674] catch NoNodeException when IStormClus...

[Ethanlm]

Github user Ethanlm closed the pull request at:

https://github.com/apache/storm/pull/2264


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2264: [STORM-2674] catch NoNodeException when IStormClus...

[Ethanlm]

GitHub user Ethanlm opened a pull request:

https://github.com/apache/storm/pull/2264

[STORM-2674] catch NoNodeException when IStormClusterState tries to delete 
znodes in reportError function

Catch the NoNodeException.

see JIRA: https://issues.apache.org/jira/browse/STORM-2674
see PR on master branch: https://github.com/apache/storm/pull/2256


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Ethanlm/storm 1.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2264.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2264


commit 93dc79557bc210b9239c6ae571c6103e5850b2ad
Author: Ethan Li 
Date:   2017-08-07T13:27:49Z

[STORM-2674] catch NoNodeException when IStormClusterState tries to delete 
znodes in reportError function




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [Propose] move website repository from svn to git

[Jungtaek Lim]

FYI: I've exported SVN repo of website and pushed to 'asf-site' branch of
GIT repo.

https://github.com/apache/storm-site

Please note that 'content' directory will be used for serving website.
(INFRA guided me to use this directory)
Need to update README.md for the new instruction.

Btw, this is still too way heavy, lots of files occupying huge space (2.3g)
making git too way slow as same as svn. Better to reduce some more
contents, maybe removing source docs (md files) which are also available in
storm repo.

- Jungtaek Lim (HeartSaVioR)

2017년 8월 7일 (월) 오전 11:55, Jungtaek Lim 님이 작성:

> FYI: storm-site git repository is created, and according to notification
> mail, github mirror will be created in a day.
>
> I filed another INFRA issues to associate website to the new repository.
> https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-14810
>
> - Jungtaek Lim (HeartSaVioR)
>
> 2017년 8월 2일 (수) 오전 8:09, Jungtaek Lim 님이 작성:
>
>> FYI: I just take a step to this, but blocked at creating git repository
>> in reporeq.apache.org.
>>
>> Just filed https://issues.apache.org/jira/browse/INFRA-14765. In that
>> issue I also asked how to serve website with non-main project repository.
>>
>> 2017년 7월 31일 (월) 오후 10:56, Bobby Evans 님이
>> 작성:
>>
>>> +1
>>> I am fine with moving to git, but I would like it to be a different repo.
>>> Our current repo is at least 160MB already (which is a lot to download)
>>> but nothing compared the the web site that has lots and lots of things
>>> checked in (I estimate it at about 1.5GB on an older version I have locally)
>>>
>>>
>>> - Bobby
>>>
>>>
>>> On Monday, July 31, 2017, 1:58:03 AM CDT, Xin Wang <
>>> data.xinw...@gmail.com> wrote:
>>>
>>> +1 for moving to git.  - Xin
>>>
>>>
>>>
>>> 2017-07-31 14:54 GMT+08:00 Jungtaek Lim :
>>>
>>> > Bump. I think this is worth to address soon, since some contributors
>>> > occasionally submit patches regarding documentations.
>>> > Personally SVN is no longer feel convenient to use. If we all feel the
>>> > same, let's change then.
>>> >
>>> > -Jungtaek Lim (HeartSaVioR)
>>> >
>>> > 2017년 7월 13일 (목) 오전 9:16, Jungtaek Lim 님이 작성:
>>> >
>>> > > Maybe we could try out Gitbox, though every committers should join
>>> their
>>> > > Github accounts to 'apache' group and enable 2FA.
>>> > >
>>> > > 2017년 7월 13일 (목) 오전 8:38, Jungtaek Lim 님이 작성:
>>> > >
>>> > >> Did we render webpage with asf-site branch? I didn't recognize it.
>>> > >>
>>> > >> Yes I meant separate git repository, like 'storm-site'. I'm happy
>>> I'm
>>> > not
>>> > >> the only one who feels inconvenient with SVN repo.
>>> > >> Would it better to initiate VOTE for this?
>>> > >>
>>> > >> Thanks,
>>> > >> Jungtaek Lim (HeartSaVioR)
>>> > >>
>>> > >> 2017년 7월 13일 (목) 오전 4:30, P. Taylor Goetz 님이 작성:
>>> > >>
>>> > >>> We were using git before, then a year ago moved back to subversion
>>> to
>>> > >>> implement versioned documentation [1].
>>> > >>>
>>> > >>> If we do decide to move back to git for this, I would recommend
>>> using a
>>> > >>> separate git repository so it doesn’t bloat our main code
>>> repository.
>>> > When
>>> > >>> generating javadoc for a new version, the svn commit to publish the
>>> > site
>>> > >>> can take around 20 minutes.
>>> > >>>
>>> > >>> -Taylor
>>> > >>>
>>> > >>> > On Jul 12, 2017, at 10:33 AM, Jungtaek Lim 
>>> > wrote:
>>> > >>> >
>>> > >>> > Hi devs,
>>> > >>> >
>>> > >>> > I think we discussed moving website repository from SVN to GIT
>>> from a
>>> > >>> long
>>> > >>> > time ago, and we were OK on that, but action was not taken.
>>> > >>> >
>>> > >>> > Now I can see number of projects (Spark, Kafka, Beam, maybe
>>> more) are
>>> > >>> using
>>> > >>> > separate GIT repository for website.
>>> > >>> > Although we may still need to have version specific document (doc
>>> > >>> > directory) from code repository and copy Jekyll build result to
>>> > website
>>> > >>> > repo, anyone can look at the whole website code and craft pull
>>> > >>> requests to
>>> > >>> > help us. Git would be more convenient for ourselves than SVN
>>> (since
>>> > >>> we're
>>> > >>> > maintaining Storm from GIT).
>>> > >>> >
>>> > >>> > So I'd like to propose having a new repository 'storm-website' or
>>> > >>> > 'storm-site' with 'asf-site' as default branch, and move SVN
>>> contents
>>> > >>> to
>>> > >>> > GIT.
>>> > >>> > (Sure we need to ask INFRA for helping Storm website to be
>>> rendered
>>> > >>> from a
>>> > >>> > new GIT repo.)
>>> > >>> >
>>> > >>> > What do you think?
>>> > >>> >
>>> > >>> > Thanks,
>>> > >>> > Jungtaek Lim (HeartSaVioR)
>>> > >>>
>>> > >>>
>>> >
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Xin
>>
>>

[GitHub] storm pull request #2263: STORM-2680 : The switch to turn on-off the cgroup ...

[cluo512]

GitHub user cluo512 opened a pull request:

https://github.com/apache/storm/pull/2263

STORM-2680 : The switch to turn on-off the cgroup in the doc should be 
“storm.resource.isolation.plugin.enable” 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cluo512/storm master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2263.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2263


commit 63a3ed588f3f597e8c6f186305bfce4ccf7d5387
Author: cluo <051...@163.com>
Date:   2017-07-31T06:08:05Z

Force to make the final-package some files use unix line ending

commit 61dc267862ef7512846080385b44b3cfb4f23f8b
Author: cluo <051...@163.com>
Date:   2017-08-04T06:43:58Z

Merge branch 'master' of https://github.com/cluo512/storm

commit 12074764f3a1732d03ab86246af0828e7320feab
Author: cluo <051...@163.com>
Date:   2017-08-07T07:11:41Z

The switch to turn on-off the cgroup in the doc should be 
"storm.resource.isolation.plugin.enable"




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---