[GitHub] storm issue #1918: STORM-2339: Python code format cleanup in storm.py

[tibkiss]

Github user tibkiss commented on the issue:

https://github.com/apache/storm/pull/1918
  
"At 2 months, your baby doesn’t yet have the coordination to play with 
toys. But she may bat at a colorful object hanging in front of her. Your baby 
may even briefly hold a toy that you place in one of her hands."

Could someone please merge this before it starts to play with toys? Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2083: STORM-2421: support lists of childopts in DaemonConfig.

[erikdw]

Github user erikdw commented on the issue:

https://github.com/apache/storm/pull/2083
  
@hmcc : I put a comment in the bug -- the problem isn't sufficiently 
specified IMHO.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: github source code gone?

[P. Taylor Goetz]

Apache infra is working on it. This is a github issue not ASF. The cononical 
ASF repos are unaffected, so our source repo is safe.

-Taylor

> On Apr 28, 2017, at 5:46 PM, Michael Moss  wrote:
> 
> https://github.com/apache/storm
> 
> "This repository is empty."
> 
> Am I looking in the wrong place?

Re: [VOTE] Java Code Style Standard for Apache Storm.

[Kishorkumar Patil]

[1] Google Java Style Guide

[2] Sun Java Code 
Conventions

[3] HBase 
style
 (sorry all I could find was an eclipse XML config)

[4] Hadoop style which is 
described as java but with 2 spaces instead of 4. 

-Kishor


On Thursday, April 27, 2017 10:01 PM, Satish Duggana  
wrote:
 

 [1] Google Java Style Guide

[2] Hadoop style which
is described as java but with 2 spaces instead of 4.

[3] Sun Java Code Conventions

[4] HBase style (sorry all I could find was an eclipse
XML config)

Thanks,
~Satish.

On Thu, Apr 27, 2017 at 7:50 PM, Kyle Nusbaum <
knusb...@yahoo-inc.com.invalid> wrote:

> [1] Google Java Style Guide io/styleguide/javaguide.html>
>
> [2] Sun Java Code Conventions codeconventions-150003.pdf>
>
> [3] HBase style support/hbase_eclipse_formatter.xml> (sorry all I could find was an
> eclipse XML config)
>
> [4] Hadoop style
> which is described as java but with 2 spaces instead of 4.
> Thanks,
> -- Kyle
>
> On Thursday, April 27, 2017, 3:51:25 AM CDT, Julien Nioche <
> lists.digitalpeb...@gmail.com> wrote:Non-binding : [1 ] Google Java Style
> Guide
>
> On 26 April 2017 at 19:50, Bobby Evans 
> wrote:
>
> > We would like to adopt a code style standard for Apache Storm.  Please
> > rank the following with 1 being the most desired and 4 being the least
> > desired (5 if you have a write in choice).  This is not an official vote
> as
> > per the ByLaws, but we will probably go with whichever wins the most 1
> > votes (unless it is really close and then we may go with some STeVe like
> > ranking, but I want to avoid it because it is hard and I might get the
> math
> > wrong) This is open to everyone so please vote.
> >
> >
> > [ ] Google Java Style Guide > io/styleguide/javaguide.html>
> >
> > [ ] Sun Java Code Conventions > codeconventions-150003.pdf>
> >
> > [ ] HBase style > support/hbase_eclipse_formatter.xml> (sorry all I could find was an
> > eclipse XML config)
> >
> > [ ] Hadoop style
> > which is described as java but with 2 spaces instead of 4.
> >
> > [ ] Other (Please specify)
> >
> >
> > I apologize if the formatting is bad I am forced to use a corporate
> > sponsored mail client that somehow messed up plain text formatting.  I
> > don't know how.
> >
> >
> > - Bobby
>
>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>


   

[GitHub] storm issue #2093: STORM-2495:Integrate checkstyle check during build

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/2093
  
@vinodkc We cannot increase the size of the log without paying money to 
travisci, or switching the build to the internal Jenkins servers at apache.  
The later has been proposed but is not likely to happen soon.

My advice would be to set the useFile config to move the warnings to a text 
file that we can output the first X lines of if the build fails.  Then add in a 
maxAllowedViolations for each project with their current number of violations 
along with a JIRA to fix the warnings in the project and remove the limit.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2094: [STORM-2191] shorten classpaths by using wildcards

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2094#discussion_r113919131
  
--- Diff: docs/Setting-up-a-Storm-cluster.md ---
@@ -102,9 +102,9 @@ The time to allow any given healthcheck script to run 
before it is marked failed
 storm.health.check.timeout.ms: 5000
 ```
 
-### Configure external libraries and environmental variables (optional)
+### Configure external libraries and environment variables (optional)
 
-If you need support from external libraries or custom plugins, you can 
place such jars into the extlib/ and extlib-daemon/ directories. Note that the 
extlib-daemon/ directory stores jars used only by daemons (Nimbus, Supervisor, 
DRPC, UI, Logviewer), e.g., HDFS and customized scheduling libraries. 
Accordingly, two environmental variables STORM_EXT_CLASSPATH and 
STORM_EXT_CLASSPATH_DAEMON can be configured by users for including the 
external classpath and daemon-only external classpath.
+If you need support from external libraries or custom plugins, you can 
place such jars into the extlib/ and extlib-daemon/ directories. Note that the 
extlib-daemon/ directory stores jars used only by daemons (Nimbus, Supervisor, 
DRPC, UI, Logviewer), e.g., HDFS and customized scheduling libraries. 
Accordingly, two environment variables STORM_EXT_CLASSPATH and 
STORM_EXT_CLASSPATH_DAEMON can be configured by users for including the 
external classpath and daemon-only external classpath. See [Classpath 
handling](Classpath-handling.html)] for more details on using external 
libraries.
--- End diff --

It might be good to include a link to Classpath-handling.html in 
Running-topologies-on-a-production-cluster.md or some other page that talks 
about running a topology.  Perhaps Command-line-client.md would be good to, so 
it can explain how this works for users launching workers.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2083: STORM-2421: support lists of childopts in DaemonCo...

[hmcc]

Github user hmcc commented on a diff in the pull request:

https://github.com/apache/storm/pull/2083#discussion_r113902383
  
--- Diff: storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java ---
@@ -313,6 +315,34 @@ public static File getWorkerDirFromRoot(String 
logRoot, String id, Integer port)
 return new File((logRoot + FILE_SEPARATOR + id + FILE_SEPARATOR + 
port));
 }
 
+/**
+ * Get the given config value as a String, if possible.
+ * @param name - the config key
+ * @param conf - the config map
+ * @return - the config value converted to a String if found, 
otherwise null.
+ * @throws IllegalArgumentException if conf is null
+ * @throws NullPointerException if name is null and the conf map 
doesn't support null keys
+ */
+public static String getConfigValueAsString(String name, Map 
conf) {
--- End diff --

@revans2 You mean, replace this method with
```
public static String getConfigValueAsList(String name, Map conf) {
// if the value is a list, return it
// if the value is a string, split it by space and return the resulting 
list
}
```
and the caller can convert it to a space-separated string if needed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2092: STORM-2493: update documents to reflect the change...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/2092#discussion_r113879015
  
--- Diff: docs/storm-pmml.md ---
@@ -0,0 +1,37 @@
+#Storm PMML Bolt
--- End diff --

I'm anal like that. ;-)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2092: STORM-2493: update documents to reflect the change...

[vesense]

Github user vesense commented on a diff in the pull request:

https://github.com/apache/storm/pull/2092#discussion_r113874786
  
--- Diff: docs/storm-pmml.md ---
@@ -0,0 +1,37 @@
+#Storm PMML Bolt
--- End diff --

Nice catch. Will fix.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2092: STORM-2493: update documents to reflect the change...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/2092#discussion_r113874334
  
--- Diff: docs/storm-pmml.md ---
@@ -0,0 +1,37 @@
+#Storm PMML Bolt
--- End diff --

nit: spacing is not consistent here with the other docs that you've added 
(and the existing ones).  i.e., `#Foo` should be `# Foo`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2094: [STORM-2191] shorten classpaths by using wildcards

[erikdw]

Github user erikdw commented on the issue:

https://github.com/apache/storm/pull/2094
  
@revans2 & @harshach & @hmcl : Here's my proposed change.  I'm probably 
misinterpreting some of the usages or behaviors with all of these various 
external library features, but I tried my best to figure them out after 
extensively reading the tickets and commits that added the features.  Also, I'm 
fine with not renaming `get_jars_full` and `getFullJars`, I just thought the 
names were misleading after this change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2094: [STORM-2191] shorten classpaths by using wildcards

[erikdw]

GitHub user erikdw opened a pull request:

https://github.com/apache/storm/pull/2094

[STORM-2191] shorten classpaths by using wildcards

Instead of fully enumerating all JARs in the lib directories, we just use
a Java classpath wildcard to allow the JVM to autodiscover all JARs in the
classpath.  This affects both the Worker and LogWriter processes, as well
as Storm daemons such as the Nimbus, UI, Logviewer, and Supervisor.

This change results in shorter commands, so that you can actually see
the full content of the Worker command in `ps` output on Linux.

Prior to this change Worker commands were easily longer than 4096 bytes,
which is the default Linux kernel limit for commands being recorded into
the process table.  Longer commands get truncated, though they do get
executed.

An example of the change in Worker classpath length can be seen here:

Before:
```
-cp 
STORM_DIR/lib-worker/asm-5.0.3.jar:STORM_DIR/lib-worker/carbonite-1.5.0.jar:STORM_DIR/lib-worker/chill-java-0.8.0.jar:STORM_DIR/lib-worker/clojure-1.7.0.jar:STORM_DIR/lib-worker/commons-codec-1.6.jar:STORM_DIR/lib-worker/commons-collections-3.2.2.jar:STORM_DIR/lib-worker/commons-io-2.5.jar:STORM_DIR/lib-worker/commons-lang-2.5.jar:STORM_DIR/lib-worker/commons-logging-1.1.3.jar:STORM_DIR/lib-worker/curator-client-2.12.0.jar:STORM_DIR/lib-worker/curator-framework-2.12.0.jar:STORM_DIR/lib-worker/disruptor-3.3.2.jar:STORM_DIR/lib-worker/guava-16.0.1.jar:STORM_DIR/lib-worker/httpclient-4.3.3.jar:STORM_DIR/lib-worker/httpcore-4.4.1.jar:STORM_DIR/lib-worker/jgrapht-core-0.9.0.jar:STORM_DIR/lib-worker/jline-0.9.94.jar:STORM_DIR/lib-worker/json-simple-1.1.jar:STORM_DIR/lib-worker/kryo-3.0.3.jar:STORM_DIR/lib-worker/kryo-shaded-3.0.3.jar:STORM_DIR/lib-worker/libthrift-0.9.3.jar:STORM_DIR/lib-worker/log4j-api-2.8.jar:STORM_DIR/lib-worker/log4j-core-2.8.jar:STORM_DIR/lib-worker/log4j-ove
 
r-slf4j-1.6.6.jar:STORM_DIR/lib-worker/log4j-slf4j-impl-2.8.jar:STORM_DIR/lib-worker/minlog-1.3.0.jar:STORM_DIR/lib-worker/netty-3.9.0.Final.jar:STORM_DIR/lib-worker/objenesis-2.1.jar:STORM_DIR/lib-worker/reflectasm-1.10.1.jar:STORM_DIR/lib-worker/servlet-api-2.5.jar:STORM_DIR/lib-worker/slf4j-api-1.7.21.jar:STORM_DIR/lib-worker/snakeyaml-1.11.jar:STORM_DIR/lib-worker/storm-client-2.0.0-SNAPSHOT.jar:STORM_DIR/lib-worker/sysout-over-slf4j-1.0.2.jar:STORM_DIR/lib-worker/zookeeper-3.4.6.jar:STORM_DIR/conf:STORM_DIR/storm-local/supervisor/stormdist/foo-topology-1-1-1493359573/stormjar.jar
```

After:
```
-cp 
STORM_DIR/lib-worker/*:STORM_DIR/extlib/*:STORM_DIR/conf:STORM_DIR/storm-local/supervisor/stormdist/foo-topology-1-1-1493359573/stormjar.jar
```

This change also includes additional documentation about the use of 
classpaths in
Storm and provides some guidance for using the various features for using 
external
libraries.

For more details on this problem and a discussion about this solution's
merits, please see 
[STORM-2191](https://issues.apache.org/jira/browse/STORM-2191).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erikdw/storm STORM-2191

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2094.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2094


commit 2ceef0dce21895c4ef0f0b8eb5bc8248fad25e13
Author: Erik Weathers 
Date:   2017-04-28T07:50:18Z

[STORM-2191] shorten classpaths by using wildcards

Instead of fully enumerating all JARs in the lib directories, we just use
a Java classpath wildcard to allow the JVM to autodiscover all JARs in the
classpath.  This affects both the Worker and LogWriter processes, as well
as Storm daemons such as the Nimbus, UI, Logviewer, and Supervisor.

This change results in shorter commands, so that you can actually see
the full content of the Worker command in `ps` output on Linux.

Prior to this change Worker commands were easily longer than 4096 bytes,
which is the default Linux kernel limit for commands being recorded into
the process table.  Longer commands get truncated, though they do get
executed.

An example of the change in Worker classpath length can be seen here:

Before:
```
-cp 

[GitHub] storm pull request #2082: expose Tuple for node.js

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/2082


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2087: Fixed DemoTest.java link

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/2087


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #2093: STORM-2495:Integrate checkstyle check during build

[vinodkc]

Github user vinodkc commented on the issue:

https://github.com/apache/storm/pull/2093
  
Due to new checkstyle warnings, size of build log file exceeded 4mb limit. 

Error from build 
"The log length has exceeded the limit of 4 MB (this usually means that the 
test suite is raising the same exception over and over)."

Can we increase the log size limit?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---