[jira] [Updated] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caofangkun updated STORM-550:
-
Description:
{code:title=Utils.java|borderStyle=solid}
Index: storm-core/src/jvm/backtype/storm/utils/Utils.java
===
--- storm-core/src/jvm/backtype/storm/utils/Utils.java (revision 4021)
+++ storm-core/src/jvm/backtype/storm/utils/Utils.java (working copy)
@@ -19,6 +19,7 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
@@ -44,6 +45,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -187,7 +189,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = findAndReadConfigFile(new File(confFile).getName(), true);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
backtype.storm.daemon.common$loading__4910__auto__.invoke(common.clj:16)
at backtype.storm.daemon.common__init.load(Unknown Source)
at backtype.storm.daemon.common__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
backtype.storm.daemon.supervisor$loading__4910__auto__.invoke(supervisor.clj:16)
at backtype.storm.daemon.supervisor__init.load(Unknown Source)
at backtype.storm.daemon.supervisor__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:415)
at backtype.storm.daemon.supervisor.clinit(Unknown Source)
Caused by: java.lang.RuntimeException: Could not find config file on classpath
/home/caokun/deploy/storm-conf/storm.yaml
at
[GitHub] storm pull request: STORM-550:fix bug get an error when use --conf...
GitHub user caofangkun opened a pull request: https://github.com/apache/storm/pull/310 STORM-550:fix bug get an error when use --config option to override conf... fix bug get an error when use --config option to override conf... bin/storm --config ~/deploy/storm-conf/user.yamls supervisor You can merge this pull request into a Git repository by running: $ git pull https://github.com/caofangkun/apache-storm storm-550 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/310.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #310 commit dd944de09ef30b45306ba3a69b660e6adc8697b7 Author: caofangkun caofang...@gmail.com Date: 2014-11-11T06:35:07Z STORM-550:fix bug get an error when use --config option to override config file --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206029#comment-14206029
]
ASF GitHub Bot commented on STORM-550:
--
GitHub user caofangkun opened a pull request:
https://github.com/apache/storm/pull/310
STORM-550:fix bug get an error when use --config option to override conf...
fix bug get an error when use --config option to override conf...
bin/storm --config ~/deploy/storm-conf/user.yamls supervisor
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/caofangkun/apache-storm storm-550
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/310.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #310
commit dd944de09ef30b45306ba3a69b660e6adc8697b7
Author: caofangkun caofang...@gmail.com
Date: 2014-11-11T06:35:07Z
STORM-550:fix bug get an error when use --config option to override config
file
get an error when use --config option to override config file
---
Key: STORM-550
URL: https://issues.apache.org/jira/browse/STORM-550
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.3-rc2
Reporter: caofangkun
Assignee: caofangkun
Priority: Minor
{code:title=Utils.java|borderStyle=solid}
Index: storm-core/src/jvm/backtype/storm/utils/Utils.java
===
--- storm-core/src/jvm/backtype/storm/utils/Utils.java(revision 4021)
+++ storm-core/src/jvm/backtype/storm/utils/Utils.java(working copy)
@@ -19,6 +19,7 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
@@ -44,6 +45,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -187,7 +189,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = findAndReadConfigFile(new File(confFile).getName(),
true);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
backtype.storm.daemon.common$loading__4910__auto__.invoke(common.clj:16)
at backtype.storm.daemon.common__init.load(Unknown Source)
at backtype.storm.daemon.common__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
[jira] [Updated] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caofangkun updated STORM-550:
-
Description:
{code:title=Utils.java|borderStyle=solid}
Index: src/jvm/backtype/storm/utils/Utils.java
===
--- src/jvm/backtype/storm/utils/Utils.java (revision 4021)
+++ src/jvm/backtype/storm/utils/Utils.java (working copy)
@@ -19,7 +19,10 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.FileNotFoundException;
import java.io.FileOutputStream;
+import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
@@ -31,6 +34,7 @@
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.HashSet;
@@ -44,6 +48,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -121,6 +126,27 @@
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
+ return Collections.String, Object emptyMap();
+ }
+ result = (MapString, Object) obj;
+ } catch (FileNotFoundException e) {
+ throw new RuntimeException(e);
+ }
+ return result;
+ }
public static Map findAndReadConfigFile(String name, boolean mustExist) {
try {
@@ -187,7 +213,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = loadConfigFile(confFile);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
backtype.storm.daemon.common$loading__4910__auto__.invoke(common.clj:16)
at backtype.storm.daemon.common__init.load(Unknown Source)
at backtype.storm.daemon.common__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
[jira] [Created] (STORM-553) Provide standard Java/Clojure Code Style Formatter profiles
caofangkun created STORM-553: Summary: Provide standard Java/Clojure Code Style Formatter profiles Key: STORM-553 URL: https://issues.apache.org/jira/browse/STORM-553 Project: Apache Storm Issue Type: Improvement Affects Versions: 0.9.3-rc2 Reporter: caofangkun Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206045#comment-14206045
]
ASF GitHub Bot commented on STORM-550:
--
Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/310#discussion_r20134395
--- Diff: storm-core/src/jvm/backtype/storm/utils/Utils.java ---
@@ -121,6 +126,27 @@ public static void sleep(long millis) {
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
--- End diff --
Same as above.
get an error when use --config option to override config file
---
Key: STORM-550
URL: https://issues.apache.org/jira/browse/STORM-550
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.3-rc2
Reporter: caofangkun
Assignee: caofangkun
Priority: Minor
{code:title=Utils.java|borderStyle=solid}
Index: src/jvm/backtype/storm/utils/Utils.java
===
--- src/jvm/backtype/storm/utils/Utils.java (revision 4021)
+++ src/jvm/backtype/storm/utils/Utils.java (working copy)
@@ -19,7 +19,10 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.FileNotFoundException;
import java.io.FileOutputStream;
+import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
@@ -31,6 +34,7 @@
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.HashSet;
@@ -44,6 +48,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -121,6 +126,27 @@
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
+ return Collections.String, Object emptyMap();
+ }
+ result = (MapString, Object) obj;
+ } catch (FileNotFoundException e) {
+ throw new RuntimeException(e);
+ }
+ return result;
+ }
public static Map findAndReadConfigFile(String name, boolean mustExist) {
try {
@@ -187,7 +213,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = loadConfigFile(confFile);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
[jira] [Commented] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206044#comment-14206044
]
ASF GitHub Bot commented on STORM-550:
--
Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/310#discussion_r20134376
--- Diff: storm-core/src/jvm/backtype/storm/utils/Utils.java ---
@@ -121,6 +126,27 @@ public static void sleep(long millis) {
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
--- End diff --
Shouldn't this throw a RunTime exception?
get an error when use --config option to override config file
---
Key: STORM-550
URL: https://issues.apache.org/jira/browse/STORM-550
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.3-rc2
Reporter: caofangkun
Assignee: caofangkun
Priority: Minor
{code:title=Utils.java|borderStyle=solid}
Index: src/jvm/backtype/storm/utils/Utils.java
===
--- src/jvm/backtype/storm/utils/Utils.java (revision 4021)
+++ src/jvm/backtype/storm/utils/Utils.java (working copy)
@@ -19,7 +19,10 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.FileNotFoundException;
import java.io.FileOutputStream;
+import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
@@ -31,6 +34,7 @@
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.HashSet;
@@ -44,6 +48,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -121,6 +126,27 @@
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
+ return Collections.String, Object emptyMap();
+ }
+ result = (MapString, Object) obj;
+ } catch (FileNotFoundException e) {
+ throw new RuntimeException(e);
+ }
+ return result;
+ }
public static Map findAndReadConfigFile(String name, boolean mustExist) {
try {
@@ -187,7 +213,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = loadConfigFile(confFile);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at
[GitHub] storm pull request: STORM-550:fix bug get an error when use --conf...
Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/310#discussion_r20134395
--- Diff: storm-core/src/jvm/backtype/storm/utils/Utils.java ---
@@ -121,6 +126,27 @@ public static void sleep(long millis) {
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
--- End diff --
Same as above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-329 : buffer message in client and recon...
Github user clockfly commented on the pull request: https://github.com/apache/storm/pull/268#issuecomment-62509705 @tedxia The thread pool size of clientScheduleService is decided by worker number (also =1 and = 10). For example, if there are 2 worker, the pool size is 1, if worker number is 4, then the pool size is 3. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-329) Add Option to Config Message handling strategy when connection timeout
[
https://issues.apache.org/jira/browse/STORM-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206050#comment-14206050
]
ASF GitHub Bot commented on STORM-329:
--
Github user clockfly commented on the pull request:
https://github.com/apache/storm/pull/268#issuecomment-62509705
@tedxia
The thread pool size of clientScheduleService is decided by worker number
(also =1 and = 10).
For example, if there are 2 worker, the pool size is 1, if worker number is
4, then the pool size is 3.
Add Option to Config Message handling strategy when connection timeout
--
Key: STORM-329
URL: https://issues.apache.org/jira/browse/STORM-329
Project: Apache Storm
Issue Type: Improvement
Affects Versions: 0.9.2-incubating
Reporter: Sean Zhong
Priority: Minor
Labels: Netty
Fix For: 0.9.2-incubating
Attachments: storm-329.patch, worker-kill-recover3.jpg
This is to address a [concern brought
up|https://github.com/apache/incubator-storm/pull/103#issuecomment-43632986]
during the work at STORM-297:
{quote}
[~revans2] wrote: Your logic makes since to me on why these calls are
blocking. My biggest concern around the blocking is in the case of a worker
crashing. If a single worker crashes this can block the entire topology from
executing until that worker comes back up. In some cases I can see that being
something that you would want. In other cases I can see speed being the
primary concern and some users would like to get partial data fast, rather
then accurate data later.
Could we make it configurable on a follow up JIRA where we can have a max
limit to the buffering that is allowed, before we block, or throw data away
(which is what zeromq does)?
{quote}
If some worker crash suddenly, how to handle the message which was supposed
to be delivered to the worker?
1. Should we buffer all message infinitely?
2. Should we block the message sending until the connection is resumed?
3. Should we config a buffer limit, try to buffer the message first, if the
limit is met, then block?
4. Should we neither block, nor buffer too much, but choose to drop the
messages, and use the built-in storm failover mechanism?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] storm pull request: STORM-550:fix bug get an error when use --conf...
Github user clockfly commented on the pull request: https://github.com/apache/storm/pull/310#issuecomment-62510083 Seems this is duplicate of STORM-188 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-550) get an error when use --config option to override config file
[
https://issues.apache.org/jira/browse/STORM-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206057#comment-14206057
]
ASF GitHub Bot commented on STORM-550:
--
Github user clockfly commented on the pull request:
https://github.com/apache/storm/pull/310#issuecomment-62510083
Seems this is duplicate of STORM-188
get an error when use --config option to override config file
---
Key: STORM-550
URL: https://issues.apache.org/jira/browse/STORM-550
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.3-rc2
Reporter: caofangkun
Assignee: caofangkun
Priority: Minor
{code:title=Utils.java|borderStyle=solid}
Index: src/jvm/backtype/storm/utils/Utils.java
===
--- src/jvm/backtype/storm/utils/Utils.java (revision 4021)
+++ src/jvm/backtype/storm/utils/Utils.java (working copy)
@@ -19,7 +19,10 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.FileNotFoundException;
import java.io.FileOutputStream;
+import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
@@ -31,6 +34,7 @@
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.HashSet;
@@ -44,6 +48,7 @@
import backtype.storm.serialization.DefaultSerializationDelegate;
import backtype.storm.serialization.SerializationDelegate;
+
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.commons.lang.StringUtils;
@@ -121,6 +126,27 @@
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
+ }
+ Yaml yaml = new Yaml();
+ InputStream in;
+ try {
+ in = new FileInputStream(new File(file));
+ Object obj = yaml.load(in);
+ if (!(obj instanceof Map)) {
+ return Collections.String, Object emptyMap();
+ }
+ result = (MapString, Object) obj;
+ } catch (FileNotFoundException e) {
+ throw new RuntimeException(e);
+ }
+ return result;
+ }
public static Map findAndReadConfigFile(String name, boolean mustExist) {
try {
@@ -187,7 +213,7 @@
if (confFile==null || confFile.equals()) {
storm = findAndReadConfigFile(storm.yaml, false);
} else {
-storm = findAndReadConfigFile(confFile, true);
+storm = loadConfigFile(confFile);
}
ret.putAll(storm);
ret.putAll(readCommandLineOpts());
{code}
bin/storm --config ~/deploy/storm-conf/storm.yaml supervisor
{code:title=Could not find config file on classpath|borderStyle=solid}
Exception in thread main java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:171)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.clinit(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.loadClassForName(RT.java:2098)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5018.invoke(core.clj:5530)
at clojure.core$load.doInvoke(core.clj:5529)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5336)
at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
at clojure.core$load_lib.doInvoke(core.clj:5374)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5417)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5507)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at
backtype.storm.daemon.common$loading__4910__auto__.invoke(common.clj:16)
at
[GitHub] storm pull request: STORM-550:fix bug get an error when use --conf...
Github user caofangkun commented on a diff in the pull request:
https://github.com/apache/storm/pull/310#discussion_r20135808
--- Diff: storm-core/src/jvm/backtype/storm/utils/Utils.java ---
@@ -121,6 +126,27 @@ public static void sleep(long millis) {
throw new RuntimeException(e);
}
}
+
+ @SuppressWarnings(unchecked)
+ public static MapString, Object loadConfigFile(String file) {
+ MapString, Object result = new HashMapString, Object();
+ if (file == null) {
+ return result;
--- End diff --
Any wrong in this way?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
