[jira] [Created] (STORM-4053) Add Hadoop client API dependency back to storm-hdfs
Julien Nioche created STORM-4053: Summary: Add Hadoop client API dependency back to storm-hdfs Key: STORM-4053 URL: https://issues.apache.org/jira/browse/STORM-4053 Project: Apache Storm Issue Type: Task Components: storm-hdfs Affects Versions: 2.6.2 Reporter: Julien Nioche See https://github.com/apache/incubator-stormcrawler/pull/1189 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-4039) Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1
[ https://issues.apache.org/jira/browse/STORM-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-4039. -- Assignee: Julien Nioche Resolution: Fixed > Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1 > --- > > Key: STORM-4039 > URL: https://issues.apache.org/jira/browse/STORM-4039 > Project: Apache Storm > Issue Type: Task >Affects Versions: 2.6.1 >Reporter: Julien Nioche >Assignee: Julien Nioche >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/storm/pull/3631 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (STORM-4039) Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1
[ https://issues.apache.org/jira/browse/STORM-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-4039: - Fix Version/s: 2.6.2 > Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1 > --- > > Key: STORM-4039 > URL: https://issues.apache.org/jira/browse/STORM-4039 > Project: Apache Storm > Issue Type: Task >Affects Versions: 2.6.1 >Reporter: Julien Nioche >Assignee: Julien Nioche >Priority: Major > Fix For: 2.6.2 > > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/storm/pull/3631 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (STORM-4039) Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1
Julien Nioche created STORM-4039: Summary: Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.1 Key: STORM-4039 URL: https://issues.apache.org/jira/browse/STORM-4039 Project: Apache Storm Issue Type: Task Affects Versions: 2.6.1 Reporter: Julien Nioche https://github.com/apache/storm/pull/3631 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-4035) Remove ConfluentAvroSerializer (storm-hdfs)
[ https://issues.apache.org/jira/browse/STORM-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-4035. -- Resolution: Fixed > Remove ConfluentAvroSerializer (storm-hdfs) > --- > > Key: STORM-4035 > URL: https://issues.apache.org/jira/browse/STORM-4035 > Project: Apache Storm > Issue Type: Improvement >Affects Versions: 2.6.1 >Reporter: Richard Zowalla >Assignee: Richard Zowalla >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > ConfluentAvroSerializer relies on a super old external library. It isn't > covcered with unit tests. > > We can remove this class as well as the related dependency. > > People, who are using it, can still copy the class content and add that into > their topology. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-4030) Dependency upgrades
[
https://issues.apache.org/jira/browse/STORM-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved STORM-4030.
--
Resolution: Fixed
> Dependency upgrades
> ---
>
> Key: STORM-4030
> URL: https://issues.apache.org/jira/browse/STORM-4030
> Project: Apache Storm
> Issue Type: Task
>Affects Versions: 2.6.1
>Reporter: Julien Nioche
>Assignee: Julien Nioche
>Priority: Minor
> Fix For: 2.7.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {code:java}
> mvn versions:display-dependency-updates
> "-Dmaven.version.ignore=.*-M.*,.*-alpha.*,.*-beta.*,.*-BETA.*,.*-b.*,.*.ALPHA.*"
> | grep '\->' | sort | uniq{code}
> {{shows a large number of upgradable dependencies.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (STORM-4030) Dependency upgrades
[
https://issues.apache.org/jira/browse/STORM-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819676#comment-17819676
]
Julien Nioche edited comment on STORM-4030 at 2/22/24 2:44 PM:
---
commons-i 2.11.0 -> :2.14.0
commons-lang 3.13.0 -> 3.14.0
log4j 2.21.1 -> 2.23.0
caffeine 2.3.5 -> 3.1.8
guava 32.1.3-jre -> 33.0.0-jre
jackson 2.15.2 -> 2.16.1
netty 4.1.100.Final -> 4.1.107.Final
junit 5.10.0 -> 5.10.2
snakeyaml 2.0 -> 2.2
RocksDB JNI 8.5.4 -> 8.10.0
checker-qual 3.37.0 -> 3.42.0
error_prone_annotations 2.21.1 -> 2.25.0
testcontainers 1.19.1 -> 1.19.6
was (Author: jnioche):
commons-i 2.11.0 -> :2.14.0
commons-lang 3.13.0 -> 3.14.0
log4j 2.21.1 -> 2.23.0
caffeine 2.3.5 -> 3.1.8
error_prone_annotations 2.21.1 -> 2.25.0
guava 32.1.3-jre -> 33.0.0-jre
jackson 2.15.2 -> 2.16.1
netty 4.1.100.Final -> 4.1.107.Final
snakeyaml 2.0 -> 2.2
RocksDB JNI 8.5.4 -> 8.10.0
hecker-qual 3.37.0 -> 3.42.0
> Dependency upgrades
> ---
>
> Key: STORM-4030
> URL: https://issues.apache.org/jira/browse/STORM-4030
> Project: Apache Storm
> Issue Type: Task
>Affects Versions: 2.6.1
>Reporter: Julien Nioche
>Assignee: Julien Nioche
>Priority: Minor
> Fix For: 2.7.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code:java}
> mvn versions:display-dependency-updates
> "-Dmaven.version.ignore=.*-M.*,.*-alpha.*,.*-beta.*,.*-BETA.*,.*-b.*,.*.ALPHA.*"
> | grep '\->' | sort | uniq{code}
> {{shows a large number of upgradable dependencies.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (STORM-4030) Dependency upgrades
[
https://issues.apache.org/jira/browse/STORM-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819676#comment-17819676
]
Julien Nioche commented on STORM-4030:
--
commons-i 2.11.0 -> :2.14.0
commons-lang 3.13.0 -> 3.14.0
log4j 2.21.1 -> 2.23.0
caffeine 2.3.5 -> 3.1.8
error_prone_annotations 2.21.1 -> 2.25.0
guava 32.1.3-jre -> 33.0.0-jre
jackson 2.15.2 -> 2.16.1
netty 4.1.100.Final -> 4.1.107.Final
snakeyaml 2.0 -> 2.2
RocksDB JNI 8.5.4 -> 8.10.0
hecker-qual 3.37.0 -> 3.42.0
> Dependency upgrades
> ---
>
> Key: STORM-4030
> URL: https://issues.apache.org/jira/browse/STORM-4030
> Project: Apache Storm
> Issue Type: Task
>Affects Versions: 2.6.1
>Reporter: Julien Nioche
>Assignee: Julien Nioche
>Priority: Minor
> Fix For: 2.7.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code:java}
> mvn versions:display-dependency-updates
> "-Dmaven.version.ignore=.*-M.*,.*-alpha.*,.*-beta.*,.*-BETA.*,.*-b.*,.*.ALPHA.*"
> | grep '\->' | sort | uniq{code}
> {{shows a large number of upgradable dependencies.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (STORM-4030) Dependency upgrades
Julien Nioche created STORM-4030:
Summary: Dependency upgrades
Key: STORM-4030
URL: https://issues.apache.org/jira/browse/STORM-4030
Project: Apache Storm
Issue Type: Task
Affects Versions: 2.6.1
Reporter: Julien Nioche
Assignee: Julien Nioche
Fix For: 2.7.0
{code:java}
mvn versions:display-dependency-updates
"-Dmaven.version.ignore=.*-M.*,.*-alpha.*,.*-beta.*,.*-BETA.*,.*-b.*,.*.ALPHA.*"
| grep '\->' | sort | uniq{code}
{{shows a large number of upgradable dependencies.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (STORM-4014) Add Docker file
[ https://issues.apache.org/jira/browse/STORM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche closed STORM-4014. Fix Version/s: (was: 2.7.0) Resolution: Fixed > Add Docker file > > > Key: STORM-4014 > URL: https://issues.apache.org/jira/browse/STORM-4014 > Project: Apache Storm > Issue Type: New Feature >Reporter: Julien Nioche >Priority: Major > > See https://lists.apache.org/thread/cdcq5c9bbjd09n574sq147vcjqpt3mqm > We want to be able to generate Docker images for Apache Storm and push them > to DockerHub -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (STORM-4014) Add Docker file
[ https://issues.apache.org/jira/browse/STORM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805503#comment-17805503 ] Julien Nioche commented on STORM-4014: -- The storm-docker repo has been donated and is now at [https://github.com/apache/storm-docker|https://github.com/apache/storm-docker] > Add Docker file > > > Key: STORM-4014 > URL: https://issues.apache.org/jira/browse/STORM-4014 > Project: Apache Storm > Issue Type: New Feature >Reporter: Julien Nioche >Priority: Major > Fix For: 2.7.0 > > > See https://lists.apache.org/thread/cdcq5c9bbjd09n574sq147vcjqpt3mqm > We want to be able to generate Docker images for Apache Storm and push them > to DockerHub -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (STORM-4014) Add Docker file
[ https://issues.apache.org/jira/browse/STORM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804239#comment-17804239 ] Julien Nioche commented on STORM-4014: -- Note The author of [https://github.com/31z4/storm-docker] has mentioned in a private discussion that he would be willing to contribute the code to Apache Storm. I will move the discussion with him to a public mailing list. > Add Docker file > > > Key: STORM-4014 > URL: https://issues.apache.org/jira/browse/STORM-4014 > Project: Apache Storm > Issue Type: New Feature >Reporter: Julien Nioche >Priority: Major > Fix For: 2.7.0 > > > See https://lists.apache.org/thread/cdcq5c9bbjd09n574sq147vcjqpt3mqm > We want to be able to generate Docker images for Apache Storm and push them > to DockerHub -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (STORM-4014) Add Docker file
[ https://issues.apache.org/jira/browse/STORM-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799042#comment-17799042 ] Julien Nioche commented on STORM-4014: -- See https://hub.docker.com/repository/docker/apache/storm/general https://issues.apache.org/jira/browse/INFRA-25280 > Add Docker file > > > Key: STORM-4014 > URL: https://issues.apache.org/jira/browse/STORM-4014 > Project: Apache Storm > Issue Type: New Feature >Reporter: Julien Nioche >Priority: Major > Fix For: 2.7.0 > > > See https://lists.apache.org/thread/cdcq5c9bbjd09n574sq147vcjqpt3mqm > We want to be able to generate Docker images for Apache Storm and push them > to DockerHub -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (STORM-4014) Add Docker file
Julien Nioche created STORM-4014: Summary: Add Docker file Key: STORM-4014 URL: https://issues.apache.org/jira/browse/STORM-4014 Project: Apache Storm Issue Type: New Feature Reporter: Julien Nioche Fix For: 2.7.0 See https://lists.apache.org/thread/cdcq5c9bbjd09n574sq147vcjqpt3mqm We want to be able to generate Docker images for Apache Storm and push them to DockerHub -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-3984) Nimbus failover causes unnecessary reassign if 600s are passed after starting Nimbus
[
https://issues.apache.org/jira/browse/STORM-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved STORM-3984.
--
Fix Version/s: 2.6.0
Resolution: Fixed
> Nimbus failover causes unnecessary reassign if 600s are passed after starting
> Nimbus
>
>
> Key: STORM-3984
> URL: https://issues.apache.org/jira/browse/STORM-3984
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-server
>Affects Versions: 2.5.0
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Fix For: 2.6.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Since the [TimeOutWorkerHeartbeatsRecoveryStrategy.startTimeSecs is set on
> Nimbus
> start-up|https://github.com/apache/storm/blob/4374fdc93982db3e511ed5e8af642eef97c89aae/storm-server/src/main/java/org/apache/storm/nimbus/TimeOutWorkerHeartbeatsRecoveryStrategy.java#L48]
> and never
> updated,[TimeOutWorkerHeartbeatsRecoveryStrategy#exceedsMaxTimeOut|https://github.com/apache/storm/blob/4374fdc93982db3e511ed5e8af642eef97c89aae/storm-server/src/main/java/org/apache/storm/nimbus/TimeOutWorkerHeartbeatsRecoveryStrategy.java#L68-L70]
> always returns true after 600s (the value of supervisor.w\
> orker.heartbeats.max.timeout.secs) are passed after Nimbus is started.
> Invalid timeout in new leader Nimbus causes unnecessary reassign right after
> failover.
> {noformat}
> 2023-09-25 15:16:46.538 o.a.s.n.NimbusInfo main-EventThread [INFO] Nimbus
> figures out its name to h02
> 2023-09-25 15:16:46.549 o.a.s.n.LeaderListenerCallback main-EventThread
> [INFO] Sync remote assignments and id-info to local
> 2023-09-25 15:16:46.571 o.a.s.n.LeaderListenerCallback main-EventThread
> [INFO] active-topology-blobs [word-count-1-1695654263] local-topology-blobs
> [word-count-1-1695654263-stormconf.ser,word-count-1-1695654263-stormjar.jar,word-count-1-1695654263-stormcode.ser]
> diff-topology-blobs []
> 2023-09-25 15:16:46.596 o.a.s.n.LeaderListenerCallback main-EventThread
> [INFO] active-topology-dependencies [] local-blobs
> [word-count-1-1695654263-stormconf.ser,word-count-1-1695654263-stormjar.jar,word-count-1-1695654263-stormcode.ser]
> diff-topology-dependencies []
> 2023-09-25 15:16:46.596 o.a.s.n.LeaderListenerCallback main-EventThread
> [INFO] Accepting leadership, all active topologies and corresponding
> dependencies found locally.
> 2023-09-25 15:16:46.596 o.a.s.z.LeaderListenerCallbackFactory
> main-EventThread [INFO] h02 gained leadership.
> 2023-09-25 15:16:46.744 o.a.s.n.TimeOutWorkerHeartbeatsRecoveryStrategy timer
> [WARN] Failed to recover heartbeats for nodes:
> [c26e72ef-b84b-4d44-820a-fec9407e38cf-172.18.0.11,
> 57ff205e-6d90-4305-abb8-b9ff0ff7bcc3-172.18.0.13,
> f10f6554-0e55-4c01-a6ce-834df068d753-172.18.0.12] with timeout 600s
> 2023-09-25 15:16:46.807 o.a.s.d.n.HeartbeatCache timer [INFO] Executor
> word-count-1-1695654263:[8, 8] not alive
> 2023-09-25 15:16:46.808 o.a.s.d.n.HeartbeatCache timer [INFO] Executor
> word-count-1-1695654263:[16, 16] not alive
> ...(snip)
> 2023-09-25 15:16:46.862 o.a.s.d.n.Nimbus timer [INFO] Reassigning
> word-count-1-1695654263 to 3 slots
> 2023-09-25 15:16:46.862 o.a.s.d.n.Nimbus timer [INFO] Reassign executors:
> [[20, 20], [14, 14], [12, 12], [16, 16], [18, 18], [28, 28], [26, 26], [10,
> 10], [8, 8], [24, 24], [6, 6], [22, 22], [2, 2], [4, 4], [13, 13], [11, 11],
> [7, 7], [9, 9], [19, 19], [23, 23], [21, 21], [25, 25], [27, 27], [5, 5]\
> , [1, 1], [3, 3], [15, 15], [17, 17]]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (STORM-3983) Refactor File.createTempFile with Files.create(...)
Julien Nioche created STORM-3983: Summary: Refactor File.createTempFile with Files.create(...) Key: STORM-3983 URL: https://issues.apache.org/jira/browse/STORM-3983 Project: Apache Storm Issue Type: Improvement Components: storm-core Affects Versions: 2.5.0 Reporter: Julien Nioche Fix For: 2.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-2708) View Logs by single point. With using Docker swarm
[ https://issues.apache.org/jira/browse/STORM-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-2708. -- Resolution: Duplicate > View Logs by single point. With using Docker swarm > -- > > Key: STORM-2708 > URL: https://issues.apache.org/jira/browse/STORM-2708 > Project: Apache Storm > Issue Type: Question >Reporter: Ivan >Priority: Major > Attachments: Screenshot from 2017-08-26 22-04-44.png, Screenshot from > 2017-08-26 22-05-29.png, Screenshot from 2017-08-26 22-06-09.png > > > Use Storm on Docker swarm system and I should debugging system. I need get > log information from whole storm system by single point. I can get log-file > from host machines where deployed containers, but i want use easy single > access point. > I know about storm logviewer, I think it is that what I need. But I cant tune > system. > I see docker containers like: a63c6da1dc17, fe2efd0149b4 (you can see at > Screenshot from 2017-08-26 22-04-44.png) > I can income to this host and see something like Screenshot from 2017-08-26 > 22-05-29.png > When I click "log" link (fe2efd0149b4 (log)) i get Screenshot from 2017-08-26 > 22-06-09.png > How I can get logs, search in logs from storm system with use Docker? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-3925) Allow user resources (in WorkerTopologyContext) to be set by Worker Hooks
[
https://issues.apache.org/jira/browse/STORM-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved STORM-3925.
--
Resolution: Fixed
> Allow user resources (in WorkerTopologyContext) to be set by Worker Hooks
> -
>
> Key: STORM-3925
> URL: https://issues.apache.org/jira/browse/STORM-3925
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
>Reporter: Bharat Gulati
>Priority: Minor
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The current implementation of WorkerTopologyContext in WorkerState will
> always lead to empty userResources as no interface exposes a way to allow
> user to set them.
> {code:java}
> private Map makeUserResources() {
> /* TODO: need to invoke a hook provided by the topology, giving it a
> chance to create user resources.
> * this would be part of the initialization hook
> * need to separate workertopologycontext into WorkerContext and
> WorkerUserContext.
> * actually just do it via interfaces. just need to make sure to hide
> setResource from tasks
> */
> return new HashMap<>();
> } {code}
> The intention will be to expose the relevant methods under a separate class
> which can then allow users to set the resources from WorkerHooks while only
> providing get access from Tasks (i.e. via TopologyContext)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (STORM-3944) ConcurrentModificationException in MultiCountMetric
[
https://issues.apache.org/jira/browse/STORM-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved STORM-3944.
--
Resolution: Fixed
> ConcurrentModificationException in MultiCountMetric
>
>
> Key: STORM-3944
> URL: https://issues.apache.org/jira/browse/STORM-3944
> Project: Apache Storm
> Issue Type: Bug
>Affects Versions: 2.5.0
>Reporter: Julien Nioche
>Priority: Major
> Fix For: 2.6
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Came across the following exception
> {code:java}
> 022-07-15 09:57:16.851 o.a.s.e.e.ReportError Thread-43-fetcher-executor[15,
> 15] [ERROR] Error
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.util.ConcurrentModificationException
> at org.apache.storm.utils.Utils$1.run(Utils.java:411)
> ~[storm-client-2.4.0.jar:2.4.0]
> at java.lang.Thread.run(Thread.java:829) [?:?]
> Caused by: java.lang.RuntimeException:
> java.util.ConcurrentModificationException
> at org.apache.storm.executor.Executor.accept(Executor.java:301)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.Utils$1.run(Utils.java:396)
> ~[storm-client-2.4.0.jar:2.4.0]
> ... 1 more
> Caused by: java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1511) ~[?:?]
> at java.util.HashMap$EntryIterator.next(HashMap.java:1544) ~[?:?]
> at java.util.HashMap$EntryIterator.next(HashMap.java:1542) ~[?:?]
> at
> org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:35)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:18)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.executor.Executor.metricsTick(Executor.java:339)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.executor.bolt.BoltExecutor.tupleActionFn(BoltExecutor.java:200)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.executor.Executor.accept(Executor.java:297)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
> ~[storm-client-2.4.0.jar:2.4.0]
> at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
> ~[storm-client-2.4.0.jar:2.4.0]
> at org.apache.storm.utils.Utils$1.run(Utils.java:396)
> ~[storm-client-2.4.0.jar:2.4.0]
> ... 1 more
> {code}
> Seems to be a straightforward case of one thread modifying the
> MultiCountMetric while another one is calling getValueAndReset.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (STORM-3944) ConcurrentModificationException in MultiCountMetric
[
https://issues.apache.org/jira/browse/STORM-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche updated STORM-3944:
-
Description:
Came across the following exception
{code:java}
022-07-15 09:57:16.851 o.a.s.e.e.ReportError Thread-43-fetcher-executor[15, 15]
[ERROR] Error
java.lang.RuntimeException: java.lang.RuntimeException:
java.util.ConcurrentModificationException
at org.apache.storm.utils.Utils$1.run(Utils.java:411)
~[storm-client-2.4.0.jar:2.4.0]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.util.ConcurrentModificationException
at org.apache.storm.executor.Executor.accept(Executor.java:301)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.Utils$1.run(Utils.java:396)
~[storm-client-2.4.0.jar:2.4.0]
... 1 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1511) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1544) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1542) ~[?:?]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:35)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:18)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.metricsTick(Executor.java:339)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor.tupleActionFn(BoltExecutor.java:200)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.accept(Executor.java:297)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.Utils$1.run(Utils.java:396)
~[storm-client-2.4.0.jar:2.4.0]
... 1 more
{code}
Seems to be a straightforward case of one thread modifying the MultiCountMetric
while another one is calling getValueAndReset.
was:
Came across the following exception
{code:java}
022-07-15 09:57:16.851 o.a.s.e.e.ReportError Thread-43-fetcher-executor[15, 15]
[ERROR] Error
java.lang.RuntimeException: java.lang.RuntimeException:
java.util.ConcurrentModificationException
at org.apache.storm.utils.Utils$1.run(Utils.java:411)
~[storm-client-2.4.0.jar:2.4.0]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.util.ConcurrentModificationException
at org.apache.storm.executor.Executor.accept(Executor.java:301)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.Utils$1.run(Utils.java:396)
~[storm-client-2.4.0.jar:2.4.0]
... 1 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1511) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1544) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1542) ~[?:?]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:35)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:18)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.metricsTick(Executor.java:339)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor.tupleActionFn(BoltExecutor.java:200)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.accept(Executor.java:297)
~[storm-client-2.4.0.jar:2.4.0]
at
[jira] [Created] (STORM-3944) ConcurrentModificationException in MultiCountMetric
Julien Nioche created STORM-3944:
Summary: ConcurrentModificationException in MultiCountMetric
Key: STORM-3944
URL: https://issues.apache.org/jira/browse/STORM-3944
Project: Apache Storm
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Julien Nioche
Fix For: 2.6
Came across the following exception
{code:java}
022-07-15 09:57:16.851 o.a.s.e.e.ReportError Thread-43-fetcher-executor[15, 15]
[ERROR] Error
java.lang.RuntimeException: java.lang.RuntimeException:
java.util.ConcurrentModificationException
at org.apache.storm.utils.Utils$1.run(Utils.java:411)
~[storm-client-2.4.0.jar:2.4.0]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.util.ConcurrentModificationException
at org.apache.storm.executor.Executor.accept(Executor.java:301)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.Utils$1.run(Utils.java:396)
~[storm-client-2.4.0.jar:2.4.0]
... 1 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1511) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1544) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1542) ~[?:?]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:35)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.metric.api.MultiCountMetric.getValueAndReset(MultiCountMetric.java:18)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.metricsTick(Executor.java:339)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor.tupleActionFn(BoltExecutor.java:200)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.executor.Executor.accept(Executor.java:297)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:113)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:154)
~[storm-client-2.4.0.jar:2.4.0]
at
org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:140)
~[storm-client-2.4.0.jar:2.4.0]
at org.apache.storm.utils.Utils$1.run(Utils.java:396)
~[storm-client-2.4.0.jar:2.4.0]
... 1 more
{code}
Seems to be a straightforward case of 2 or more threads concurrently
_getValueAndReset_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (STORM-3940) Missing dependency prevents Storm from being built
[ https://issues.apache.org/jira/browse/STORM-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-3940. -- Resolution: Fixed > Missing dependency prevents Storm from being built > -- > > Key: STORM-3940 > URL: https://issues.apache.org/jira/browse/STORM-3940 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Julien Nioche >Priority: Blocker > Labels: easyfix > Fix For: 2.5.0 > > > Storm can't be built because the dependency _pentaho-aggdesigner-algorithm_ > is not available. > This dependency is excluded in some places of the poms but not > systematically, excluding it everywhere allows the code to be built. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (STORM-3940) Missing dependency prevents Storm from being built
[ https://issues.apache.org/jira/browse/STORM-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750214#comment-17750214 ] Julien Nioche commented on STORM-3940: -- Fixed in https://github.com/apache/storm/pull/3556 > Missing dependency prevents Storm from being built > -- > > Key: STORM-3940 > URL: https://issues.apache.org/jira/browse/STORM-3940 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Julien Nioche >Priority: Blocker > Labels: easyfix > Fix For: 2.5.0 > > > Storm can't be built because the dependency _pentaho-aggdesigner-algorithm_ > is not available. > This dependency is excluded in some places of the poms but not > systematically, excluding it everywhere allows the code to be built. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (STORM-3940) Missing dependency prevents Storm from being built
[ https://issues.apache.org/jira/browse/STORM-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche closed STORM-3940. > Missing dependency prevents Storm from being built > -- > > Key: STORM-3940 > URL: https://issues.apache.org/jira/browse/STORM-3940 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Julien Nioche >Priority: Blocker > Labels: easyfix > Fix For: 2.5.0 > > > Storm can't be built because the dependency _pentaho-aggdesigner-algorithm_ > is not available. > This dependency is excluded in some places of the poms but not > systematically, excluding it everywhere allows the code to be built. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (STORM-3940) Missing dependency prevents Storm from being built
[ https://issues.apache.org/jira/browse/STORM-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-3940: - Summary: Missing dependency prevents Storm from being built (was: Missing dependency prevents Stom from being built) > Missing dependency prevents Storm from being built > -- > > Key: STORM-3940 > URL: https://issues.apache.org/jira/browse/STORM-3940 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Julien Nioche >Priority: Blocker > Labels: easyfix > Fix For: 2.5.0 > > > Storm can't be built because the dependency _pentaho-aggdesigner-algorithm_ > is not available. > This dependency is excluded in some places of the poms but not > systematically, excluding it everywhere allows the code to be built. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (STORM-3940) Missing dependency prevents Stom from being built
Julien Nioche created STORM-3940: Summary: Missing dependency prevents Stom from being built Key: STORM-3940 URL: https://issues.apache.org/jira/browse/STORM-3940 Project: Apache Storm Issue Type: Bug Affects Versions: 2.4.0 Reporter: Julien Nioche Fix For: 2.5.0 Storm can't be built because the dependency _pentaho-aggdesigner-algorithm_ is not available. This dependency is excluded in some places of the poms but not systematically, excluding it everywhere allows the code to be built. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (STORM-3742) Topology arguments starting with -c get removed
[ https://issues.apache.org/jira/browse/STORM-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-3742. -- Resolution: Not A Problem > Topology arguments starting with -c get removed > --- > > Key: STORM-3742 > URL: https://issues.apache.org/jira/browse/STORM-3742 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Julien Nioche >Assignee: Bipin Prasad >Priority: Major > > with the following command > storm local target/stormcrawler-1.0-SNAPSHOT.jar > global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf > es-conf.yaml . seeds.txt > the topology class is only getting [crawler-conf.yaml, es-conf.yaml, ., > seeds.txt]and the -conf have been removed. > Anything after the class name should be left as-is. > This does not happen with Storm 1.2.3 > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3742) Topology arguments starting with -c get removed
[ https://issues.apache.org/jira/browse/STORM-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296519#comment-17296519 ] Julien Nioche commented on STORM-3742: -- thanks [~bipinprasad], the aim is to have the class process the arguments. These topologies expect to get '-conf xxx'. Inserting '–' seems to have done the trick! Thanks a lot This is to use with [ConfigurableTopologies|[https://github.com/apache/storm/blob/2.2.x-branch/storm-client/src/jvm/org/apache/storm/topology/ConfigurableTopology.java#L136];] they expect to get such arguments. > Topology arguments starting with -c get removed > --- > > Key: STORM-3742 > URL: https://issues.apache.org/jira/browse/STORM-3742 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Julien Nioche >Assignee: Bipin Prasad >Priority: Major > > with the following command > storm local target/stormcrawler-1.0-SNAPSHOT.jar > global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf > es-conf.yaml . seeds.txt > the topology class is only getting [crawler-conf.yaml, es-conf.yaml, ., > seeds.txt]and the -conf have been removed. > Anything after the class name should be left as-is. > This does not happen with Storm 1.2.3 > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3742) Topology arguments starting with -c get removed
Julien Nioche created STORM-3742: Summary: Topology arguments starting with -c get removed Key: STORM-3742 URL: https://issues.apache.org/jira/browse/STORM-3742 Project: Apache Storm Issue Type: Bug Affects Versions: 2.2.0 Reporter: Julien Nioche with the following command storm local target/stormcrawler-1.0-SNAPSHOT.jar global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf es-conf.yaml . seeds.txt the topology class is only getting [crawler-conf.yaml, es-conf.yaml, ., seeds.txt]and the -conf have been removed. Anything after the class name should be left as-is. This does not happen with Storm 1.2.3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3582) Kryo errors when using custom serialization
[
https://issues.apache.org/jira/browse/STORM-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081733#comment-17081733
]
Julien Nioche commented on STORM-3582:
--
have commented in STORM-3620
> Kryo errors when using custom serialization
> ---
>
> Key: STORM-3582
> URL: https://issues.apache.org/jira/browse/STORM-3582
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 2.1.0
>Reporter: Julien Nioche
>Priority: Blocker
>
> (this has been reported on the user list in January and previously by a
> different user)
>
>
>
> My code works fine with Storm 1.x but the workers crash constantly with Storm
> 2.x
>
> Some exceptions look like
>
> {{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
> {{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
> ~[kryo-3.0.3.jar:?]_}}
> {{whereas others are}}
>
> {{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the
> request_}}
> {{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class
> ID: *95*
> at
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
> ~[storm-client-2.1.0.jar:2.1.0]_}}
> {{the class ID values are random integers. }}
> {{}}
> The tuples in my topologies contain mostly standard classes like String; the
> only exception is the following class
>
> [https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
>
> for which we specify a custom serialization for Kryo.
>
> My configurations contain
>
> | topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
> to register the custom class.
>
> The user who had reported the problem first also used custom serialization.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3620) OutputCollector in Storm 2.x is not thread-safe
[
https://issues.apache.org/jira/browse/STORM-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081732#comment-17081732
]
Julien Nioche commented on STORM-3620:
--
IMHO it would be preferable to make the outputCollector threadsafe - this is
the case in 1.x and apart from solving the problem experienced in STORM-3582 ,
it would greatly simplify things. Having to change the user code to enforce
synchronization would be a pain. As for eliminating multithreading inside a
bolt, well, again this would be a regression compared to 1.x and simply not
possible in some cases.
> OutputCollector in Storm 2.x is not thread-safe
> ---
>
> Key: STORM-3620
> URL: https://issues.apache.org/jira/browse/STORM-3620
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Priority: Major
>
> OutputCollector is not thread-safe in 2.x.
> It can cause data corruption if multiple threads in the same executor calls
> OutputCollector to emit data at the same time:
> 1. Every executor has an instance of ExecutorTransfer
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L146
> 2. Every ExecutorTransfer has its own serializer
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java#L44
> 3. Every executor has its own outputCollector
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/bolt/BoltExecutor.java#L146-L147
> 4. When outputCollector is called to emit to remote workers, it uses
> ExecutorTransfer to transfer data
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java#L66
> 5. which will try to serialize data
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/daemon/worker/WorkerTransfer.java#L116
> 6. But serializer is not thread-safe
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/serialization/KryoTupleSerializer.java#L33-L43
>
> But in the doc, http://storm.apache.org/releases/2.1.0/Concepts.html, it says
> outputCollector is thread-safe.
> {code:java}
> Its perfectly fine to launch new threads in bolts that do processing
> asynchronously. OutputCollector is thread-safe and can be called at any time.
> {code}
> We should either fix it to make it thread-safe, or update the document to not
> mislead users
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3582) Kryo errors when using custom serialization
[
https://issues.apache.org/jira/browse/STORM-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081189#comment-17081189
]
Julien Nioche commented on STORM-3582:
--
Based on Simon's findings, this should be fixed with the synchronization issue.
I'll try again once STORM-3620 has been done
> Kryo errors when using custom serialization
> ---
>
> Key: STORM-3582
> URL: https://issues.apache.org/jira/browse/STORM-3582
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 2.1.0
>Reporter: Julien Nioche
>Priority: Blocker
>
> (this has been reported on the user list in January and previously by a
> different user)
>
>
>
> My code works fine with Storm 1.x but the workers crash constantly with Storm
> 2.x
>
> Some exceptions look like
>
> {{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
> {{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
> ~[kryo-3.0.3.jar:?]_}}
> {{whereas others are}}
>
> {{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the
> request_}}
> {{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class
> ID: *95*
> at
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
> ~[storm-client-2.1.0.jar:2.1.0]_}}
> {{the class ID values are random integers. }}
> {{}}
> The tuples in my topologies contain mostly standard classes like String; the
> only exception is the following class
>
> [https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
>
> for which we specify a custom serialization for Kryo.
>
> My configurations contain
>
> | topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
> to register the custom class.
>
> The user who had reported the problem first also used custom serialization.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (STORM-3582) Kryo errors when using custom serialization
[
https://issues.apache.org/jira/browse/STORM-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche updated STORM-3582:
-
Priority: Blocker (was: Critical)
> Kryo errors when using custom serialization
> ---
>
> Key: STORM-3582
> URL: https://issues.apache.org/jira/browse/STORM-3582
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 2.1.0
>Reporter: Julien Nioche
>Priority: Blocker
>
> (this has been reported on the user list in January and previously by a
> different user)
>
>
>
> My code works fine with Storm 1.x but the workers crash constantly with Storm
> 2.x
>
> Some exceptions look like
>
> {{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
> {{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
> ~[kryo-3.0.3.jar:?]_}}
> {{whereas others are}}
>
> {{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the
> request_}}
> {{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class
> ID: *95*
> at
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
> ~[storm-client-2.1.0.jar:2.1.0]_}}
> {{the class ID values are random integers. }}
> {{}}
> The tuples in my topologies contain mostly standard classes like String; the
> only exception is the following class
>
> [https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
>
> for which we specify a custom serialization for Kryo.
>
> My configurations contain
>
> | topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
> to register the custom class.
>
> The user who had reported the problem first also used custom serialization.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3582) Kryo errors when using custom serialization
Julien Nioche created STORM-3582:
Summary: Kryo errors when using custom serialization
Key: STORM-3582
URL: https://issues.apache.org/jira/browse/STORM-3582
Project: Apache Storm
Issue Type: Bug
Components: storm-core
Affects Versions: 2.1.0
Reporter: Julien Nioche
(this has been reported on the user list in January and previously by a
different user)
My code works fine with Storm 1.x but the workers crash constantly with Storm
2.x
Some exceptions look like
{{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
{{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
~[kryo-3.0.3.jar:?]_}}
{{whereas others are}}
{{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the
request_}}
{{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID:
*95*
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689) ~[kryo-3.0.3.jar:?]
at
org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
~[storm-client-2.1.0.jar:2.1.0]_}}
{{the class ID values are random integers. }}
{{}}
The tuples in my topologies contain mostly standard classes like String; the
only exception is the following class
[https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
for which we specify a custom serialization for Kryo.
My configurations contain
| topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
to register the custom class.
The user who had reported the problem first also used custom serialization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3276) Can't run Flux with Storm 2.0.0
Julien Nioche created STORM-3276: Summary: Can't run Flux with Storm 2.0.0 Key: STORM-3276 URL: https://issues.apache.org/jira/browse/STORM-3276 Project: Apache Storm Issue Type: Bug Components: Flux Affects Versions: 2.0.0 Reporter: Julien Nioche I try to run a Flux-based topology with Storm 2.0.0 _apache-storm-2.0.0/bin/storm local target/2-1.0-SNAPSHOT.jar org.apache.storm.flux.Flux crawler.flux --local-ttl 999_ Am getting _17:41:22.191 [main] ERROR o.a.s.f.Flux - To run in local mode run with 'storm local' instead of 'storm jar'_ _17:41:22.191 [main] INFO o.a.s.LocalCluster -_ _RUNNING LOCAL CLUSTER for 999 seconds._ and nothing happens after that. The documentation for Flux [http://storm.apache.org/releases/2.0.0-SNAPSHOT/flux.html] still mentions using 'storm jar' as well as --local and --sleep. My test topology can be found at [https://github.com/DigitalPebble/storm2] and requires the branch 2.x of StormCrawler [https://github.com/DigitalPebble/storm-crawler/tree/2.x] to by installed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (STORM-174) Implement DBMetricsConsumer to store Metrics in Databases like MySQL, PostgreSQL and so on .
[ https://issues.apache.org/jira/browse/STORM-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche resolved STORM-174. - Resolution: Invalid The patch provided is not related to the issue > Implement DBMetricsConsumer to store Metrics in Databases like MySQL, > PostgreSQL and so on . > - > > Key: STORM-174 > URL: https://issues.apache.org/jira/browse/STORM-174 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: caofangkun >Priority: Minor > Attachments: STORM-177-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (STORM-2428) Flux-core jar contains unpacked dependencies
[ https://issues.apache.org/jira/browse/STORM-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-2428: - Affects Version/s: 1.2.2 > Flux-core jar contains unpacked dependencies > > > Key: STORM-2428 > URL: https://issues.apache.org/jira/browse/STORM-2428 > Project: Apache Storm > Issue Type: Bug > Components: Flux >Affects Versions: 1.1.0, 1.0.3, 1.2.1, 1.2.2 >Reporter: Julien Nioche >Priority: Major > > The jar file for flux-core contains classes from /org/apache/http/. This was > not the case before and causes problems with projects which rely on a > different version of http-client. > I can't see any references to http-client in the pom though. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (STORM-2428) Flux-core jar contains unpacked dependencies
[ https://issues.apache.org/jira/browse/STORM-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-2428: - Affects Version/s: 1.2.1 > Flux-core jar contains unpacked dependencies > > > Key: STORM-2428 > URL: https://issues.apache.org/jira/browse/STORM-2428 > Project: Apache Storm > Issue Type: Bug > Components: Flux >Affects Versions: 1.1.0, 1.0.3, 1.2.1 >Reporter: Julien Nioche >Priority: Major > > The jar file for flux-core contains classes from /org/apache/http/. This was > not the case before and causes problems with projects which rely on a > different version of http-client. > I can't see any references to http-client in the pom though. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (STORM-3011) Use default bin path in flight.bash if $JAVA_HOME is undefined
[ https://issues.apache.org/jira/browse/STORM-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche closed STORM-3011. > Use default bin path in flight.bash if $JAVA_HOME is undefined > -- > > Key: STORM-3011 > URL: https://issues.apache.org/jira/browse/STORM-3011 > Project: Apache Storm > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Julien Nioche >Assignee: Julien Nioche >Priority: Minor > Labels: pull-request-available > Fix For: 2.0.0, 1.1.3, 1.0.7, 1.2.2 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > I had to modify the _flight.bash_ script as $JAVA_HOME is undefined, this > yields > > _/usr/share/apache-storm-1.2.1/bin/flight.bash: line 62: /bin/jstack: No such > file or directory_ > > however, the files can be found in the default path _/usr/bin_ > > I will submit a PR so that the script checks that $JDKPATH is not empty and > exist before using it for BINPATH. I can't see any downsides in doing so. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (STORM-3011) Use default bin path in flight.bash if $JAVA_HOME is undefined
Julien Nioche created STORM-3011: Summary: Use default bin path in flight.bash if $JAVA_HOME is undefined Key: STORM-3011 URL: https://issues.apache.org/jira/browse/STORM-3011 Project: Apache Storm Issue Type: Improvement Affects Versions: 1.2.1 Reporter: Julien Nioche I had to modify the _flight.bash_ script as $JAVA_HOME is undefined, this yields _/usr/share/apache-storm-1.2.1/bin/flight.bash: line 62: /bin/jstack: No such file or directory_ however, the files can be found in the default path _/usr/bin_ I will submit a PR so that the script checks that $JDKPATH is not empty and exist before using it for BINPATH. I can't see any downsides in doing so. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (STORM-2502) Use PID in file name for heap dumps
Julien Nioche created STORM-2502: Summary: Use PID in file name for heap dumps Key: STORM-2502 URL: https://issues.apache.org/jira/browse/STORM-2502 Project: Apache Storm Issue Type: Improvement Affects Versions: 1.1.0 Reporter: Julien Nioche Priority: Minor The default JVM options for the workers specify the path to use for the heap dumps, see [https://github.com/apache/storm/blob/1.x-branch/conf/defaults.yaml#L171], however when a memory error happens more than once for the same worker, only the first dump is kept as the file can't be overridden. Instead, would it make sense to use something like _"-XX:HeapDumpPath=artifacts/heapdump_.hprof"_ so that a different dump is generated for each JVM instance? Or is the current pattern used on purpose to avoid too much disk space being used? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (STORM-2326) Upgrade log4j and slf4j
[ https://issues.apache.org/jira/browse/STORM-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883410#comment-15883410 ] Julien Nioche commented on STORM-2326: -- thanks [~ptgoetz]. I'll open a new PR for 2.x next week > Upgrade log4j and slf4j > --- > > Key: STORM-2326 > URL: https://issues.apache.org/jira/browse/STORM-2326 > Project: Apache Storm > Issue Type: Dependency upgrade >Affects Versions: 2.0.0, 1.x >Reporter: Julien Nioche >Priority: Minor > Fix For: 1.1.0, 1.0.4 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The dependencies to log4j could be upgraded from 2.1 to 2.7, same for slf4j > to 1.7.21. > This would help fix [STORM-1386] > BTW any idea why we need log4j-over-slf4j? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (STORM-2327) Abstract class ConfigurableTopology
[ https://issues.apache.org/jira/browse/STORM-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849726#comment-15849726 ] Julien Nioche commented on STORM-2327: -- @kabhwan 1.x does not have the class LocalCluster in Java so we can't have ConfigurableTopology in storm-core as far as I can tell > Abstract class ConfigurableTopology > --- > > Key: STORM-2327 > URL: https://issues.apache.org/jira/browse/STORM-2327 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Affects Versions: 2.0.0 >Reporter: Julien Nioche >Assignee: Julien Nioche >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Classes which run topologies often repeat the same code and pattern to: > * populate the configuration from a file instead of ~/.storm > * determine whether to run locally or remotely > * set a TTL for a topology > Flux provides an elegant way of dealing with these but sometimes it is > simpler to define a topology in Java code. > In [StormCrawler|http://stormcrawler.net], we implemented an abstract class > named ConfigurableTopology which can be extended and saves users the hassle > of having to write code for the things above. I will open a PR containing > this class so that we can discuss and comment whether it is of any use at all. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (STORM-2327) Abstract class ConfigurableTopology
[ https://issues.apache.org/jira/browse/STORM-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-2327: - External issue URL: https://github.com/apache/storm/pull/1897 > Abstract class ConfigurableTopology > --- > > Key: STORM-2327 > URL: https://issues.apache.org/jira/browse/STORM-2327 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Affects Versions: 2.0.0 >Reporter: Julien Nioche >Priority: Minor > > Classes which run topologies often repeat the same code and pattern to: > * populate the configuration from a file instead of ~/.storm > * determine whether to run locally or remotely > * set a TTL for a topology > Flux provides an elegant way of dealing with these but sometimes it is > simpler to define a topology in Java code. > In [StormCrawler|http://stormcrawler.net], we implemented an abstract class > named ConfigurableTopology which can be extended and saves users the hassle > of having to write code for the things above. I will open a PR containing > this class so that we can discuss and comment whether it is of any use at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (STORM-2055) Exception when running topology from Maven exec with Flux
[
https://issues.apache.org/jira/browse/STORM-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche updated STORM-2055:
-
Comment: was deleted
(was: Related to that one?)
> Exception when running topology from Maven exec with Flux
> -
>
> Key: STORM-2055
> URL: https://issues.apache.org/jira/browse/STORM-2055
> Project: Apache Storm
> Issue Type: Bug
> Components: Flux
>Affects Versions: 1.0.2
>Reporter: Julien Nioche
>
> When running a topology from Maven with Flux as a dependency, we get
> {code}
> 11335 [Thread-8] ERROR o.a.s.event - Error when processing event
> java.io.FileNotFoundException: Source
> 'file:/home/julien/.m2/repository/org/apache/storm/flux-core/1.0.1/flux-core-1.0.1.jar!/resources'
> does not exist
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1368)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1261)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1230)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.daemon.supervisor$fn__9359.invoke(supervisor.clj:1194)
> ~[storm-core-1.0.1.jar:1.0.1]
> at clojure.lang.MultiFn.invoke(MultiFn.java:243) ~[clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9078$fn__9096.invoke(supervisor.clj:582)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9078.invoke(supervisor.clj:581)
> ~[storm-core-1.0.1.jar:1.0.1]
> at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40)
> [storm-core-1.0.1.jar:1.0.1]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
> {code}
> The same topology runs fine when executed with Eclipse or via the storm
> command.
> See [https://github.com/DigitalPebble/storm-crawler/issues/324]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-2055) Exception when running topology from Maven exec with Flux
[
https://issues.apache.org/jira/browse/STORM-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839751#comment-15839751
]
Julien Nioche commented on STORM-2055:
--
Related to that one?
> Exception when running topology from Maven exec with Flux
> -
>
> Key: STORM-2055
> URL: https://issues.apache.org/jira/browse/STORM-2055
> Project: Apache Storm
> Issue Type: Bug
> Components: Flux
>Affects Versions: 1.0.2
>Reporter: Julien Nioche
>
> When running a topology from Maven with Flux as a dependency, we get
> {code}
> 11335 [Thread-8] ERROR o.a.s.event - Error when processing event
> java.io.FileNotFoundException: Source
> 'file:/home/julien/.m2/repository/org/apache/storm/flux-core/1.0.1/flux-core-1.0.1.jar!/resources'
> does not exist
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1368)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1261)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.shade.org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1230)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.daemon.supervisor$fn__9359.invoke(supervisor.clj:1194)
> ~[storm-core-1.0.1.jar:1.0.1]
> at clojure.lang.MultiFn.invoke(MultiFn.java:243) ~[clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9078$fn__9096.invoke(supervisor.clj:582)
> ~[storm-core-1.0.1.jar:1.0.1]
> at
> org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9078.invoke(supervisor.clj:581)
> ~[storm-core-1.0.1.jar:1.0.1]
> at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40)
> [storm-core-1.0.1.jar:1.0.1]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
> {code}
> The same topology runs fine when executed with Eclipse or via the storm
> command.
> See [https://github.com/DigitalPebble/storm-crawler/issues/324]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (STORM-2326) Upgrade log4j and slf4j
[ https://issues.apache.org/jira/browse/STORM-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Nioche updated STORM-2326: - External issue URL: https://github.com/apache/storm/pull/1896 > Upgrade log4j and slf4j > --- > > Key: STORM-2326 > URL: https://issues.apache.org/jira/browse/STORM-2326 > Project: Apache Storm > Issue Type: Dependency upgrade >Affects Versions: 2.0.0, 1.x >Reporter: Julien Nioche >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The dependencies to log4j could be upgraded from 2.1 to 2.7, same for slf4j > to 1.7.21. > This would help fix [STORM-1386] > BTW any idea why we need log4j-over-slf4j? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-2326) Upgrade log4j and slf4j
Julien Nioche created STORM-2326: Summary: Upgrade log4j and slf4j Key: STORM-2326 URL: https://issues.apache.org/jira/browse/STORM-2326 Project: Apache Storm Issue Type: Dependency upgrade Affects Versions: 2.0.0, 1.x Reporter: Julien Nioche Priority: Minor The dependencies to log4j could be upgraded from 2.1 to 2.7, same for slf4j to 1.7.21. This would help fix [STORM-1386] BTW any idea why we need log4j-over-slf4j? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
