Robert Joseph Evans created STORM-2448:
------------------------------------------
Summary: Support running workers using older JVMs/storm versions
Key: STORM-2448
URL: https://issues.apache.org/jira/browse/STORM-2448
Project: Apache Storm
Issue Type: Improvement
Components: storm-core
Affects Versions: 2.0.0
Reporter: Robert Joseph Evans
As a part of STORM-2441 we are separating out the classpaths for the
client+worker process from everything else in storm. This is great but it
really will make some of our users upset, because it is not a rolling upgrade,
and because they will need to recompile their topologies (again).
We have done a really good job in maintaining compatibility with older versions
of storm because we use thrift for all communication and state storage. This
means that a new supervisor and or nimbus should be able to talk to just about
any existing client/worker out there. So we should explicitly support this.
We should add in config options to supervisors.
{{storm.supported.jvms}} which is a map of the version of the JVM to the
JAVA_HOME path for it.
and
{{storm.legacy.worker.classpath}} which is a map of the version of storm to a
CLASSPATH that can be used to launch a legacy worker process.
They should be set for all supervisors and nimbus.
Then we also add in some metadata that the client submits to nimbus along with
a topology. Namely the version of the storm client they are running on and the
version of the JVM they are running on.
Nimbus can then decide (possibly through another config, but probably just
through convention with some config overrides) if the version of the client +
JVM is compatible with the version of storm + JVM currently on the cluster. If
so it should just let it through. If not it should pick a version of the JVM +
storm that is compatible. If there are none available it will reject the
request. We should also allow end users to set these configs.
For this to work well we need some good version matching/sorting code that is
lenient, even during rolling upgrades.
For example if a user submits a topology with a 1.0.3 client to a 2.0.0
cluster. Nimbus sees that it has 1.0.3 installed great it will start running
with that, but then we do a rolling upgrade of the cluster and upgrade move to
1.0.4. The supervisors should be able to launch the workers for that topology
with a 1.0.4 classpath.
As such part of the worker heartbeat should also include the version of storm +
JVM that the worker is running with.
We should display that on the UI and display the version it was submitted with
on the UI too.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
