[
https://issues.apache.org/jira/browse/S4-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129861#comment-13129861
]
Leo Neumeyer commented on S4-4:
-------------------------------
My notes from today's discussion:
- Matthieu will look at app loading
- Our problem is constrained because the entry point is always a class of type
App. All interaction with the app is via the App methods: ini,t start, close,
etc and the APi only uses known framework classes: Event, EventSource, etc.)
- Each will have its own classloader to make it easy to release and GC.
- we will test using Matthieu's forked node unit tests to validate
- create a separate minimal project as template for creating apps. Requirement:
output should be a single JAR file (we could call it S4R: HelloApp.s4r to
indicate that it has valid s4 attributes in the manifest).
- The S4R file is dropped in a dir. Server will load/unload based on what files
are in the dir.
- Loaded apps are tracked by Server. Apps must must have a unique ID. Here is
an idea: S4R includes properties to identify app: for example:
S4-App-Description: An app to count words.
S4-App-OwnerId: 79f0eea01c
...
Server (or ZK) will assign a unique ID (0,1,2,...). Server will map AppIDs to
AppModules. An AppModule has a reference to the loaded app, the S4-App-
properties, deployment time, and any other administrative info. In this way we
can always identify the loaded app and guarantee that there are unique IDs in
the cluster.
- No app to app communication in first milestone.
For later:
- Test app to app communication using EventSource.
- Coordinate load/unload using ZK and a deployment server.
- Restrict loaded App so they cannot execute System.exit(), they cannot create
threads, etc.
- Test app loading by creating lots of apps and releasing them, memory should
not grow indefinitely and we can probably track that apps get garbage collected.
- what happens if jar in app is diff than framework jar. It should use its own
jar without interfering with system jar.
> Dynamic App Loading
> -------------------
>
> Key: S4-4
> URL: https://issues.apache.org/jira/browse/S4-4
> Project: Apache S4
> Issue Type: New Feature
> Reporter: Leo Neumeyer
>
> We want to be able to load and unload applications dynamically. Here is some
> background information:
> Once a server is started, we expect it to run indefinitely until there is a
> failure or we need to upgrade the framework software or we change the cluster
> configuration. In S4, the cluster is symmetric, that is, all the nodes are
> identical, with the same code base and the same configuration.
> Applications are loaded dynamically after the cluster is started. If app1
> depends on app2, then app2 needs to use an EventSource to publish events.
> Apps must be able to find EventSource and subscribe to it during
> initialization.
> When an app is unloaded, it will close its EventSource(s). This in turn, will
> close all the streams that are subscribed. The apps that own the streams are
> responsible for taking action once their dependencies are gone.
> Possible roadmap:
> M1:
> - Server starts with no apps.
> - Apps are loaded during initialization, for example by searching an
> app directory.
> - Apps are initialized (no dependencies among apps).
> - Apps are started.
> M2:
> - Add dependencies among apps using EventSource.
> M3:
> - Add runtime loading/unloading functionality
> Leo implemented a solution using JBOSS Modules which seems simpler than using
> OSGI. Modules seems to work but is not well documented which makes it hard to
> use without understanding the low level details. See the commented out code
> here:
> https://github.com/leoneu/s4-piper/blob/master/subprojects/s4-core/src/main/java/io/s4/core/Server.java
> From Adam:
> Yes I have a few ideas but I need couple days to implement them.
> First, is to build S4 and examples as proper OSGi bundles using gradle osgi
> plugin http://gradle.org/osgi_plugin.
> Its using bnd tool http://www.aqute.biz/Bnd/Bnd.
> Then embed osgi container (using standard osgi api) in Controller instead of
> JBoss modules see
> http://felix.apache.org/site/apache-felix-framework-launching-and-embedding.html.
> and configure directory from which felix should auto deploy our examples
> http://felix.apache.org/site/apache-felix-framework-usage-documentation.html#ApacheFelixFrameworkUsageDocumentation-autodeploy.
> For dynamic deployment we can use file install bundle
> http://felix.apache.org/site/apache-felix-file-install.html, it will watch
> directory from which it loads new bundles.
> To configure properly examples we can use ipojo
> http://felix.apache.org/site/apache-felix-ipojo.html or declarative services
> http://felix.apache.org/site/apache-felix-service-component-runtime.html,
> they both using service component model and are not intrusive in the code.
> That will be enough to load properly the apps.
> Later we have to think how we want to use osgi in s4, just to load apps or
> properly use service management. Bundles are hiding the internals from other
> bundles and should communicate through well defined services. And also I
> don't have clear picture how s4 is going to be distributed (have to look more
> at https://github.com/s4/s4). Maybe you can write me some small description
> of s4 concept.
> ...
> I've implemented ideas which I sent you before. Now I can load the apps using
> embedded osgi container but there is still class loading issue
> (com.esotericsoftware.kryo.SerializationException: Unable to find class:
> io.s4.example.counter.UserEvent). To be able to run s4 I had to put plenty
> system packages in configuration file for felix.
> Around 15 jars defined in libraries are not osgi bundles (don't have proper
> manifest file). The biggest issue are dependencies and classloading (for
> example Kryo is trying to load UserEvent class but that class is not visible
> for him so Class.forName is not going to work).I will try to fix these
> problems, in the meantime you can have a look at my repo
> https://github.com/adamwojtuniak/s4-piper. I added task createRepo, it will
> create bundles directory with all dependencies and project jars. When you
> launch Main class from s4-core, it launches embedded felix which will auto
> deploy all bundles located in bundles directory. In the console you will see
> felix gogo shell ( http://www.packtpub.com/article/apache-felix-gogo ).
> From Henry:
> Ugh, I was actually trying to avoid OSGI if I can =)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
