Hello,
I got sick and tired of mmbase not wanting to run as a war, and clumsy loading of
configuration
files.
So I decided to do something about it, and I think I nearly succeeded. I am going to
offer it as a hack on CVS
HEAD, but first, I'd like to start a bit of discussion on the subject, to avoid
harvesting unnecessary
-1's because of implementation details.
Actually, the problem is not very big, because most configuration files are handled by
one base
class (org.mmbase.util.xml.DocumentReader), and the idea was that if that class could
be a bit
smarter lots of the issues would be solved.
Of course there would remain some problems, because on a few other spots in the MMBase
code explicit
file handling is done. Most noticebly when loading the modules and builders because
that depends on
File#listFiles.
Some other configuration implementations like those of logging and security also
proved to depend
explicitely on Files but those can relatively easily be changed.
The idea is to delegate all this stuff to one new class. This class I called
'ResourceLoader' and
extends from ClassLoader. You load a resource from it approximately like this:
InputStream loggingConfiguration =
ResourceLoader.getRoot().getResourceAsStream("/log/log.xml");
or, because we need XML all the time:
InputSource loggingConfiguration =
ResourceLoader.getRoot().getInputSource("/log/log.xml");
There are also non-root ResourceLoaders, wich may come in handy when resolving
'relative' XML.
Under the hood, it works with a specialized protocol for URL object ('mm:', the
precise string for this
protocol is open for discussion)
This code will first look for a file WEB-INF/config/log/log.xml and if not exists then
for the file
WEB-INF/classes/mmbase/config/log/log.xml and if that too does not exist, then for the
resource
mmbase/config/log/log.xml.
The location of this resource-base is open for discussion. I tried to go for a short
path, so e.g. I
let away the 'org' (these are not java classes any way).
Also something must be thought of for loading modules and builders as a resource.
I came up with this:
Set builders = builderLoader.getSubResources(ResourceLoader.XML_PATTERN, true/*
recursive*/, ResourceLoader.INDEX);
log.info("Loading " + builders);
Iterator i = builders.iterator();
while (i.hasNext()) {
So, besides all files of WEB-INF/config/builders it also loads all builders mentioned
in the
resource ResourceLoader.INDEX (which I now set to "INDEX", also open to discussion).
When implemented we gain the following:
- The possibility of running on a war. This is really quite confortable. You can
point in the
tomcat-manager to the war, click 'deploy' and you have a running MMBase. No need to
restart
tomcat any more. It will be even easier to try MMBase or distribute your own MMBase
web-app.
I think monday or tuesday I will post an URL to an mmbase.war and you can try this
out.
- The possibility to have several mmbase config repositories. I e.g. tried to put a
mmbase-config.jar containing only the core-builder and other stuff I'm not that
interested
in to see in my config dir. That worked fine. I imagine that I will develop sites
starting with a
my-site-framework.jar containing all the builders and configuration I'd always want
(like
e.g. cloud context security)
We could e.g. also add the essential builders like object.xml and typedef.xml to
mmbase.jar
itself, and we will have ensured that you cannot not have those builders.
Of course the fall-back-mechanism will still ensure that you can still override
them (placing
your own versions as file).
- We will have decided how resource-loading _must_ be done in MMBase. I recall the
discussion about
the location of the storage resources. Every resource will be loaded in the same
way, and there
will be one clearly defined path to it (though it still can have very many
locations).
Impact
- Most configuration code will need a simple rewrite. MMBase.getConfigPath() will no
longer give an
absolute path. If this last thing is seen as a problem, it could merely be
deprecated. and a
similar functions could be added for use with ResourceLoader. Then at least old
code using
getConfigPath will remain to work (but of course you cannot run a war with that
code).
Some configuration code will be quite simplified.
Remaining problems:
- Reading of web.xml of MMBaseServlet will not be possible any-more. It currently
gives an
stack-trace in my log, (but mmbase starts oterwise correctly). It is only used for
that silly
function to servlet mapping. I'm sure we can come up with some solution (if needed,
by a copy of
web.xml under WEB-INF/classes...)
- File-watching. I plan to implement ResourceWatcher based on FileWatcher. This will
make sure that
the onChange are only called when really necessary. (e.g. if
WEB-INF/classes/mmbase/config/log/log.xml changes and WEB-INF/config/log/log.xml
exists, this
event must be ignored). I don't foresee difficulties with this.
How could you further improve on this:
- Since it is implemented as a ClassLoader, you could also decide to actually load
classes with
it. For applications you could e.g. decide to put jars with classes and
configurations in
WEB/classes/mmbase/jars. We can now decide to search for configuration in those
jars to, and load
all classes. For example you could add fieldtypedefinition.xml's to those jars and
those can all
be loaded and merged to the 'field type definition map'. You can then also
implement that
resources in such jars override those in mmbase.jar (which is not possible when
depending on the
standard ClassLoader).
Keesj problably has quite nice ideas about this.
- Since this class-loader is MMBase specific it would also be possible to instruct it
how to load
resources and ever classes from the MMBase database. You could e.g. store
builder-xml and even
jars in the database. For some reason. I can imagine that can come in quite handy
in some
packaging implementation for those installations which don't have write-permession
on the
file-system or so.
For reference, I'll attach the current version of my 'ResourceLoader.java'.
Michiel
--
Michiel Meeuwissen mihxil'
Mediacentrum 140 H'sum [] ()
+31 (0)35 6772979 nl_NL eo_XX en_US
/*
This software is OSI Certified Open Source Software.
OSI Certified is a certification mark of the Open Source Initiative.
The license (Mozilla version 1.0) can be read at the MMBase site.
See http://www.MMBase.org/license
*/
package org.mmbase.module.core;
import java.io.*;
import java.util.*;
import java.util.regex.Pattern;
import java.net.*;
import javax.servlet.ServletContext;
import org.xml.sax.InputSource;
import org.mmbase.util.logging.Logger;
import org.mmbase.util.logging.Logging;
/**
* MMBase resource loader, for loading config-files and those kind of things. It knows about MMBase config file locations.
*
* I read [EMAIL PROTECTED] http://www.javaworld.com/javaqa/2003-08/02-qa-0822-urls.html}
*
* It is in this package because I figured that only MMBaseContext may call [EMAIL PROTECTED] #init}, but
* perhaps that is a unnecessary restriction, and perhaps org.mmbase.util. with be a better
* location, then.
*
*
* @author Michiel Meeuwissen
* @since MMBase-1.8
* @version $Id: $
*/
public class ResourceLoader extends ClassLoader {
private static final Logger log = Logging.getLoggerInstance(ResourceLoader.class);
public static final String PROTOCOL = "mm";
public static final String RESOURCE_ROOT = "/mmbase/config";
public static final String INDEX = "INDEX";
private static final MMURLStreamHandler mmStreamHandler = new MMURLStreamHandler();
private static final ResourceLoader root = new ResourceLoader();
/**
* Creates a new URL object, which is used to load resources. First a normal java.net.URL is
* instantiated, if that fails, we check for the 'mmbase' protocol. If so, a URL is instantiated
* with a URLStreamHandler which can handle that.
*
* If that too fails, it should actually already be a MalformedURLException, but we try
* supposing it is some existing file and return a file: URL. If no such file, only then a
* MalformedURLException is thrown.
*/
protected static URL newURL(String url) throws MalformedURLException {
// Try already installed protocols first:
try {
return new URL (url);
} catch (MalformedURLException ignore) {
// Ignore: try our own handler next.
}
final int firstColon = url.indexOf (':');
if (firstColon <= 0) {
if (new File(url).exists()) return new URL("file:" + url); // try it as a simply file
throw new MalformedURLException ("No protocol specified: " + url);
} else {
final String protocol = url.substring (0, firstColon);
if (protocol.equals(PROTOCOL)) {
return new URL (null/* no context */, url, mmStreamHandler);
} else {
if (new File(url).exists()) return new URL("file:" + url);
throw new MalformedURLException ("Unknown protocol: " + protocol);
}
}
}
private static List /* <File> */ fileRoots = new ArrayList();
private static List /* <String> */ resourceRoots = new ArrayList();
static {
// make sure it works a bit before servlet-startup.
init(null);
}
/**
* Initializes the Resourceloader using a servlet-context (makes e.g. resolving relatively to WEB-INF/config possible).
* @param servletContext The ServletContext used for determining the mmbase configuration directory. Or <code>null</code>.
*/
static void init(ServletContext servletContext) {
fileRoots.clear();
resourceRoots.clear();
if (servletContext != null) {
String s = servletContext.getRealPath("/WEB-INF/config");
if (s != null) {
fileRoots.add(new File(s));
}
s = servletContext.getRealPath("/WEB-INF/classes" + RESOURCE_ROOT); // prefer opening as a files.
if (s != null) {
fileRoots.add(new File(s));
}
}
// mmbase.config settings
String configPath = null;
if (servletContext != null) {
configPath = servletContext.getInitParameter("mmbase.config");
}
if (configPath == null) {
configPath = System.getProperty("mmbase.config");
}
if (configPath != null) {
if (servletContext != null) {
// take into account configpath can start at webrootdir
if (configPath.startsWith("$WEBROOT")) {
configPath = servletContext.getRealPath(configPath.substring(8));
}
}
fileRoots.add(new File(configPath));
}
if (fileRoots.size() == 0) {
File [] roots = File.listRoots();
fileRoots.addAll(Arrays.asList(roots));
}
resourceRoots.add(RESOURCE_ROOT);
}
/**
* Returns a set of File object for a given resource (relative to the file roots).
* @param path A path relative to the fileRoots
* @return A List.
*/
protected static List getRootFiles(final String path) {
return new AbstractList() {
String p = path;
public int size() { return fileRoots.size(); }
public Object get(int i) { return new File((File) fileRoots.get(i), p); }
};
}
/**
* The one ResourceLoader which loads from the mmbase conifg root is static, and can be obtained with this method
*/
public static ResourceLoader getRoot() {
return root;
}
/**
* The URL relative to which this class-loader resolves. Cannot be <code>null</code>.
*/
private URL context;
/**
* This constructor instantiates the root resource-loader. There is only one such ResourceLoader
* (acquirable with [EMAIL PROTECTED] #getRoot}) so this constructor is private.
*/
private ResourceLoader() {
super();
try {
context = newURL(PROTOCOL + ":/");
} catch (MalformedURLException mue) {
throw new RuntimeException(mue);
}
}
/**
* Instantiates a new ResourceLoader relative to the root ResourceLoader.
*/
public ResourceLoader(final String context) {
this(getRoot(), context);
}
/**
* Instantiates a ResourceLoader for a 'sub directory' of given ResourceLoader
*/
public ResourceLoader(final ResourceLoader cl, final String context) {
super();
this.context = cl.findResource(context + "/");
}
public ResourceLoader(final URL context) {
super();
this.context = context;
}
/**
* If name starts with '/' the root resourceloader is used.
* If name starts with <protocol>: a new [EMAIL PROTECTED] java.net.URL} will be created.
* Otherwise the name is resolve relatively. (For the root ResourceLoader that it the same as starting with /)
*
* [EMAIL PROTECTED]
*/
public URL findResource(final String name) {
try {
if (name.startsWith("/")) {
return newURL(name);
} else {
return new URL(context, name);
}
} catch (MalformedURLException mfue) {
log.info(mfue);
return null;
}
}
/**
* Can be used as an argument for [EMAIL PROTECTED] #getSubResource(Pattern, boolen)}. MMBase works mainly
* with xml configuration files, so this comes in handy.
*/
public static final Pattern XML_PATTERN = Pattern.compile(".*\\.xml");
/**
* Returns a set of 'sub resources' (read: 'files in the same directory'), which can succesfully be be loaded by the ResourceLoader.
* It only considers real files, so this list is not complete.
*
* @param pattern A Regular expression pattern to which the file-name must match, or <code>null</code> if no restrictions apply
* @param recursive If true, then also subdirectories are searched.
* @return A Set of Strings which can be successfully loaded with the resourceloader.
*/
public Set getSubResources(final Pattern pattern, final boolean recursive) {
FilenameFilter filter = new FilenameFilter() {
public boolean accept(File dir, String name) {
File f = new File(dir, name);
return pattern == null || (f.isDirectory() && recursive) || pattern.matcher(f.toString()).matches();
}
};
return getSubResources(filter, recursive ? "" : null);
}
/**
* The set of [EMAIL PROTECTED] #getSubResources(Pattern, boolean)} is merged with all entries of the
* resource with the given index, which is a simply list of resources.
*
* @param pattern A Regular expression pattern to which the file-name must match, or <code>null</code> if no restrictions apply
* @param recursive If true, then also subdirectories are searched.
* @param index An index of resources, if this index cannot be loaded, it will be ignored.
* @return A Set of Strings which can be successfully loaded with the resourceloader.
*/
public Set getSubResources(final Pattern pattern, final boolean recursive, String index) {
Set result = getSubResources(pattern, recursive);
InputStream inputStream = getResourceAsStream(index);
if (inputStream != null) {
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
try {
String line = reader.readLine();
while (line != null) {
result.add(line);
line = reader.readLine();
}
} catch (IOException ioe) {
}
} else {
}
return result;
}
/**
* Used by [EMAIL PROTECTED] #getSubResource(Pattern, boolean}. This is the function which does the
* recursion.
*/
protected Set getSubResources(FilenameFilter filter, String recursive) {
Set results = new LinkedHashSet(); // a set with fixed iteration order
Iterator i = getFiles(recursive == null ? "" : recursive).iterator();
while (i.hasNext()) {
File f = (File) i.next();
if (f.isDirectory()) { // should always be true
File [] files = f.listFiles(filter);
for (int j = 0; j < files.length; j++) {
if (recursive != null) {
if (files[j].isDirectory()) {
results.addAll(getSubResources(filter, files[j].getName() + "/"));
} else {
results.add(recursive + files[j].getName());
}
} else {
results.add(files[j].getName());
}
}
}
}
return results;
}
/**
* Returns the givens resource as a InputSource (XML streams). ResourceLoader is often used for
* XML.
* The System ID is set, otherwise you could as wel do new InputSource(r.getResourceAsStream());
* @param name The name of the resource to be loaded
* @return The InputSource if succesfull, <code>null</code> otherwise.
*/
public InputSource getInputSource(String name) {
try {
URL url = findResource(name);
InputStream stream = url.openStream();
if (stream == null) return null;
InputSource is = new InputSource(stream);
//is.setCharacterStream(new InputStreamReader(stream));
is.setSystemId(url.toExternalForm());
return is;
} catch (MalformedURLException mfue) {
log.info(mfue);
return null;
} catch (IOException ieo) {
log.error(ieo);
return null;
}
}
/**
* @return A List of all files associated with the resource
*/
public List getFiles(String name) {
URL url = findResource(name);
return getRootFiles(url.getPath());
}
public String toString() {
return "" + context.getProtocol() + ":" + context.getPath() + " fileroots:" + fileRoots + " resourceroots: " + resourceRoots;
}
/***
* The MMURLStreamHandler is a StreamHandler for the protocol PROTOCOL.
*/
protected static class MMURLStreamHandler extends URLStreamHandler {
MMURLStreamHandler() {
super();
}
protected URLConnection openConnection(URL u) throws IOException {
return new MMURLConnection(u);
}
/**
* mm: cannot be an external form, so the 'external' form of that will be
* http://www.mmbase.org/mmbase/config
*
* ExternalForms are mainly used in entity-resolving.
* [EMAIL PROTECTED]
*/
protected String toExternalForm(URL u) {
return "http://www.mmbase.org/mmbase/config"; + u.getPath();
}
}
/**
* Implements the logic for our MM protocol.
*/
protected static class MMURLConnection extends URLConnection {
private URL url;
private InputStream stream = null;
MMURLConnection(URL url) {
super(url);
//log.debug("Connection to " + url + Logging.stackTrace(new Throwable()));
if (! url.getProtocol().equals(PROTOCOL)) {
throw new RuntimeException("Only supporting URL's with protocol " + PROTOCOL);
}
this.url = url;
}
/**
* [EMAIL PROTECTED]
*/
public void connect() throws IOException {
if (stream == null) {
Iterator files = ResourceLoader.getRootFiles(url.getPath()).iterator();
while (files.hasNext()) {
File file = (File) files.next();
if (file.exists()) {
stream = new FileInputStream(file);
log.debug("Found file " + file);
break;
}
}
if (stream == null) {
Iterator resources = ResourceLoader.resourceRoots.iterator();
while (resources.hasNext()) {
String root = (String) resources.next();
stream = ResourceLoader.class.getResourceAsStream(root + url.getPath());
if (stream != null) {
log.debug("Found resource " + root + url.getPath());
break;
}
}
}
if (stream != null) {
connected = true;
}
}
}
/**
* [EMAIL PROTECTED]
*/
public InputStream getInputStream() throws IOException {
connect();
return stream;
}
}
/**
* For testing purposes only
*/
public static void main(String[] argv) {
ResourceLoader cl = getRoot();
try {
String arg = argv[0];
if (argv.length > 1) {
cl = new ResourceLoader(argv[0]);
arg = argv[1];
}
InputStream resource = cl.getResourceAsStream(arg);
if (resource == null) {
System.err.println("No such resource " + arg + " for " + cl);
return;
}
System.out.println("-------------------- resolved " + arg + " with " + cl + ": ");
BufferedReader reader = new BufferedReader(new InputStreamReader(resource));
while(true) {
String line = reader.readLine();
if (line == null) break;
System.out.println(line);
}
} catch (Exception mfeu) {
System.err.println(mfeu.getMessage() + Logging.stackTrace(mfeu));
}
}
}
