Am 26.07.2010 13:05, schrieb Brett Henderson:
I'm happy to apply it, just needs a few cleanups first.
* If you run "ant clean all" from the replication directory you'll
notice it fails at the checkstyle step (You may need to run "ant
publish" from the root directory first). Some code formatting
needs to be cleaned up to fix the checkstyle errors.
* Kinda picky, but while you're fixing the checkstyle errors, might
be worth fixing some typos in the comments (locale->local,
beeing->being, etc).
* One comment states "If local state isn't available we need to copy
server state to be the initial local state then exit.". That
comment has been copied from my code but it's not true for your
implementation. That threw me off course when reading it for the
first time.
* The munin config files can't live in the source tree. They
definitely won't be included in the final package if they're
there, they'll only get embedded inside the
osmosis-replication.jar file which isn't much help. They'll need
to live in the "package" project somewhere. There's already a
"script" directory which gets bundled with the release, would a
"munin" directory in there be appropriate?
I fixed all this and attached a new version of the patch. Thank you for
reviewing it. If you still see typos I'm sorry about it - i don't have
eclipse and thus no automatic spell checking - and I'm no native
speaker, so sorry for them.
* And finally, once it's been committed you'll need to document it
on the 0.36 release wiki page
http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.36
including references to the munin files.
That shall not be the problem.
Peter
Index: package/script/munin/osm-replication-lag
===================================================================
--- package/script/munin/osm-replication-lag (Revision 0)
+++ package/script/munin/osm-replication-lag (Revision 0)
@@ -0,0 +1,35 @@
+#!/bin/sh
+# -*- sh -*-
+
+# load the munin plugin helper
+. $MUNIN_LIBDIR/plugins/plugin.sh
+
+# if no workingDirectory has been configures
+if [ ! $workingDirectory ]; then
+ # exit with an error
+ echo "no workingDirectory configured" >&2
+ exit 1
+fi
+
+# path to osmosis binary
+[ $osmosis ] || osmosis="osmosis"
+
+# configuration section
+if [ "$1" = "config" ]; then
+
+ echo 'graph_title OSM PostGIS Database Replag'
+ echo 'graph_args --base 1000'
+ echo 'graph_vlabel seconds behind main database'
+ echo 'graph_category osm'
+
+ echo 'lag.label replication lag'
+ echo 'lag.draw LINE'
+
+ exit 0
+fi
+
+# invoke osmosis to calculate the replication lag
+lag=$($osmosis --read-replication-lag workingDirectory="$workingDirectory"
2>/dev/null)
+echo "lag.value=$lag"
+
+exit 0
Index: package/script/munin/osm-replication.conf
===================================================================
--- package/script/munin/osm-replication.conf (Revision 0)
+++ package/script/munin/osm-replication.conf (Revision 0)
@@ -0,0 +1,15 @@
+[osm*]
+
+# the osmosis invocation may take some time
+timeout 60
+
+# the system user that has access to the working directory, if it'S different
+# from "munin"
+#user osm
+
+# path to the osmosis binary. if not set, osmosis is assumed to be in $PATH
+#env.osmosis /opt/osmosis/bin/osmosis
+
+# working directory of the osmosis replication.
+# this must be set to make the munin plugin work
+#env.workingDirectory /path/to/state.txt
Index: package/script/munin/README
===================================================================
--- package/script/munin/README (Revision 0)
+++ package/script/munin/README (Revision 0)
@@ -0,0 +1,7 @@
+to activate the munin plugins
+ - copy "osm-replication-lag" to "/usr/share/munin/plugins"
+ - make "/usr/share/munin/plugins/osm-replication-lag" executable
+ - symlink "/usr/share/munin/plugins/osm-replication-lag" to
"/etc/munin/plugins"
+ - copy "osm-replication.conf" to "/etc/munin/plugin-conf.d"
+ - edit "/etc/munin/plugin-conf.d/osm-replication.conf" and set the
workingDirectory
+ - restart the munin-node
Index:
replication/src/org/openstreetmap/osmosis/replication/ReplicationPluginLoader.java
===================================================================
---
replication/src/org/openstreetmap/osmosis/replication/ReplicationPluginLoader.java
(Revision 22443)
+++
replication/src/org/openstreetmap/osmosis/replication/ReplicationPluginLoader.java
(Arbeitskopie)
@@ -12,6 +12,7 @@
import
org.openstreetmap.osmosis.replication.v0_6.ReplicationDownloaderInitializerFactory;
import org.openstreetmap.osmosis.replication.v0_6.ReplicationFileMergerFactory;
import
org.openstreetmap.osmosis.replication.v0_6.ReplicationFileMergerInitializerFactory;
+import org.openstreetmap.osmosis.replication.v0_6.ReplicationLagReaderFactory;
/**
@@ -43,6 +44,9 @@
factoryMap.put("merge-replication-files-init", new
ReplicationFileMergerInitializerFactory());
factoryMap.put("mrfi", new
ReplicationFileMergerInitializerFactory());
+ factoryMap.put("read-replication-lag", new
ReplicationLagReaderFactory());
+ factoryMap.put("rrl", new ReplicationLagReaderFactory());
+
factoryMap.put("read-change-interval-0.6", new
IntervalDownloaderFactory());
factoryMap.put("read-change-interval-init-0.6", new
IntervalDownloaderInitializerFactory());
factoryMap.put("read-replication-interval-0.6", new
ReplicationDownloaderFactory());
Index:
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReaderFactory.java
===================================================================
---
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReaderFactory.java
(Revision 0)
+++
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReaderFactory.java
(Revision 0)
@@ -0,0 +1,57 @@
+// This software is released into the Public Domain. See copying.txt for
details.
+package org.openstreetmap.osmosis.replication.v0_6;
+
+import java.io.File;
+
+import org.openstreetmap.osmosis.core.pipeline.common.RunnableTaskManager;
+import org.openstreetmap.osmosis.core.pipeline.common.TaskConfiguration;
+import org.openstreetmap.osmosis.core.pipeline.common.TaskManager;
+import org.openstreetmap.osmosis.core.pipeline.common.TaskManagerFactory;
+
+
+/**
+ * The task manager factory for a replication lag reader.
+ *
+ * @author Peter Körner
+ */
+public class ReplicationLagReaderFactory extends TaskManagerFactory {
+ private static final String ARG_HUMAN_READABLE = "humanReadable";
+ private static final boolean DEFAULT_HUMAN_READABLE = false;
+ private static final String ARG_WORKING_DIRECTORY = "workingDirectory";
+ private static final String DEFAULT_WORKING_DIRECTORY = "./";
+
+
+ /**
+ * {...@inheritdoc}
+ */
+ @Override
+ protected TaskManager createTaskManagerImpl(TaskConfiguration
taskConfig) {
+ String workingDirectoryString;
+ boolean humanReadableFlag;
+ File workingDirectory;
+
+ // Get the task arguments.
+ workingDirectoryString = getStringArgument(
+ taskConfig,
+ ARG_WORKING_DIRECTORY,
+ getDefaultStringArgument(taskConfig,
DEFAULT_WORKING_DIRECTORY)
+ );
+ humanReadableFlag = getBooleanArgument(
+ taskConfig,
+ ARG_HUMAN_READABLE,
+ DEFAULT_HUMAN_READABLE
+ );
+
+ // Convert argument strings to strongly typed objects.
+ workingDirectory = new File(workingDirectoryString);
+
+ return new RunnableTaskManager(
+ taskConfig.getId(),
+ new ReplicationLagReader(
+ workingDirectory,
+ humanReadableFlag
+ ),
+ taskConfig.getPipeArgs()
+ );
+ }
+}
Index:
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReader.java
===================================================================
---
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReader.java
(Revision 0)
+++
replication/src/org/openstreetmap/osmosis/replication/v0_6/ReplicationLagReader.java
(Revision 0)
@@ -0,0 +1,162 @@
+// This software is released into the Public Domain. See copying.txt for
details.
+package org.openstreetmap.osmosis.replication.v0_6;
+
+import java.io.File;
+import java.util.Properties;
+import java.util.logging.Logger;
+import java.text.MessageFormat;
+
+import org.openstreetmap.osmosis.core.OsmosisRuntimeException;
+import org.openstreetmap.osmosis.core.task.common.RunnableTask;
+import org.openstreetmap.osmosis.core.util.FileBasedLock;
+import org.openstreetmap.osmosis.core.util.PropertiesPersister;
+import org.openstreetmap.osmosis.replication.common.ReplicationState;
+import org.openstreetmap.osmosis.replication.common.ServerStateReader;
+import
org.openstreetmap.osmosis.replication.v0_6.impl.ReplicationDownloaderConfiguration;
+
+/**
+ * Compares the timestamp of a local replication directory and the timestamp
on the
+ * HTTP server that is configured to provide the replication files. It
calculates
+ * the number of seconds the local replication directory is behind the HTTP
server
+ * and prints it to stdout.
+ *
+ * @author Peter Körner
+ */
+public class ReplicationLagReader implements RunnableTask {
+ private static final Logger LOG =
Logger.getLogger(ReplicationLagReader.class.getName());
+ private static final String LOCK_FILE_NAME = "download.lock";
+ private static final String CONFIG_FILE = "configuration.txt";
+ private static final String LOCAL_STATE_FILE = "state.txt";
+
+ private boolean humanReadable;
+ private File workingDirectory;
+ private ServerStateReader serverStateReader;
+
+ /**
+ * Creates a new instance.
+ *
+ * @param workingDirectory
+ * The directory containing configuration and tracking files.
+ * @param humanReadable
+ * Print the replication lag in a Hours, Minutes and Seconds
+ * instead of the raw number of seconds
+ */
+ public ReplicationLagReader(File workingDirectory, boolean
humanReadable) {
+ this.workingDirectory = workingDirectory;
+ this.humanReadable = humanReadable;
+
+ serverStateReader = new ServerStateReader();
+ }
+
+
+ /**
+ * Calculate the replication lag and print it to stdout
+ */
+ private void getLag() {
+ ReplicationDownloaderConfiguration configuration;
+ ReplicationState serverState;
+ ReplicationState localState;
+ PropertiesPersister localStatePersistor;
+ Properties localStateProperties;
+
+ // Instantiate utility objects.
+ configuration = new ReplicationDownloaderConfiguration(new
File(workingDirectory, CONFIG_FILE));
+
+ // Obtain the server state.
+ LOG.fine("Reading current server state.");
+ serverState =
serverStateReader.getServerState(configuration.getBaseUrl());
+
+ // Build the local state persister which is used for both
loading and storing local state.
+ localStatePersistor = new PropertiesPersister(new
File(workingDirectory, LOCAL_STATE_FILE));
+
+ // If local state isn't available we need to fail because no
lag can be calculated.
+ if (!localStatePersistor.exists()) {
+ throw new OsmosisRuntimeException("Can't read local
state.");
+ }
+
+ // fetch the local state from the file
+ localStateProperties = localStatePersistor.load();
+ localState = new ReplicationState(localStateProperties);
+
+ // extract the time of the local and the remote state files
+ long local = localState.getTimestamp().getTime();
+ long server = serverState.getTimestamp().getTime();
+
+ // we assume the server being before the local state while
calculating the difference
+ long lag = (server - local) / 1000;
+
+ // check if a human readable version is requested
+ if (this.humanReadable) {
+
+ if (lag > 86400) {
+
+ // more than a day
+ Object[] args = {
+ new Long(lag / 86400),
+ new Long((lag % 86400) / 3600)
+ };
+ System.out.println(
+ new MessageFormat("{0} day(s) and {1}
hour(s)").format(args)
+ );
+
+ } else if (lag > 3600) {
+
+ // morte than an hour
+ Object[] args = {
+ new Long(lag / 3600),
+ new Long((lag % 3600) / 60)
+ };
+ System.out.println(
+ new MessageFormat("{0} hour(s) and {1}
minute(s)").format(args)
+ );
+
+ } else if (lag > 60) {
+
+ // more than a minute
+ Object[] args = {
+ new Long(lag / 60),
+ new Long(lag % 60)
+ };
+ System.out.println(
+ new MessageFormat("{0} minute(s) and
{1} second(s)").format(args)
+ );
+
+ } else {
+
+ // just some seconds
+ System.out.println(
+ new MessageFormat("{0}
second(s)").format(lag)
+ );
+
+ }
+
+ } else {
+
+ // print out the raw number of seconds
+ System.out.println(lag);
+
+ }
+ }
+
+
+ /**
+ * {...@inheritdoc}
+ */
+ @Override
+ public void run() {
+ FileBasedLock fileLock;
+
+ fileLock = new FileBasedLock(new File(workingDirectory,
LOCK_FILE_NAME));
+
+ try {
+ fileLock.lock();
+
+ getLag();
+
+ fileLock.unlock();
+
+ } finally {
+ fileLock.release();
+ }
+ }
+}
_______________________________________________
osmosis-dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/osmosis-dev
