On 12/08/2010 11:37 PM, Jonathan Robie wrote:
I was talking with Ted about qpid-config and qpid-route, and we
thought it might be useful to have a simple, consistent syntax using
one set of conventions for both utilities.
The syntax you propose is really only more consistent in that the verb
and the noun are in the same order (which certainly makes sense). Other
than that there isn't a great deal of consistency between, for example:
declare exchange a topic
and
declare route from broker=localhost:10001 exchange=amq.topic
key=global.# to broker=localhost:10002
In some sense qpid-config and qpid-route actually have a different
purpose. The former is concerned with listing, creating or deleting
objects on a single broker. The latter, it seems to me, is concerned
with presenting a higher level conceptual view of routing between two
(or more) brokers (how successful it is in that is open to debate).
I think before changing anything like this we should start by
re-examining the rationale for each tool we have.
Some other developers suggested that using optparse for all the Python
command line utilities might lead to more consistency. That helps with
most utilities, but not with qpid-config and qpid-route, which each
have a syntax based on sequences of strings.
They do in addition have lots of options, however and a consistent
approach to those is still relevant.
The use of optparse throughout primarily unifies and simplifies the code
for maintainers. The benefit to users is that it avoids inconsistent
weirdness in actual option *parsing*; it doesn't in and of itself result
in consistent options being chosen. Both things are beneficial however.
Interpretation of arguments (as opposed to options) is a third issue
(and is certainly also important).
In this discussion, it became clear that different people are thinking
of different use cases, and it's not completely clear what the real
requirements are or which solution would be best.
Here are some requirements that have been mentioned:
- We need a consistent syntax.
- A consistent API that corresponds to the commands these tools offer
would be helpful. All of the commands in qpid-config and qpid-route
are also useful within programs
- Machine-readable logging is important for some applications.
- It's important to be able to run these commands in batch mode. Start
up time is significant, so running a sequence of Python command-line
utilities in a batch file is slow.
One solution: improve start-up time.
(The use of the QMF API in qpid-config for example seems to add an order
of magnitude to the time taken to e.g. declare a queue over a simpler
program that does the same thing...)
Another solution: design a Python API for these tasks, and use a
Python script, with one connection, rather than a batch file.
Another solution: allow a sequence of qpid-config or qpid-route
commands to be placed in a file and executed in batch mode.
>
- Running commands interactively, in a command shell, and using the
same commands in a batch file. That makes it easier to write / debug
scripts.
What other requirements / use cases should we keep in mind?
One thing that none of the command line tools do at present is provide
an easy way to invoke arbitrary management methods (e.g. purge, move
etc). I recently came across this when adding a command to change the
log-level.
What other questions should we be asking?
The functionality we need to support is illustrated by the commands
below my signature. We might choose to support this syntax in some
form, or perhaps support an API that corresponds to these commands, or
perhaps support both, with a direct mapping between the two. Are there
other approaches we should consider?
I think we should have a standard approach to invoking management
commands (QMF methods at present)[1]. That in itself defines an API. We
then need to carefully consider how exactly to expose various operations
through the schema to ensure usability and consistency overall.
The querying and listing of management objects is the other part of the
picture[2]. Consistency there is fairly easy to achieve I think.
[1] I checked in a little test program to do this (currently in
cpp/src/tests/qpid-ctrl). I needed it specifically to test runtime
changes to log-level. An additional objective was to explore the use of
a message based approach to programmatic broker management (i.e.
initially at least, through direct use of QMFv2).
You get a minimal usage statement from qpid-ctrl --help. Essentially you
supply the QMF method name and then key-value pairs for any parameters
as arguments. There are options to choose the broker to connect to and
some other operational aspects. By default the methods are assumed to be
intended to be invoked on the Broker object itself E.g.
qpid-ctrl setLogLevel level=info+
qpid-ctrl getLogLevel
qpid-ctrl queueMoveMessages srcQueue=a destQueue=b qty=3
qpid-ctrl connect host=localhost port=5673
However you can also invoke methods on any object by providing the class
of the object and an identifier for it. E.g.
qpid-ctrl --class queue --id my-queue purge request=5
This tool would need quite a bit more work before it would be ready for
consideration as something other than a test utility and the details of
the implementation are not necessarily the best choices. However I think
its a good demonstration of the basic concept of focusing on a usable
and consistent management schema as a way to drive consistency and
uniformity in tool support for that schema.
[2] Attached is a *very* rough example program that uses QMFv2 to list
objects. This was really an experiment I did for my own purposes but it
may be of interest to this thread in conjunction with qpid-ctrl.
#!/usr/bin/env python
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
import optparse
from qpid.messaging import *
from qpid.util import URL
from qpid.log import enable, DEBUG, WARN
def list_map_entries(m):
r = ""
for t in m:
r += "%s=%s " % (t, m[t])
return r
def get_qmfv2_result(m):
if m.properties['x-amqp-0-10.app-id'] == 'qmf2':
if m.properties['qmf.opcode'] == '_method_response':
return m.content['_arguments']
elif m.properties['qmf.opcode'] == '_exception':
raise Exception("Error: %s" % list_map_entries(m.content['_values']))
else: raise Exception("Invalid response received, unexpected opcode: %s" %
m)
else: raise Exception("Invalid response received, not a qmfv2 method: %s" % m)
parser = optparse.OptionParser(usage="usage: %prog [options]",
description="List managed objects.")
parser.add_option("-b", "--broker", default="localhost",
help="connect to specified BROKER (default %default)")
parser.add_option("-c", "--class", dest="qmfclass", default="",
help="class of object on which command is being invoked
(default %default)")
parser.add_option("-p", "--package", default="",
help="package of object on which command is being invoked
(default %default)")
parser.add_option("-i", "--id", default="",
help="identifier of object on which command is being invoked
(default %default)")
parser.add_option("-a", "--address", default="qmf.default.direct/broker",
help="address to send commands to (default %default)")
parser.add_option("-t", "--timeout", type="float", default=5,
help="timeout in seconds to wait for response before exiting
(default %default)")
parser.add_option("-l", dest="detailed", action="store_true",
help="display all properties of listed objects")
parser.add_option("-v", dest="verbose", action="store_true",
help="enable logging")
opts, args = parser.parse_args()
if opts.verbose:
enable("qpid", DEBUG)
else:
enable("qpid", WARN)
conn = Connection(opts.broker)
try:
conn.open()
ssn = conn.session()
snd = ssn.sender(opts.address)
reply_to = "qmf.default.direct/%s; {node: {type: topic}}" % str(uuid4())
rcv = ssn.receiver(reply_to)
content = {"_what": "OBJECT"}
if opts.qmfclass:
if opts.id:
if opts.package:
package = opts.package
else:
package = "org.apache.qpid.broker"
object_name = "%s:%s:%s" % (package, opts.qmfclass, opts.id)
content["_object_id"] = {"_object_name": object_name}
else:
content["_schema_id"] = {"_class_name": opts.qmfclass}
if opts.package: content["_schema_id"]["_package_name"] = opts.package
msg = Message(reply_to=reply_to, content=content)
msg.properties["x-amqp-0-10.app-id"] = "qmf2"
msg.properties["qmf.opcode"] = "_query_request"
snd.send(msg)
try:
result = rcv.fetch(timeout=opts.timeout)
items = result.content
for i in items:
n = i["_object_id"]["_object_name"]
print n.split(':', 2)[2]
if opts.detailed:
props = i["_values"]
for k in props:
print " %s = %s" % (k, props[k])
except Empty:
print "No response received!"
except Exception, e:
print e
except ReceiverError, e:
print e
except KeyboardInterrupt:
pass
conn.close()
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
