On 15/10/2019 04:47, Michael Pittaro wrote:
I've been playing around with the UDP server capability in wsjt-x, and
I think I've got most of the lower level details figured out. However,
I got blocked on the schema negotiation part, which raised a deeper
question.
I have been assuming wsjt-x is a server, and my code is a client. So
I sent heartbeats like a client, and I was expecting some sort of
response back, which never arrived. I definitely see traffic
(heartbeats, decodes, status, etc.) from wsjt-x, using the WSJT-X
client id.
Digging deeper, it seems wsjt-x is acting more like a client. The
docs also imply that. For example, The location message is described
as in, allowing a 'server' to set the location. Logged ADIF is
described as out, being sent to servers when a QSO is logged. The
example message aggregator is also a server, listening to wsjt-x and
other clients.
I think I'm just missing something obvious here in terms of the frame
of reference for client and server, and what the client/server model
looks like. Should my code act like a client or a server?
So far, I'm I'm using unicast, with just wsjt-x and my code running.
mike, kj6vcp
Hi Mike,
the terminology of client and server are somewhat loose in UDP protocols
but with the WSJT-X UDP protocol WSJT-X is definitely a client and any
application wishing to interoperate with WSJT-X is a server. The
interoperating application should listen on a well known address and
port which is the address and port configured in one or more instances
of WSJT-X via "Settings->Reporting->UDP Server", i.e. many WSJT-X
instances (clients) interoperate with one (or more, see below) servers
listening on well know address and port pairs.
With unicast UDP there can only be one server per address and port pair
(UDP broadcast does not have that limitation but has other severe
routing limitations that make it unsuitable for the WSJT-X protocol),
this is a fundamental limitation of IP addressing and information
transport. WSJT-X expects servers to implement multicast group UDP
address membership, this removes the limitation of there only being one
server per address and port pair. Unfortunately almost all applications
that have so far been written to interoperate with WSJT-X have failed to
implement multicast group membership which means that they are not good
citizens in the WSJT-X protocol because they consume messages that would
be sent to all multicast group members. This means that using such an
application precludes other applications joining the party without
over-complex, and only partially successful, message forwarding
arrangements.
The reference applications provided with WSJT-X (message_aggregator and
udp_daemon) do support multicast group membership and can be easily used
to demonstrate the protocol working with multiple WSJT-X instances *and*
multiple interoperating sever applications. To do this simply set the
UDP server address in all WSJT-X instances to a multicast group address
of suitable scope, e.g. 239.255.0.0 will work on the local sub-net which
includes multiple servers and clients on a single host. The service port
number choice for multicast is no different than for unicast and the
default 2237 is fine so long as no unicast servers within the scope are
listening on that port. With message_aggregator the multicast group
address is set in the UI, with udp_daemon it is a command line argument
(use the -h command line option for usage information).
With respect to Heartbeat messages, they are bidirectional but not
required in the server to client direction, currently the only action is
taken by WSJT-X on receipt of a Heartbeat message is initial schema
negotiation, other than that they only keeping the protocol alive by
informing that the server is still alive and kicking. Servers can choose
to send regular Heartbeat messages to each client they are aware of but
it is not a requirement, sending Heartbeat messages at 15 s intervals
would not be unreasonable if servers choose to do so. Note that traffic
from servers to clients is not multicast since all messages to clients
are specific to that client. Traffic sent from servers shall be sent to
the sender address and port pair gathered from a prior message receipt
from the client, this is normal IP messaging as used by any server
application serving content to a client. WSJT-X instances do not listen
on the multicast group address, that is the role of the servers.
With respect to the schema number, firstly the schema number does not
define the message types or fields, it is there to define the low level
serialization of field contents on the wire. The schema number will only
be incremented if we need to include a new field data type that is not
defined by the current schema and that in turn is defined by the Qt
serialization formats for the Qt types that we use on the wire. I should
add that more than one application developer has claimed that the use of
Qt serialization formats precludes them from consuming the WSJT-X UDP
datagrams, this is not correct as the serialization formats are
straightforward industry standard ones for compact binary data, i.e. we
have to choose some "on the wire" format and Qt is particularly
convenient and efficient but easily consumed by non-Qt applications.
With respect to schema negotiation, it is a requirement of the WSJT-X
UDP protocol that servers and clients written at different times
interoperate correctly. To do so any node whether it be client or server
can expect to only receive messages using the schemas that were known at
the time the application was written (or updated). The consequence of
this, particularly in a multicast grid like topology where there may be
multiple clients *and* multiple servers all communicating on the same
multicast group address and service port number, is that all nodes must
be prepared to degrade to an older schema number if an application joins
the party that knows nothing about the currently negotiated schema. This
negotiation is done when an application receives it's first Heartbeat
message. That Heartbeat message will contain a schema number (all
messages do), and a maximum schema number. The server shall not send any
messages with a schema number greater than the number received in the
first message from a client. The server may use a lower schema number if
it was not written at a time the received schema was specified, that
will negotiate the lower schema number with the client. Once a lower
schema number is negotiated that schema number will propagate across all
active nodes this allowing communications to continue without issue
other than certain messages using field types only defined in the
discarded schema will not be sent any more. This necessary degradation
of the UDP protocol messages is unavoidable so interoperating
application developers should make efforts to track schema updates with
new versions, note that these schema updates will be very rare.
73
Bill
G4WJS.
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
