-----Original Message-----
From: Eric Rescorla [mailto:[email protected]]
Sent: Wednesday, July 08, 2009 10:43 AM
To: Narayanan, Vidya
Cc: [email protected]
Subject: Re: [P2PSIP] Client operation in RELOAD
At Wed, 1 Jul 2009 12:18:13 -0700,
Narayanan, Vidya wrote:
The client operation in RELOAD is currently underspecified and
cannot be implemented based on the spec.
I'm certainly prepared to believe that's correct, but I'm not really
getting that from your message. From my perspective, the major
feature of the RELOAD client design is that there *isn't* any
special support for clients, but that it just drops out of
the rest of the specified protocol.
Message changes necessary to support clients:
---------------------------------------------
We need to introduce a "Node Type" in the JOIN Request message that
indicates that the joining node wishes to do so as a client. Also,
Update messages need to have a "case clients" and the "case full"
needs to include client information.
In the current design, clients simply don't send Join or Update, so
this
isn't an issue. Because they don't really participate in either the
storage or routing fabric, as far as I can tell they don't need
to do either.
I suggest adding a section on clients - suggested text below:
Clients:
--------
Clients are nodes that may provide and receive services in an
overlay without participating in overlay routing or incurring the
same amount of maintenance overhead as the overlay peers. Clients
are modeled as close to peers as possible in terms of overlay
functionality. In order to allow these nodes to participate in the
overlay without inherently being a routing member of the overlay,
some changes are required to the join process and the manner in
which neighbor relationships are handled for such nodes. However,
all other procedures (e.g., data management using STORE, FETCH,
etc.) are handled in exactly the same way as for any other peer in
the overlay.
We denote the point of attachment of the Non-Routing Peer on to the
overlay as Overlay Attachment Point (OAP). The peer responsible for
the node ID of the client provides the OAP functionality. This
allows the integration of clients into the architecture with the
least deviation from the peer. Hence, any node that wishes to be a
client finds the node that owns its identity space and attaches to
it. By definition, any node serving as a peer in the overlay MUST
provide the OAP functionality as well. The mode of using a
different node as the OAP may be allowed as an extension to this
protocol - this can be done for availability or load balancing
reasons, for instance.
Clients obtain overlay node identities just like peers do. Hence,
the enrollment server may be completely unaware of the distinction
between a peer and a client. By virtue of this, the enrollment
procedure is identical for peers and clients. The join procedure is
somewhat simplified for a client and the section below describes
that process. All data management operations to/from a client are
indistinguishable from those of a peer.
The main difference between a client and a peer is that a client
only has a single neighbor in the overlay, i.e., the OAP. The
client routes all messages through the OAP - i.e., it has a default
route with the OAP as next hop that is used for all messages. The
OAP has the client as a neighbor and has it in its routing table.
None of the nodes in the overlay except the OAP are aware of the
presence of the client. The messages to/from a client are natively
routed on the overlay. All messages that are sent to the client
arrive at the OAP by regular overlay routing (since it owns the
identity space that includes that of the client). The OAP then
looks up its routing table to route the message to the client.
The OAP may make use of the presence of a client to offload some of
its storage responsibilities or to replicate data stored at it.
This may be mutually beneficial in some cases - e.g., the client may
be limited in terms of bandwidth and battery, while the OAP may be
limited in memory. However, this is out of scope for this document
itself and any such storage sharing that occurs between a client and
OAP is not visible to the rest of the overlay. Hence, any such
thing must be managed out of band between the OAP and the client.
I agree with almost all of this text, but it's not normative
and I'm not sure really clarifies things that much. As far as
clients storing data for OAPs goes, IMO that should be out
of scope of the protocol. If clients and OAPs want to design
some new protocol that handles that great, but it's not RELOADs
business.
Client Bootstrap Process:
-------------------------
A client obtains a list of bootstrap peers from the enrollment
server as any other node in the overlay. The client triggers the
join process the same way as a peer does. But, the process is a bit
more abridged than for a normal peer. It is described below:
1. Client connects to its chosen bootstrap node. The client MUST
form a TLS session with the bootstrap node, using the certificate or
PSK required for overlay membership. 2. The client sends a Join
request message to its own Node-ID c, through the BP. The Join
Request message indicates that the joining entity is a client (with
Node Type being set to Client). This is routed to the admitting
peer (AP). The AP sends a response to the Join, after noting that
it must serve as the OAP. 3. The client also sends an Attach
request message to c through the BP. This may be sent in parallel
with the Join message. The AP sends an Attach response, resulting
in connection establishment between the client and AP. At this
point, the AP has the client included in its routing and connection
tables and the client has established a default route through the
AP. The join process is now complete.
The OAP MUST NOT send any Update messages to other peers in the
overlay as a result of a client joining the overlay. The OAP simply
updates its routing and connection tables with the client
information. The client does the same. The OAP may, however,
follow up the join process with a series of STORE request messages
to transfer some data it stores to the client.
I disaree with this: the client should not be sending a Join because
that indicates the desire to be a peer. There is no need for the
join as mere attachment enables direct routing to the client
from the OAP.
Impact of Churn on Clients:
----------------------------
As the ownership of the overlay identity space changes, the clients
may be affected. For instance, if the OAP to which a client is
attached leaves the overlay or fails, the client needs to find the
appropriate OAP to attach to. Also, when the identity ownership
changes due to a new peer attaching between the OAP and the client
in the identity space, the client needs to use the new peer as its
OAP. Essentially, the need to change OAP may arise from one of the
following three conditions:
* A new peer joined the overlay with an identity that lies between
* the identities of a client and its OAP
* The OAP of a client performs a graceful leave from the overlay
* The OAP of a client abruptly leaves the overlay or fails
These three cases are explained below.
Handling New Peers:
-------------------
Consider the case where a client (C1) is attached to an OAP (OAP1).
A new peer, P, attaches to the overlay, with an identity that falls
between C1's and OAP1's identity. P is now responsible for the
identity that belongs to C1 and hence needs to serve as the OAP to
C1 (say, OAP2). When P completes its Join process, OAP1 detects
that it no longer owns the identity corresponding to C1. OAP1 now
sends an Update message to C1, indicating that it has a new
neighbor. The Update message provides information about P to C1.
Upon receiving the Update message, C1 sends a Join Request via OAP1
to P. The difference between this JOIN process and the regular one
is that C1 can perform this Join via OAP1 instead of finding a
bootstrap peer to perform the join through. Since P is part of the
overlay and is OAP1's neighbor, OAP1 can reach P directly, thereby
greatly reducing the number of hops the Join messages need to
traverse. Once the new Join process completes, P takes over as C1'
s OAP (OAP2). C1 may send a Leave Request to OAP1 subsequent to
its attachment to OAP2 - this will trigger cleaning up of state
corresponding to C1 at OAP1.
If OAP1 stored any of its data at C1, C1 must transfer that back to
OAP1 before joining again with OAP2. The mechanism to handle this
is out of scope of this specification.
When a change in the identity space ownership causes the OAP of a
particular node to change, the present OAP MUST send an Update
message to the client, indicating that it has a new neighbor. Upon
receiving the update message, the client MUST send a Join message to
the new OAP - this JOIN message can be routed through its current
OAP. The client SHOULD send a Leave request to its old OAP, once it
has successfully attached to the new OAP.
In general, whenever a peer's view of its neighbor changes it
should Update all its neighbors. As "client" isn't a first-class
term in RELOAD, this implies clients as well. I agree we should
explicitly state that clients need to disconnect and reconnect
to the new responsible peer. As indicated above, I believe the
discussion of Join is incorrect.
There is an issue that nodes may send attach messages to the
client's node id when looking for a finger. The client needs to
reject such Attach requests coming to it. However, the client will
be unable to distinguish Attach messages that are being sent to it
to really set up an end-to-end connection for other purposes. We
may want to consider introducing a "purpose" in the Attach request
messages that indicate the reason for the Attach. This will make it
possible for the client to reject a request that is sent to
establish a finger. The node sending the Attach request can then
slightly alter the resource id to issue another request.
There is supposed to be an AppAttach message that makes this
clear. It just somehow didn't make it into the draft. It will
be in the next revision.
-Ekr
