Xiao,
My first take was that update stream resources should be 1-1 with the
underlying full map resources. Then I realized that cost maps depend on
network maps, and clients & servers should coordinate changes to the two.
The easiest way to accomplish that was to use the same stream for both
network map and cost map updates, with the event type distinguishing network
map updates from cost map updates.
But your comment reminded me that network maps are 1-n with cost maps. Eg, a
network map can have two cost maps, one for routingcost and the other for
hopcount.
So I suggest that an update stream resource can provide updates for an
arbitrary set of resources. The server picks the set. Each update event has
a media-type (basically, full-map vs merge-patch), and the resource-id of
the map it updates. Because SSE only allows three fields (event, id and
data), I suggest we encode the media-type,resource-id pair in the event
type, as a JSON object with those field names, as in:
event:
{"media-type":"application/alto-network-map+json","resource-id":"network-map
"}
Here¹s a detailed example of an IRD with an update stream service that gives
updates for the network map and the associated routingcost and hopcount
maps. It offers merge-patch incremental updates for the cost maps, but only
full updates for network maps:
"network-map": {...},
"routingcost-map": {...},
"hopcount-map": {...},
"network-map-updates": {
"uri": "http://somewhere.com/network-map-updates";,
"media-type": "text/event-stream",
"uses": ["network-map", "routingcost-map", "hopcount-map"],
"accepts": "application/alto-updatestreamfilter+json",
"capabilities": {
"events": [
{"media-type": "application/alto-networkmap+json",
"resource-id": "network-map"},
{"media-type": "application/alto-costmap+json",
"resource-id": "routingcost-map"},
{"media-type": "application/merge-patch+json",
"resource-id": "routingcost-map"},
{"media-type": "application/alto-costmap+json",
"resource-id": "hopcount-map"},
{"media-type": "application/merge-patch+json",
"resource-id": "hopcount-map"}
]
}
}
This is a POST request. The client sends
application/alto-updatestreamfilter+json data (a new MIME type) to select
the events it wishes to get. Here is how a client would request updates for
the network and routingcost maps, but not the hopcount map:
POST /network-map-updates
Host: somewhere.com
Content-Length: ###
Content-Type: application/alto-updatestreamfilter+json
Accept: test/event-stream,application/alto-error+json
{"events": [
{"media-type": "application/alto-networkmap+json",
"resource-id": "network-map"},
{"media-type": "application/alto-costmap+json",
"resource-id": "routingcost-map"},
{"media-type": "application/merge-patch+json",
"resource-id": "routingcost-map"}
]}
The server would respond with something like this.
HTTP/1.1 200 OK
Content-Type: text/event-stream
: Full network map, sent immediately.
event:
{"media-type":"application/alto-network-map+json","resource-id":"network-map
"}
data: { ... full network map ... }
: Full routingcost map, sent immediately.
event:
{"media-type":"application/alto-costmap+json","resource-id":"routingcost-map
"}
data: { ... full routingcost cost map ... }
: Incremental routingcost map update, sent when enough costs have changed.
event:
{"media-type":"application/merge-patch+json","resource-id":"routingcost-map"
}
data: { ... merge-patch for routingcost cost map ... }
Note that this omits the ids. They aren't really necessary. But if the
underlying SSE library expects them, the server can always use sequence
numbers 1, 2, 3, etc.
How do people feel about this approach?
- Wendy Roome
From: Xiao SHI <[email protected]>
Date: Fri, October 17, 2014 at 10:10
To: Wendy Roome <[email protected]>
Cc: Xiao SHI <[email protected]>, "Y. Richard Yang" <[email protected]>, IETF
ALTO <[email protected]>
Subject: Re: [alto] Incremental updates transport using SSE?
Hi Wendy,
I completely agree with you. I was thinking maybe there's some clever way of
using the SSE event-id, but the questionable saving is simply not worth the
logging and synchronization trouble.
1. One thing that follows this choice is to use one single SSE stream (open
connection) for each resource. I briefly considered having one open
connection and send updates for all resources on the server via that stream.
The advantage of having only one connection open is smaller overhead, and
the disadvantage is that it would require a more complicated ID and version
tagging in event name/id maybe. What do you think?
2. If the mapping between SSE stream and underlying map is one-to-one, would
it make sense for the resourceID of a stream to be the resourceID of the
underlying map and ".update" concatenated together? This way, when the
client actually requests and initiates the stream, the client would be aware
of the resourceID of the underlying map.
3. Following that, we should define id field in the events. Richard proposed
using a tag as the id if it's a full network map, and newtag_oldtag if it's
a patch. The vtag for network map is ResourceID+tag tuple, if the client
knows about the resourceID, could we only use the tag? And should the cost
maps simply use dependent-vtags? Since the id field is a string, what format
do we want the tags in? A few options are 1) json with all whitespaces
removed; 2) an even more compact patterned string.
e.g. for network map patch, it could be:
{"old-tag":{"resource-id":"my-default-network-map","tag":"3ee2cb7e8d63d9fab7
1b9b34cbf764436315542e"},"new-tag":{"resource-id":"my-default-network-map","
tag":"c0ce023b8678a7b9ec00324673b98e54656d1f6d"}}
or (since it must be the same resource):
{"resource-id":"my-default-network-map","old-tag":"3ee2cb7e8d63d9fab71b9b34c
bf764436315542e","new-tag":"c0ce023b8678a7b9ec00324673b98e54656d1f6d"}
or (if the client knows the resource-id for the underlying map of the
stream, just use a delimiter):
3ee2cb7e8d63d9fab71b9b34cbf764436315542e_c0ce023b8678a7b9ec00324673b98e54656
d1f6d
What do you think?
Best,
Xiao
On Fri, Oct 17, 2014 at 9:21 AM, Wendy Roome <[email protected]>
wrote:
> Xiao,
>
> That is a very good point. I hoped someone would ask about that.
>
> It is a tradeoff. The disadvantage of requiring the server to send full map(s)
> at the start of each stream is that if the connection drops, we have to send
> the full map over again. But the advantage is that incremental updates are
> only within the context of this particular stream. We don't need keep track of
> where we were.
>
> To see the issue, suppose we wanted to avoid re-sending the full map when the
> client reestablishes the connection. Then the server must assign a unique id
> to every incremental update event. When the client reconnects, the client
> gives the server the last known event id, and the server sends any events the
> client missed.
>
> BTW, SSE does define event ids. Each event has an id, and when connecting, the
> client gives a Last-Event-Id header with the last id it received (if any).
>
> So what is the problem with that? First, the server MUST assign a unique id to
> every incremental update, and they must be the same for all clients. That is,
> if the server has several increment-update clients, the server must "clock"
> their update event streams at the same rate. Second, the server must keep a
> buffer of old update events, in case a client needs to reconnect. And third,
> because a reconnecting client may send an old (or bogus) Last-Event-Id, the
> server may have send the full map(s) anyway.
>
> If we do not use event ids, then it is much simpler. Incremental updates are
> in the context of this stream. Period. No persistent state that must survive
> the end of the connection. And if desired, the server can send updates are
> different rates to different clients.
>
> So I think it boils down to how often we expect connections to drop
> unintentionally. If that is common, then, yes, we need to define event ids to
> avoid resending the full maps. But if we can assume accidental drops are rare,
> then I think it is simpler just to send the full map(s) for every new
> connection.
>
> - Wendy Roome
>
> From: Xiao SHI <[email protected]>
> Date: Thu, October 16, 2014 at 18:22
> To: Wendy Roome <[email protected]>
> Cc: "Y. Richard Yang" <[email protected]>, IETF ALTO <[email protected]>
> Subject: Re: [alto] Incremental updates transport using SSE?
>
> Hi Wendy,
>
> If a client accidentally dropped the connection and hopes to re-connect and
> receive the merge-patch events, would it make sense for the server not to send
> the first full-map event? It will save quite some effort that way.
>
> Best,
> Xiao
>
_______________________________________________
alto mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/alto
