Hi Glenn,
On May 16, 2012, at 1:49 AM, Glenn Pierce wrote:
> Hi. I am having a little trouble with a server / client design and
> wonder if anyone had any advice.
>
> I have a Thrift server that abstracts a data store. The idea is that
> there will be a number of clients that are essentially
> out of process plugins that use the interface provided by the server
> to receive, manipulate the underlying data store and also provide
> their own data.
> There will be a number of other clients which simply access the data
> provided by the server and its "plugins".
>
> The problem case is when one of these "plugins" wishes to provide its
> own data and provide an interface to that data.
> The server should have no knowledge of the plugins data or interface.
>
> I would ideally like all clients to access functionality through the
> main thrift server so it acts as a facade for the plugins. If a client
> requested some data provided by a plugin the main server could
> delegate to the plugin to provide that data. I guess this would mean
> have each plugin being a thrift client and server. I have written the
> server in python so could probably handle thrift calls that are not
> yet defined but would it be possible to forward these calls another
> thrift server IE act as a proxy ?
Assuming that you really can "handle thrift calls that are not yet defined"
(which you couldn't in C++, which is what I use), then having the main server
act as a proxy for "server" plugins is fine. We have a daemon process that
presents a Thrift service API to the local system on a Unix domain socket. The
calls that clients make are sometimes handled locally, and other times passed
on to our servers that present a Thrift service interface to the internet, and
other times are handled by a combination of local and remote processing.
Internally, the daemon also uses a third Thrift service that serves a local
database. IN fact, there's a fourth Thrift server buried inside the daemon too,
but all clients see on the local system is the one daemon API.
>
>
> An alternative is maybe have the plugins be clients only and push data
> to the server. But the format of these messages
> would unknown to the server and would have to be generic enough to
> accommodate different types of data. I not sure how I would provide a
> useful interface to this data to other clients.
>
> As far as I can see only the plugins knows how to store and manipulate
> the data it owns so this idea probably would not work.
The only way I know of (from direct experience) to make something like this
work would be if your real server had a RPC call that was defined something
like this (written off the top of my head - not compiled, etc.):
struct PassThroughCallParams
{
1: list<byte> RealRpcStruct,
}
struct PassThroughReturnParams
{
1: list<byte> RealRpcReturnStruct,
}
struct ClientPassThroughReturnParams
{
1: bool wasHandled,
2: PassThroughReturnParams actualReturnParams,
}
service MyService
{
PassThroughReturnParams passThroughCall (1: PassThroughCallParams),
}
Then your server could always field this call. It would need to know that each
of its clients has a server interface that defines a passThroughCall() that
takes the same calling structure and returns the ClientPassThroughReturnParams.
The server would call this on each client until the wasHandled struct member
came back true, or the call wasn't handled.
To pull this off, you need to have a standardized way of flattening Thrift
structures to buffers, with some sort of identification. We wrote a variant on
the TMemoryBuffer transport to do this. If you get a flattened Thrift structure
you can try to unflatten it into a struct type that you expect. The flattened
buffer has a checksum block and an ID in a header that catches when things
don't match up.
>
> Thanks for any advice. Any suggestions welcomed.
>
> --
> Regards
> Glenn P Pierce
Best regards,
Rush
