Mark Slee wrote:
Yeah, I would also be against making SCons our primary build system. I've also
had some very painful experiences with doing anything beyond the very basics
with SCons that have cost me many hours.
My main motivation for working on the linkage, in the beginning, was
to get my head into the Thrift code. I've since started hacking on it a
little anyway, read through most of the C++ library, and got a general
picture of how it could replace XORP's XRL.
As part of that I've submitted a very simple patch for JIRA 570
(dupe methods):-
http://issues.apache.org/jira/browse/THRIFT-570
There are a number of changes we'd need to make to support fully
async client invocation, possibly building on Rush Manbert's work in the
process.
I quickly found that hacking SCons isn't needed to get up and
running with those changes first of all, so I've backed off on that for now.
However, I may need to revisit later. It's likely that I can do what
I need to do without modifying the Thrift C++ libraries as they ship in
Thrift itself.
So, I've crafted a few patches to t_cpp_generator.cc to support
doing a few things now. Any changes I've made default to off for normal
'thrift' invocation, so that it doesn't violate POLA for other users:
* generate *only* server-side or client-side stubs, allowing separate
output directories;
* split the server-side stub generation into a separate set of files;
* rename the suffixes of the generated C++ files.
One of these experiences, in fact, was trying to get SCons to handle generated thrift files properly -- it was extraordinarily difficult, because the dependency system employed by SCons is entirely based upon filenames.
We found this was an issue even with the XORP XRL generation, which
is very similar to Thrift in intent, until we wrapped the generators in
appropriate SCons 'Builder' targets. Same issue with all 'Jam' tools
too, and other build languages. SCons being implemented in Python just
made it much easier.
[AFAIK the Boost guys are still kind of struggling with
Boost.BuildV2; they have the problem that it's its own darned thing, and
nobody but the authors understand it. FWIW it looks like Thrift isn't
using much of Boost beyond shared_ptr at the moment, which is a header
template w/o specialization in a library, so linkage isn't an issue here.]
We end up having to ensure the generated file always appears in a
known location. The wrappers then know enough for dependencies to be
instantiated in the build graph OK.
The file suffix is one point, as suffix rules can be used to build
stubs for the Make targets, and then set up dependencies on the
'*.thrift' service definitions themselves -- independent of the build
language.
In our tree it would be nice, to keep the file extensions
consistent, to allow for tricks like this, so I've been hacking on
changing the '.cpp' / '.h' in t_cpp_generator.cc for each service
invocation.
The other thing is: output directories. What we'd like to do in our
tree, is to generate the client and server side stubs separately, so we
can keep the server-side code in the subdirectories for the processes
themselves.
Multiple service support is also a bit of an issue for us just now,
as the existing XRL code expects to be able to use the same endpoint to
talk to multiple services in the same process.
Implementing an adaptor like TFramedTransport looks like the way to
go here, however, it would require a TServer instance which is aware of
the service name in the framing, and which can dispatch using a map of
TProcessors.
Because of this, it's much easier for us to have those half of the
stubs in another set of files. At the moment the server-side glue goes
into <service>.h.
With my patch, the 'split_processor' argument in generator_strings
puts it into <service>_proc.h/cpp.
Of course the server-side stubs partly depend on what goes into the
client stubs; so we may need to be a little smarter about how we split
the code up for linkage in our own processes.
Would also note that SCons has very limited/poor support for lots of the
languages that Thrift employs, and as David mentions, extending SCons requires
significant time and knowledge.
You're quite right that it may not be the right answer for Thrift as
a project just now. It seems to work for XORP quite well, as it is
almost completely C++, and there are cross directory shared library
dependencies which Automake did not cope with very well.
Rush's patches change a lot of stuff, and I guess I just want to be
sure that everything's tight, particularly in a dynamic executable.
Having said that, it's looking like going to ASIO right away is too
expensive a change to make for XORP, and a real deployment of Thrift in
that project probably means writing our own TServer class to integrate
with the existing EventLoop.
cheers,
BMS
P.S. Now that I've got my head around the C++ library, building an AMQP
adaptor doesn't look so bad. Esteve has blazed the trail in a
framework/language which actually has an I/O model...
