On Tue, Dec 29, 2020 at 5:00 AM Tristan Van Berkom < [email protected]> wrote:
> Hi Justin, > > On Mon, 2020-12-28 at 14:29 -0500, Justin Erenkrantz wrote: > > On Tue, Oct 27, 2020 at 3:28 PM Chandan Singh <[email protected]> > > wrote: > [...] > > Thanks. I'm picking this up again as there's an upstream change in > OpenSSL > > which is apparently breaking serf. For reference, my serf-bst 2.0 > project > > is at [1]. > > > > Buildstream is *so* close to doing what I want here: control my version > of > > OpenSSL and Serf in a clean manner and do so in an isolated manner from > my > > host environment. > > I took a look at your linked serf-bst project, and I feel like I am > missing some context to be properly equipped to help more. > > What I am seeing is a docker used to create an input runtime upon which > you are building some software, but I don't understand the context > under which this is intended to be deployed or run. > The intention is to create the binary artifacts associated with the dependency tree in a controlled manner. Serf itself is just an HTTP library intended to be consumed by other projects (such as Subversion, OpenOffice, etc, etc). While there isn't a lot of activity in the project itself, there has been recent churn in OpenSSL which silently breaks their API contracts. So, having an integration build be able to easily swap out versions of OpenSSL is very very useful. > However, I need the debugger available to step through the test binaries > to > > figure out what's going on. It also seems like the bst shell doesn't > have > > network access - I can't run any of the produced binaries to remote > hosts. > > Is there a step I'm missing to allow network access? And, is the only > way > > to bring in the debugger is to build it into the base platform (or have a > > special layer with gdb and such)? > > Regarding network access in `bst shell` > --------------------------------------- > It appears that network access should be enabled by default, and > specifying the `--isolate` flag will ensure that there is no network > access. > > That said, it is usually not enough to merely allow networking in order > to have a networking enabled shell, to get networking to actually work, > you will probably at least need to mount your host's /etc/resolv.conf > into the shell environment. > > You can try this with: > > bst shell --mount /etc/resolv.conf /etc/resolv.conf serf.bst > > It is recommended to setup your preferred shell configuration in your > project.conf to make things easier to run the sorts of software you > build in a shell for testing/debugging: > > See the examples below for the `host-files` configuration: > > https://docs.buildstream.build/master/format_project.html#customizing-the-shell > > Here is a more elaborate example of a shell configuration which allows > the user to launch graphics applications on (most) linux variants which > use X (or XWayland) and pulseaudio: > > > https://gitlab.gnome.org/GNOME/gnome-build-meta/-/blob/master/project.conf#L165 Yup, it was the lack of resolv.conf - thanks! I'll incorporate some of these tricks into serf-bst's project.conf. Does it make sense to have slightly saner defaults in buildstream? Regarding debugging tool presence in `bst shell` > ------------------------------------------------ > This has been the subject of much debate a couple of years ago. > > In short: Yes, the only reliable way is to build your tools or wrap up > some binaries known to work in that project's specific runtime > environment and put them there, that means using elements. > > That however only answers half of the question when it comes to > practical use of debuggers in `bst shell`, the other half is that a > project does not truly depend on debuggers and profilers and such > tooling. > > As far as I can see the conclusion of these debates is issue #422: > > https://gitlab.com/BuildStream/buildstream/-/issues/422 > > This states that basically one should be allowed to stage multiple > elements into the same `bst shell` environment, possibly resulting in > issuing some file overlap warnings. > > This should not be all that difficult to implement, and implementing > this would mean that one could have a defined set of debugging tools > which one could optionally stage in `bst shell`. > Well, the issue is that there is a vicious cycle between gdb's source dependencies. I tried creating elements for gdb - but, it requires makeinfo to build due to this issue: https://sourceware.org/bugzilla/show_bug.cgi?id=14678 Texinfo requires gettext which requires wget...which requires makeinfo to build. So, I can't technically define an upstream element for gdb from source. So, it must be injected outside Buildstream (which we have via our platform Docker image)...unless there's another trick to break these dependencies somehow? Related to the above, is the still unresolved issue #413: > > https://gitlab.com/BuildStream/buildstream/-/issues/413 > > This pertains to adding build directories to sandbox for `bst shell`, > at the same locations where they were originally built, so that > debuggers can easily find the sources for the binaries they were built > for. > Also, it doesn't look like we're keeping the debug symbols around - so, even if I inject gdb into the platform, I don't have the debug symbols. I'll hunt around to figure out if there's an obvious approach. So, I'm not even getting to the point of having the symbols linking to the (non-existent) source files... > Of course, when we start talking about graphical debuggers and IDEs > then this becomes a much more complex discussion. > One point Sander mentioned is that some of the IDEs support remote symbol serving - so, perhaps something we can explore that might ease the barrier to entry. Cheers. -- justin
