Bug#976462: tech-ctte: Should dbgsym files be compressed via objcopy --compress-debug-section or not?
On 3/8/21 6:27 PM, Sam Hartman wrote: >>>>>> "Matthias" == Matthias Klose writes: > > Matthias> Maybe you should be more specific about "those who can't > Matthias> use" uncompressed debug info in the first place. > > So, you've argued that the disk savings are not significant inside a > package, because packages are themselves compressed. > > What people are arguing is that they want to have debug info for large > programs like firefox or chromium installed, or really debug info for > large parts of their system. > They are in effect arguing that they care about the installed size not > the package size. > They have explicitly argued that having to uninstall and then later > reinstall disadvantages their debug cycle. > > This situation is particularly unfortunate because it sounds like we > have a conflict between two techniques for saving space. > > On one hand we have dwz which tries to optimize and reduce overall size > of debug symbols > > which is incompatible (apparently--no one has explicitly confirmed this) > compressed debug symbols. > > Presumably we can still run dwz within a single package by doing so > before debug symbols are compressed. > But presumably this gets in the way of people running dwz themselves or > something. > > I'll be blunt. > The people who say that they want debug symbols installed on their > system have made a simple, easy to understand argument. let my be blunt as well. The only reference I can find regarding the size on disk is #922744. Contrary to what you're saying "that they want to have debug info for large programs like firefox *or* chromium installed", they want to have debug symbols for firefox *and* chromium *and* more installed at the same time. #631985 speaks about a 10G space requirement for debugging KDE alone. The decision about the compressed debug symbols was made ten years ago. Maybe it's time to re-evaluate what expectations for a debug installation should be set. > The argument that compressed debug symbols break things is still porrly > stated. > We've had a claim that dwz might not work with compressed debug symbols > (and didn't used to). > We've had no one explain how that creates a problem in practice or even > confirm it's still the case. > It felt like pulling teeth to even get an answer that might be a tool we > care about. #87 asked for "postprocessing in dh_strip", however it was implemented in * dh_dwz: Add new experimental tool to run dwz(1) to deduplicate ELF debugging symbols. It should be generally be run before dh_strip (as dh_strip compresses the debug symbols and dwz expects uncompressed debug symbols). (Closes: #87) as pre-processing. So we know since about three years that dwz doesn't support compressed debug symbols. Your language about "claims", "might", and so on is not appropriate. Upstreams are currently looking at issues seen with valgrind about .gnu_debuglink section and .gnu_debugaltlink section in https://bugs.kde.org/show_bug.cgi?id=427969 https://bugs.kde.org/show_bug.cgi?id=396656 so apparently there are issues with another tool (valgrind), and how the debug information is created and split in debhelper. Also see https://github.com/rpm-software-management/rpm/blob/master/scripts/find-debuginfo.sh how dwz is run *after* separating the debug info, not touching the stripped binaries. Apparently the choice for compressed debug sections resulted later in an implementation for dh_dwz which is causing issues on it's own. Unrelated to that, but to not create conflicting dbg and dbgsym packages, there is #968710 and #981245, and it will be difficult to integrate within debhelper without introducing a new debhelper compat level. Also unrelated, there are #971724, #971680, and packages manually installing additional files in auto-generated dbgsym packages. Maybe any of these decisions to dh_strip were maybe mad to the best knowledge at the time, but the current situation is a mess. Sticking to compressed debug sections is just one issue ... Matthias
Bug#976462: tech-ctte: Should dbgsym files be compressed via objcopy --compress-debug-section or not?
On 3/7/21 11:51 PM, Sean Whitton wrote: > Hello, > > On Sun 07 Mar 2021 at 03:50PM -07, Sean Whitton wrote: > >> This is not much good if your network is weak or you're offline, or you >> don't want information on your debugging to go out to a web service. > > What I mean is: debuginfod is great in many scenarios, but we should > probably care about those who can't or won't use it too. Sorry if the > above is a bit blunt. your comment is unfocused. This was just another use case where uncompressed debug info could be harmful, and I pointed out a configuration how to avoid it. Maybe you should be more specific about "those who can't use" uncompressed debug info in the first place. Matthias
Bug#976462: tech-ctte: Should dbgsym files be compressed via objcopy --compress-debug-section or not?
yes, the rationale for uncompressed debug sections is that any tool can access them. On disk as deb/dbgsym package, there is no big difference in size. Also a debuginfod server can be configured to send the debuginfo compressed on the fly. The "only" downside is to have the uncomressed debuginfo on the disk when doing the debugging. Matthias
Bug#766708: counterfeiting the summary of the bootstrap sprint
So in the last email Wookey enumerates a lot of things what he did during the last months. Maybe he should have mentioned his ballerina lessons used for his performances during the DebConf talks too. However ever all of these have in common, that this has nothing to do at all with the work he committed to do. Further he cites a paragraph from the debian-bootstrap sprint summary, which reads: The report from that meeting https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results says: -- 3.13. Multiarch cross-toolchains vs single-arch cross-toolchains This contentious issue was discussed, and is partly covered under other headings. Wookey prefers the multiarch builds, Doko prefers the single-arch bootstrap builds. We agreed that either provides useful cross-toolchains. It's not clear whether it's easier to fix the Ubuntu cross-toolchain-base packages to do a bootstrap build in Debian, or to fix the blockers for multiarch builds in the archive. Whichever is working first should get uploaded. According to https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results?action=diffrev1=30rev2=31 the last sentence was added on Aug 29 during DebConf, long after the sprint happened, with a commment must be almost the final review, without mentioning anything. I call this counterfeiting the summary of the sprint. I assume this helped to convince other people to sneak in these packages into Debian. What is this if not bad faith? Again, the rest of the email talking about willing to work together doesn't match the his actions at all. Matthias -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5480b905.2010...@debian.org
Bug#771070: requirements for cross toolchain packages in the distribution
Package: tech-ctte Recently some crippled cross toolchain packages were uploaded to Debian unstable, without having any consent with the Debian GCC maintainers, nor announcing these packages anywhere. Maintained by a single person, not even trying to form a maintainer group, and downgrading bug reports without addressing concerns of the GCC maintainers. I'm asking the ctte to state that - a cross compiler based on GCC within Debian is able to cross build the existing gcc-x.y packages without disabling any features that are offered by the packaging, and used for the native compilers. This includes frontends and multilib'ed runtime libraries. - a cross compiler based on GCC within Debian provides all the front ends required to cross-build GCC. - a cross compiler based on GCC within Debian uses the very same sources, patches and configuration, so that it is equivalent to the native compiler provided by the Debian GCC maintainers. The current maintainer of the cross compiler packages has shown that he is not able to work together on these issues in a group. I'm asking the ctte that anybody who is able and willing to maintain the cross compiler packages as outlined above, becomes the new maintainer of the cross compiler packages targeting Debian architectures. It would be good if the GCC maintainer group is involved in this group. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5475e14d.2010...@debian.org
Bug#766708: supported GCC based cross compilers in Debian
While the rebootstrap project is a laudable goal, it is not part of the Debian archive, and I doubt that I'm bound by any decision for something not being part of the Debian archive. Nevertheless I'm outlining below how to proceed with the rebootstrap. Contrary to some claims in this thread there never was any commitment by myself to support the ability to rely on dependencies on foreign architectures within the Debian archive. The person claiming this is plain wrong. Please use the standard way for cross building packages for the rebootstrap project. The rebootstrap project uses local patches anyway, so in the short term it shouldn't be a problem to keep track of the packaging. In the long term, please use the supported way to do staged builds and cross builds of the GCC packages. Some background and history. The cross compiler support in the GCC packaging dates back to 2000, but was largely unused in the Debian archive, the biggest users were outside of Debian within the Emdebian project. Prominent contributors for the cross build support were Arthur Loiret (2007-2009, former Debian developer), Al Viro (2010, external), Marcin Juszkiewicz (2010, 2011, external), and myself doing integration, testing, and regular updates. There were probably many more people involved, but not those contributing to this thread. With Marcin's patches it finally was possible to upload the cross compiler packages to a distribution (Ubuntu). All packaging changes for binutils and GCC were available at the same time in the Debian packaging. This is the supported way to build a cross compiler in Debian targeting a Debian architecture, and might be different from Emdebian. During a SoC project, support was added to target the runtime libraries of a foreign architecture. This project was never communicated to the GCC maintainers before it started. After the project was accepted, I felt obliged to merge these patches for a successful project, however this maybe was a mistake. There was absolutely no support by the mentor of this student to get these patches reviewed and merged, these patches were neither tested for the supported cross builds, nor for the native builds, and broke either one of those or both. A considerate amount of my time did go into testing, fixing issues, and mentoring this student. The mentor was no help, neither understanding the existing binutils and GCC packaging, nor willing to understand it, and still claiming that he is able to simplify it. This continues and the claims made in this issue that pending patches were tested using the supported standalone builds are plain wrong, and I don't trust should work claims anymore made in new patch submissions for the unsupported cross build only. There is nothing wrong with the supported standalone GCC cross build, this configuration is able to do anything what the unsupported configuration can do, however the latter is limited to Debian release architectures, doesn't work for new architectures and doesn't work with partial architectures. There is a plethora of cross build support scripts and designs, however there are not many people working on these within Debian. The choice for one method which is fit for all use cases in Debian is the most maintainable one. The past years did prove that. Pet projects should be maintained outside of Debian. Reading the MultiArchCrossToolchain wiki page, one might get the impression that the supported cross toolchain is not able to make use of foreign architectures. Whether this is impression is wanted or not, it is wrong. It is supported. This method was used by myself to bootstrap two new architectures without having access to some prebuilt binaries like it was the case for the Debian arm64 and ppc64el bootstraps. It works. This shouldn't be news to the contributors within this thread. I mentioned this at least at the two last DebConf's, plus at Linaro Connect's, and during the bootstrap sprint in July (Paris), but apparently there seems to be temporary or permanent distortion of perception. Earlier this year, and last time at the bootstrap sprint in Paris, Wookey committed to work on the cross-toolchain-base package (this package isolates the bootstrap process and builds binary cross glibc packages). I hope other participants of the bootstrap sprint can confirm that this commitment was made. Then nothing happened. I can't remember that he said anything about a change of mind during DebConf. The big surprise then came when Wookey was uploading initially six cross toolchain packages to unstable without saying anything, without having worked on the cross-toolchain-base packages at any time. You may call this politics, or tactics, however I call this sneaking in these packages, and his communication behaviour as plain XXing (somebody mentioned I should just call this not telling the truth). At this point I decided to remove the unsupported cross build support, and then was accused on irc by
Bug#766708: set correct title
Control: retitle -1 Request to override gcc maintainer changes breaking unsupported way of cross-building cross builds supporting multiarch exists and works. claiming otherwise is just populistic and/or marketing. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/544ed6d6.30...@debian.org
Bug#552688: Please decide how Debian should enable hardening build flags
On 07/27/2011 11:56 PM, Raphael Hertzog wrote: Hi, On Tue, 26 Jul 2011, Raphael Hertzog wrote: Assuming that all those improvements are done, the consensus was that it's fine for dpkg-buildflags to start emitting the hardening build flags by default. According to Ubuntu's experience only a few dozen of packages are broken by the presence of these flags and those packages should just be updated to use the new STRIP operation to drop the for strip you need to know the exact options to be passed; wouldn't it be better to have them stripped by something like `nohardening'? In the course of doing this I discovered that this won't have the expected result: --- export DEB_CFLAGS_MAINT_APPEND = -Wall [...] ./configure $(shell dpkg-buildflags --export=configure) --- Apparently make doesn't export the variables to the sub-shell run in this way but only to shells run for commands in the various targets. So instead I have to do it this way: ./configure $(shell DEB_CFLAGS_MAINT_APPEND=-Wall dpkg-buildflags --export=configure) One thing that is really not clear to me is how we should handle PIE. It's not enabled by default by the gcc patch. This means that it's not safe to enable it by default in dpkg-buildflags because we have no idea of its impact. While all the other options have been well tested thanks to Ubuntu, this one was not. Yet it seems that we should still aim to use it by default at some point. How should we handle that transition? I did see performance penalties of more than 20% on i386 and armel when enabling PIE by default. This shouldn't be the scope of this discussion, and I still don't see value to rebuild the whole archive using PIE. The current implementation in my branch is that PIE is disabled by defaut but if you set DEB_BUILD_HARDENING_PIE=1 then it will be used. This was easily done on top of the compatibility layer with hardening-includes/hardening-wrapper but I'm not convinced it's an interface we want to use for this transition. In that case, it means that we should rebuild the archive with PIE enabled, see what breaks, report bugs and ask people to add DEB_CFLAGS_MAINT_STRIP=-fPIE DEB_LDFLAGS_MAINT_STRIP=-fPIE -pie where required. Once most packages have been fixed, we can add PIE to the default flags. Does this sound reasonable? No, there's no value having cc1 built with PIE, same for number crunching libraries, doubtful for interpreters. Matthias -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e31946f.9000...@debian.org
Bug#552688: [hert...@debian.org: Bug#552688: Please decide how Debian should enable hardening build flags]
On 07/27/2011 04:09 PM, Kees Cook wrote: - there needs to be a way to identify those architectures that are register starved, since those should _not_ get the PIE flags by default (e.g. i386 should not get PIE, but amd64 should get PIE by default). Right now if one uses hardening-wrapper, it's expected that everything that can be enabled is enabled, so you gain PIE even on i386 at the moment. please communicate the trade off even for amd64. It's measurable, and I don't see any value in slowing down cc1* for this. Matthias -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e3195ce.80...@debian.org
Bug#552688: Please decide how Debian should enable hardening build flags
On 21.11.2010 08:39, Raphael Hertzog wrote: CCing Kees Cook, he has been the one leading the efforts up to now. I hope he can answer your queries. Hi, On Sat, 20 Nov 2010, Don Armstrong wrote: There are a couple of things here that should be worked out first before the CTTE can make a decision: I assume that there is a decision to turn on hardening defaults? Who made it, and which defaults to turn on? Which ports should it use? Where is it documented? So involvement of the ctte seems to be a bit premature, asking the *how* before the *if*. As said before in the report, it should be reassigned to `general'. 1) Has gcc's upstream been approached about including this patch? What was their response? No idea. Afaik, not submitted, and not upstreamable in this form. 2) Has the archive been successfully rebuilt with the proposed patch? I think this patch is used in Ubuntu, so mostly yes. I guess Kees Cook or Steve Langasek should be able to tell us a bit more. 3) Since Matthias has indicated that he doesn't have the resources to steward this patch in Debian, who is going to work on maintaining it if upstream isn't interested in the patch and the CTTE decides to override Matthias? Kees, would you be willing to take this responsibility in that case? The patch itself is maintained, however it requires patches to the testsuite which are not maintained. They are in 4.4, partially forwarded, incomplete for 4.5 and not done at all for trunk. So I do have an answer about the responsibility (and no, you won't convince me otherwise in a few weeks or months having seen this for years). An additional effort is testing with upstream builds for submitting bug reports. I didn't see anybody to step up with this testing, when the toolchain diverges much more, compared to upstream. Alternatives to patching gcc include making dpkg-buildflags more prevalent, a wrapper that we require to install on buildds (coupled with throwing away binary builds), or some combination of the above. yes, I consider the current solution a hack, introduced in some derivates by the lack of resources to get it done properly as nearly any other distribution is doing it. Changes to the build flags should be injected in the package build system, not by changing the compiler itself. I first submitted a patch to introduce default flags from the environment, this was replaced/refined by dpkg-buildflags. Now please work on getting it honored in the package builds and maybe make it a policy for packages with a certain priority. Matthias -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ce8d697.2060...@debian.org
