Bug#682010: #682010 re celt and mumble referred to the TC
Hi Ian, Being cc'd on this would have been nice, so apologies in advance if I messed up any quoting by pasting stuff in. On Thu, Jul 19, 2012 at 12:26:01AM +0100, Ian Jackson wrote: Note: this evening we think we have found a security expert who is willing to audit the CELT 0.7.1 codec for issues and possibly provide patches, assuming this is reasonably feasible. This sounds like a good option to me. I will write to the security team and ask them for their opinion about CELT. From what you say I think: - We should try to address the security problems, if any, in the celt 0.7.1 codec. An audit would be very good. You understand this is a fairly arbitrary 'daily' snapshot of an experimental research codec, from ~2.5 years ago, that nobody has looked at since that day, that nobody has committed to maintaining, that its author has explicitly said he has no interest in maintaining, that has had large parts completely rewritten since for all manner of reasons, and is bitstream incompatible with every other snapshot that was ever released, right? And that its successor code has been reviewed in detail by some of the brightest minds of the IETF, prior to accepting that code as being the normative part of a proposed standard (which has now occurred) - and without exception, all of them said this is way too deep for me, I'm going to have to take the WG report on faith that it's sufficiently debugged now ... If skilled people have time for auditing, I'd love to see the code from the RFC pass under their eyes as a better, more enduring, use of their time. But this isn't something people are likely to find high school programming errors in from a quick read through or automated scan, or are likely to shake things out of just with simple fuzz testing. On the contrary, I fully expect mumble to have also EOL'd this long before anyone else, except perhaps the most dedicated blackhats, have understood even half of what goes on in this code, or the astronomical number of state transitions that are possible within it. And they have a 2.5 year head start on anyone who might try starting from today. - This is a serious problem for mumble at least and is arguably RC. Yes, mumble has a serious problem, that is arguably RC. In fact it has several of them aside from this corner that people have painted themselves into with it ... - Its primary author, who is also a DD and comaint of the package, is currently MIA with a new job that has consumed pretty much every minute of his time for the last 2 years or so now. The people who remain that are hacking on it, can't even do a formal new release at present, because he is the only one with access to the systems they need to do that, and is not available to do so. They are all busy too, so there is very much a culture of close your eyes and pretend it's all ok there at present. - It has other deps in the distro aside from this one that appear to be near enough to completely unmaintained too. Have a look at the code for zeroc-ice, and the ABI breaking patch that was blindly applied for gcc 4.7 and then left unfixed until Svedrin and I got stuck with the job of unscrewing that, and share in our tears ... - It has a small subset of users who would rather risk everyone's systems, than simply update their most ancient servers and tell their friends that it's time for a security update too. Which is all it takes to 'fix' this. FWIW, even the original poster of the bug in question later agreed in calmer discussion on IRC that the Right Thing for mumble to do is to release 1.3 and accelerate the migration, and that is only being delayed now by the first point above. This particular problem being raised here was entirely avoidable. Only a lack of foresight that the primary author of this software was going to be taken by the rapture, and a subsequent failure to plan for that loss by migrating to options that actually have maintainers ahead of the crunch date, have left us in the situation we are in today. These aren't things I usually associate with the idea of something being viable stable release candidate software ... If I'd known that Thorvald was not going to be here to manage this transition for Wheezy, I'd have never agreed to shipping libcelt in the Squeeze release either, and would have instead kept it in sid only. If I'd known that his plan to have all other distros ship the 0.7.1 release as a temporary interoperability snapshot would fail as dismally as it did (no other distro shipped this version except debian derivatives who took it from us), I'd have similarly vetoed the idea of shipping this as a public library in the last stable release too. Mumble already ships this as an embedded private library on every system other than direct Debian derivatives. I assume it would be possible to reintroduce a celt package which was very similar to the one recently
Bug#682010: #682010 re celt and mumble referred to the TC
Ron writes (Re: #682010 re celt and mumble referred to the TC): You understand this is a fairly arbitrary 'daily' snapshot of an experimental research codec, from ~2.5 years ago, that nobody has looked at since that day, that nobody has committed to maintaining, that its author has explicitly said he has no interest in maintaining, that has had large parts completely rewritten since for all manner of reasons, and is bitstream incompatible with every other snapshot that was ever released, right? However, it appears that some other distros have chosen to ship this particular version. For example at least Ubuntu are still distributing it. - This is a serious problem for mumble at least and is arguably RC. Yes, mumble has a serious problem, that is arguably RC. In fact it has several of them aside from this corner that people have painted themselves into with it ... - Its primary author, who is also a DD and comaint of the package, is currently MIA with a new job that has consumed pretty much every minute of his time for the last 2 years or so now. The people who remain that are hacking on it, can't even do a formal new release at present, because he is the only one with access to the systems they need to do that, and is not available to do so. I'm not convinced by this complaint. The whole point of free software is that it is possible to carry on without needing the cooperation or involvement of one's upstreams. Also, are you saying you think mumble should be removed ? Now is a bit late to be deciding this. If I'd known that Thorvald was not going to be here to manage this transition for Wheezy, I'd have never agreed to shipping libcelt in the Squeeze release either, and would have instead kept it in sid only. If I'd known that his plan to have all other distros ship the 0.7.1 release as a temporary interoperability snapshot would fail as dismally as it did (no other distro shipped this version except debian derivatives who took it from us), I'd have similarly vetoed the idea of shipping this as a public library in the last stable release too. I think interoperability with the ecosystem of our derivatives is sufficiently important that it's worth preserving. Mumble already ships this as an embedded private library on every system other than direct Debian derivatives. I'm not sure exactly what you mean. Do you mean they support some other version of the celt codec ? Or are you just talking about how they manage the packaging ? I certainly think it's better to have the celt codec, even if it's an unreleased and now-unmaintained-upstream snapshot, in a separate package, than in an embedded library. I see that this proposal has already resulted in Chris skating down the slippery slope of let's re-enable it for everything else too. And we'll get people with no experience or prior involvement to maintain it, and we'll enable multi-arch too, and ... Your message has too much personal animosity in it. So I really hope that it's actually clear to the people presiding over the tech-ctte, that the *whole point* of a codec is that it lets you *interoperate* with others. Which this snapshot does not. It lets you interoperate with Ubuntu, and with users running squeeze. If people really want to do this for mumble, then it really ought to be done as a private implementation detail, like it is for every other platform - not by setting traps in public for the unsuspecting and otherwise uninformed. If we ship celt publicly, we are sending the message that people should use it. That's the wrong message for an obsolete, unmaintained codec, and there is no reason to tie 'being pragmatic' about the mumble screwup to making an even bigger mistake. That's not 'avoiding risk', it's not avoiding risk. And one that is really, really trivial to avoid. Just to be clear, you seem to be suggesting that while reintroducing libcelt as a package is a bad thing, it would be fine to reintroduce it as an embedded library in mumble ? My general feeling is that mumble is currently in an awkward state which really doesn't make it a good release candidate for Wheezy, and we'd probably be best served by simply dropping it from testing, fixing this in sid as the fixes come or are needed, and then pushing snapshots to bpo as we have reasonable candidates for that. I don't think this is a good approach. But I'm pleased to see that you agree that this is probably an RC problem. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#682010: #682010 re celt and mumble referred to the TC
On Thursday, July 19, 2012 05:38:47, Ron wrote: ... - This is a serious problem for mumble at least and is arguably RC. Yes, mumble has a serious problem, that is arguably RC. In fact it has several of them aside from this corner that people have painted themselves into with it ... […] FWIW, even the original poster of the bug in question later agreed in calmer discussion on IRC that the Right Thing for mumble to do is to release 1.3 and accelerate the migration, and that is only being delayed now by the first point above. I don't disagree, but I'm wondering how the migration could be (or could have been) accelerated. […] If I'd known that Thorvald was not going to be here to manage this transition for Wheezy, I'd have never agreed to shipping libcelt in the Squeeze release either, and would have instead kept it in sid only. If I'd known that his plan to have all other distros ship the 0.7.1 release as a temporary interoperability snapshot would fail as dismally as it did (no other distro shipped this version except debian derivatives who took it from us), I'd have similarly vetoed the idea of shipping this as a public library in the last stable release too. This was definitely not clear; if it had been I wouldn't have considered re- enabling it as a potential solution. Mumble already ships this as an embedded private library on every system other than direct Debian derivatives. Just for clarification: does the Mumble client as shipped with Debian also contain the embedded private library? I assume it would be possible to reintroduce a celt package which was very similar to the one recently removed, so as to avoid risking unnecessary bugs. I see that this proposal has already resulted in Chris skating down the slippery slope of let's re-enable it for everything else too. And we'll get people with no experience or prior involvement to maintain it, and we'll enable multi-arch too, and ... In terms of re-enabling CELT, I was simply looking upon this as a matter of fairness to other maintainers. If it's decided only Mumble requires an exception to use CELT 0.7.1 (which right now doesn't sound like the right thing to do), I'm fine with that. […] My general feeling is that mumble is currently in an awkward state which really doesn't make it a good release candidate for Wheezy, and we'd probably be best served by simply dropping it from testing, fixing this in sid as the fixes come or are needed, and then pushing snapshots to bpo as we have reasonable candidates for that. That was my original recommendation to the release team, but Phil offered to cut us some slack with letting things in if I did try to salvage it for Wheezy proper, so Svedrin and I put in the effort to make that as possible as it might be. Maybe that really was a mistake. I don't mind taking full responsibility for my mistakes - but being bullied into making even bigger mistakes, by people who didn't understand the set that created the problem in the first place, is not in my usual definition of wisdom, and the crux of my disagreement with the crusade that Chris has embarked on here. The disagreement was that I wanted a working Mumble, or a warning in NEWS.Debian concerning the compatability issues. That's not a crusade, so I object to this mischaracterization. Since he didn't bother to wait for Josh and I to discuss that further, now we're here ... There was no indication of any kind that the discussion was going to continue, otherwise I would have delayed the summary to the TC. -- Chris -- Chris Knadle chris.kna...@coredump.us -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#682010: #682010 re celt and mumble referred to the TC
Chris Knadle writes (Bug#682010: #682010 re celt and mumble referred to the TC): There was no indication of any kind that the discussion was going to continue, otherwise I would have delayed the summary to the TC. I think you were right to escalate this to the TC when you did. The longer we get into the wheezy freeze, the more difficult it becomes to fix this for wheezy. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#682010: #682010 re celt and mumble referred to the TC
On Thu, Jul 19, 2012 at 12:55:59PM +0100, Ian Jackson wrote: Ron writes (Re: #682010 re celt and mumble referred to the TC): You understand this is a fairly arbitrary 'daily' snapshot of an experimental research codec, from ~2.5 years ago, that nobody has looked at since that day, that nobody has committed to maintaining, that its author has explicitly said he has no interest in maintaining, that has had large parts completely rewritten since for all manner of reasons, and is bitstream incompatible with every other snapshot that was ever released, right? However, it appears that some other distros have chosen to ship this particular version. For example at least Ubuntu are still distributing it. I don't want to niggle over words here, but chosen would imply that somebody actually exercised some conscious judgement in that decision. Which there doesn't really seem to be a whole lot of evidence for. Nobody from Ubuntu has ever spoken to me or upstream about this, and the version they are shipping appears to simply have been auto-imported from Debian with no changes or human intervention. There are open bugs in launchpad like: - LibCelt Package Breaking Apt-Get - package libcelt0-0 0.7.1-1 failed to install/upgrade: Which nobody has commented on or triaged. So the only rational conclusion there is that Ubuntu will do whatever we do - and naturally they will then lag somewhat. If we are to insist on not changing things until they do, then we're going to be deadlocked shipping obsolete, unmaintained, code for a long time ... If they *had* asked, they would have got the same advice on offer here. (and yes, those are almost surely bogus bugs - but that just reinforces the point that not even minimal attention is being paid to this there) However if this list is any indication: https://bugs.launchpad.net/ubuntu/+source/mumble Then the old version of mumble that Ubuntu is shipping looks long overdue for an update anyway, since there's a long list of reasons there why its users can't communicate with anyone at all, none of which have anything to do with celt ... - This is a serious problem for mumble at least and is arguably RC. Yes, mumble has a serious problem, that is arguably RC. In fact it has several of them aside from this corner that people have painted themselves into with it ... - Its primary author, who is also a DD and comaint of the package, is currently MIA with a new job that has consumed pretty much every minute of his time for the last 2 years or so now. The people who remain that are hacking on it, can't even do a formal new release at present, because he is the only one with access to the systems they need to do that, and is not available to do so. I'm not convinced by this complaint. The whole point of free software is that it is possible to carry on without needing the cooperation or involvement of one's upstreams. Sure, but in order for that possibility to be real, someone has to collapse the waveform and step up to do the work. So far nobody has stepped in to fill Thorvald's shoes. I only stepped up to help with the packages because I consider him to be a friend (and indeed I also advocated him to NM because I consider him a prolific and talented programmer - they aren't easy shoes to fill by a part time dabbler). And I don't want to sound like I'm taking potshots at Ubuntu here (because absolutely I am not) but this seems directly relevant to your first point above, because maintenance of mumble there appears to have completely stopped dead in its tracks when Thorvald vanished off there scene there too, and they haven't yet replaced him either. The important question for the release is reality, not theoretical possibilities. The simple reality is, he's not easy to replace, and so far hasn't been. And the result of that is clearly showing. Also, are you saying you think mumble should be removed ? Now is a bit late to be deciding this. I'm saying that what I've seen in the short time since I stepped into this tarpit, along with the rush of RC bugs like #675955 and #676816 and things like #628847, #615492, #673602 -- all of which have nothing to do with anything I changed -- don't exactly inspire confidence in its release readiness to me. I'm not saying they can't be fixed. But given that #676816 was a guaranteed crasher since the day it was introduced in March 2010 and only just noticed in the last month - I'm not terribly confident that we've seen the last of this sort of thing. I talked through this with people from the release team as the freeze deadline loomed -- deciding to remove it earlier would have been premature. But you're quite correct about the 'lateness' in the cycle limiting our choices. If I'd known that Thorvald was not going to be here to manage this transition for Wheezy, I'd have never agreed to shipping libcelt in the Squeeze release either, and would
Bug#682010: #682010 re celt and mumble referred to the TC
Ron writes (Bug#682010: #682010 re celt and mumble referred to the TC): I don't want to niggle over words here, but chosen would imply that somebody actually exercised some conscious judgement in that decision. Which there doesn't really seem to be a whole lot of evidence for. Nobody from Ubuntu has ever spoken to me or upstream about this, and the version they are shipping appears to simply have been auto-imported from Debian with no changes or human intervention. There are open bugs in launchpad like: - LibCelt Package Breaking Apt-Get - package libcelt0-0 0.7.1-1 failed to install/upgrade: You are implying that the package does not work in Ubuntu. However I have personally witnessed Ubuntu users using it. You seem to be grasping at straws. So the only rational conclusion there is that Ubuntu will do whatever we do - and naturally they will then lag somewhat. If we are to insist on not changing things until they do, then we're going to be deadlocked shipping obsolete, unmaintained, code for a long time ... Alternatively we could wait until the new opus codec is widely deployed. Mumble upstream do have a transition plan. I don't think pull the plug on celt is a transition plan. I'm not convinced by this complaint. The whole point of free software is that it is possible to carry on without needing the cooperation or involvement of one's upstreams. Sure, but in order for that possibility to be real, someone has to collapse the waveform and step up to do the work. So far nobody has stepped in to fill Thorvald's shoes. I only stepped up to help with the packages because I consider him to be a friend (and indeed I also advocated him to NM because I consider him a prolific and talented programmer - they aren't easy shoes to fill by a part time dabbler). I have spoken privately to people who are involved in mumble upstream and they seem to be keen to continue. Given how easy this really is to fix without creating that kind of exposure, I'm a bit lost for words at the push back it seems to be getting from some quarters ... Your plan for a fix is total incompatibility with other deployed installations. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#682010: #682010 re celt and mumble referred to the TC
On Thu, Jul 19, 2012 at 07:39:37PM +0100, Ian Jackson wrote: Ron writes (Bug#682010: #682010 re celt and mumble referred to the TC): I don't want to niggle over words here, but chosen would imply that somebody actually exercised some conscious judgement in that decision. Which there doesn't really seem to be a whole lot of evidence for. Nobody from Ubuntu has ever spoken to me or upstream about this, and the version they are shipping appears to simply have been auto-imported from Debian with no changes or human intervention. There are open bugs in launchpad like: - LibCelt Package Breaking Apt-Get - package libcelt0-0 0.7.1-1 failed to install/upgrade: You are implying that the package does not work in Ubuntu. However I have personally witnessed Ubuntu users using it. You seem to be grasping at straws. mumble hasn't been updated in Ubuntu in its last 4 releases. Are you saying the list of bugs in launchpad that are fixed in Debian are not real? And that it isn't actually unmaintained there? So the only rational conclusion there is that Ubuntu will do whatever we do - and naturally they will then lag somewhat. If we are to insist on not changing things until they do, then we're going to be deadlocked shipping obsolete, unmaintained, code for a long time ... Alternatively we could wait until the new opus codec is widely deployed. Mumble upstream do have a transition plan. I don't think pull the plug on celt is a transition plan. Firefox has it enabled in their beta, which will ship as stable in something like 6 weeks time. How much more widely deployed than 30% of the world did you have in mind? All the name brand distros are already shipping it. The plug was pulled on celt a long time ago. I'm not convinced by this complaint. The whole point of free software is that it is possible to carry on without needing the cooperation or involvement of one's upstreams. Sure, but in order for that possibility to be real, someone has to collapse the waveform and step up to do the work. So far nobody has stepped in to fill Thorvald's shoes. I only stepped up to help with the packages because I consider him to be a friend (and indeed I also advocated him to NM because I consider him a prolific and talented programmer - they aren't easy shoes to fill by a part time dabbler). I have spoken privately to people who are involved in mumble upstream and they seem to be keen to continue. I have spoken to them publicly. All of them refused to take any responsibility for actually maintaining celt beyond we'll apply a patch if you give us one. If they had committed to that we wouldn't be having this discussion. But yes, other than that they were quite keen for us to just close our eyes and keep shipping it until news of the disaster hit /. They also recommended we just ignore the zeroc-ice ABI breakage and pretend it didn't happen too. Oh, and they also removed support for speex in the last couple of weeks, which otherwise was a baseline for interop that is still maintained. Is this the kind of technical excellence you want to overrule a maintainer to achieve? Given how easy this really is to fix without creating that kind of exposure, I'm a bit lost for words at the push back it seems to be getting from some quarters ... Your plan for a fix is total incompatibility with other deployed installations. And your alternate plan is make everyone insecure rather than propagating a security fix release? How long do you really think it will take for that to be rolled out? Alternatively, you could send me a patch to restore speex support. I'll gladly apply that if incompatibility is your actual concern. Ron -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
