Re: [Bitcoin-development] Linux packaging letter
My signature: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Linux distribution packaging and Bitcoin 2013-07-23 This note summarises the dangers inherent in the Linux distribution packaging model for Bitcoin, and forms a request from upstream maintainers to not distribute Bitcoin node software as part of distribution package repositories without understanding the special requirements of Bitcoin. Distributors typically unbundle internal libraries and apply other patches for a variety of generally good reasons, including ensuring that security-critical fixes can be applied once, rather than multiple times for many different packages. In most cases, the common distribution packaging policy has many advantages. However, Bitcoin nodes are an unusual category of software: they implement a complex group consensus in which every client verifies the behaviour of every other exactly. Even an exceptionally subtle change - including apparently harmless bugfixes - can cause a failure to reach consensus. A consensus failure of one client is a security risk to the user of that client. A significant number of nodes failing to reach consensus - as happened in March 2013 due to a change in database libraries[1] - is a critical problem that threatens the functionality and security of the system for all users. For this reason, it is _vital_ that as much of the network as possible uses _unmodified_ implementations that have been carefully audited and tested, including dependencies. For instance, if the included copy of LevelDB in bitcoind is replaced by a system-wide shared library, _any_ change to that shared library requires auditing and testing, a requirement generally not met by standard distributor packaging practices. Because distributed global consensus is a new area of computer science research, the undersigned request that distributors refrain from packaging Bitcoin node software (including bitcoind and Bitcoin-Qt) and direct users to the upstream-provided binaries instead _until they understand the unique testing procedures and other requirements to achieve consensus_. Beyond being globally consistent, upstream binaries are produced using a reproducible build system[2], ensuring that they can be audited for backdoors. 1. https://en.bitcoin.it/wiki/BIP_0050 2. http://gitian.org/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBCAAGBQJR9WC5AAoJEEWCsU4mNhiPg6UH/2oHzBWBPaQMhH/GCTHQEi5U 7GSRfqwihIs/M2ROHLNq0HhgWR7mPAh5TTI6+tG95FCGCGNZq0seqw9wW4ZyGCoc VueY51q4hcn23405oLD/QGK2lDxxywWY8XtFYVPqAzXTq6zRzgpNJkjoRtOAUOP7 3PrRkimYYyj0KrqFg+cEvZDT27dkeX+5PXM6Ua0o7h/TlhR2RJPhej5DI8cNLXgA f0t+mES4Apb6zLgnEYYlPp22FR9vuFcJO3z1akhVL4DLUMqr58NYHLVnH1FH0Jhn hVuld159QtCjQ5Qyn19cn86akTQJVi+ikCXqaKriCc2jBFX7TCI8WTDc6aSZpsQ= =oX5d -END PGP SIGNATURE- -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
Gregory Maxwell gmaxw...@gmail.com writes: It's portable to anything that can run the relevant VMs. Uh provided you don't mind cross compiling everything from an unbuntu VM. It certainly would be nice if the trusted-computing-base for gitian were a bit smaller, thats an area for long term improvement for sure. Thanks - I'll look forward to this being portable someday. Right now it sounds similar to a windows binary but you can use wine with substitution of variables :-) People may want to look at the NetBSD build system, which I think achieves bit-identical builds from different hosts (but I haven't really checked), by having the toolchain be part of the source and building cross-compilers from host to target and then using those to build the system. Say Bitcoin used a backing database which had an unknown a bug where any item with a key that begins with 0xDEADBEEF returns not found when queried, even if its in the DB. Once discovered, any database library would want to fix that quickly and they'd fix it in a point release without reservation. They might not even release note that particular fix it if went along with some others, it could even be fixed accidentally. Now say that we have a state where half the Bitcoin network is running the old buggy version, and half is running the fixed version. Someone creates a transaction with ID 0xDEADBEEF... and then subsequently spends the output of that transaction. This could be by pure chance or it could be a malicious act. To half the network that spending transaction looks like someone spending coin from nowhere, a violation of the rules. The consensus would then fork, effectively partitioning the network. On each fork any coin could be spent twice, and the fork will only be resolvable by one side or the other abandoning their state (generally the more permissive side would need to be abandoned because the permissive one is tolerant of the restrictive one's behavior) by manually downgrading or patching software. As a result of this parties who believed some of their transactions were safely settled would find them reversed by people who exploited the inconsistent consensus. Thanks for the explanation - that indeed makes sense. multiple packages is difficult, and runs into A wants only n of C, while B wants only m. My understanding is that gentoo is actually able to handle this (and does, for Bitcoin)— and really I presume just about everything else could with enough effort. I certainly wouldn't ask anyone else to do that. If you're really getting into the rathole of building separate libraries just for Bitcoin the value of packaging it goes away. Well, if you insist on not having updates and bugfixes, then either it's the included version or there's a special package just for you. Typically packaging systems don't like included versions because often a package will have a security bug fixed long before there are updates of packages that bundle that fixed version.But given bitcoin's special needs, that means you have to stay on top of these dependent included packages and re-release if there are security fixes (that don't break consensus). Running a complete set of tests is a start— though the unit tests are not and cannot be adequate. There is a full systems testing harnesses which should be used on new platforms. Even that though isn't really adequate, as it is currently infeasible to even achieve complete test coverage in things like cryptographic libraries and database environments. It would be nice if the regression tests were installed and it were normal culturallly for end-users to run them. Thanks again for the explanation; I understand where you are coming from now. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Wed, Jul 24, 2013 at 4:28 AM, Mike Hearn m...@plan99.net wrote: Yeah, if anyone wants to make the letter more digestable please do propose an alternative, although by this point it's probably not worth it as people have already signed. I'm working on a more digestable alternative: https://gist.github.com/jgarzik/6065679 Should be ready in another ~24 hours, as its obviously incomplete right now. Alas the publishing of the lame version (which yes, I did ACK) didn't give me time to finish my version. I worked on Fedora packaging while at Red Hat, so hopefully, I have a bit of insight here. Was also thinking about publishing this on opensource.com. -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Wed, Jul 24, 2013 at 10:28:16AM +0200, Mike Hearn wrote: Yeah, if anyone wants to make the letter more digestable please do propose an alternative, although by this point it's probably not worth it as people have already signed. Okay, here's my attempt: https://docs.google.com/document/d/1m3wyBIjqwPQ3wxVT7P_wJtdWt9a9RXvt9NV7rggLAOs/edit# Please feel free to use any or all of it as you see fit. FWIW, Gregory is right that my original draft was much more brusque. The pain in the packaging relationship travels both ways. I have in the past wasted a lot of time due to bogus packaging applied by non-expert packagers that broke things. In fact the project I was a part of adopted a policy of automatically closing bug reports from people who were using distributor packages (any distro) because the quality was so inconsistent and so many subtle bugs were introduced. If packagers hear upstreams cry about packaging a lot, I think you should keep an open mind that some of them probably know what they're talking about. We really shouldn't have to beg and cajole here. Saying we have our reasons and we want you to stop should be enough. Yes, I know what you mean. Regards, Zooko -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tue, Jul 23, 2013 at 4:01 PM, Mike Hearn m...@plan99.net wrote: Hi, Some of us have put together an open letter to the Linux packaging community, outlining why Bitcoin is different to other programs and asking them to not patch or modify the upstream sources. Please consider signing it if you agree (I think the wording by now is fine, so don't edit the contents - use the comment feature if you want to though). https://docs.google.com/document/d/1naenR6N6fMWSpHM0f4jpQhYBEkCEQDbLBs8AXC19Y-o/edit#heading=h.i7tz3gqh65mi The trigger for this is the discovery that Debian bitcoind's got split out of the consensus some time in April, for reasons that nobody yet figured out but is presumably related to a patch (eg it uses system leveldb). Hi Mike, Debian's bitcoin is maintained on an open and archived mailing list and git repo: Debian Bitcoin Packaging Team pkg-bitcoin-de...@lists.alioth.debian.org If there is a problem or question, getting an answer should be really easy. It would be good to include them in the discussion there (I CC:ed the list). If the upstream developers have a consensus that distribution packaging is not in the best interest of the project, then I personally would defer to their judgement and request removal. I'm leaving this open for other opinions from the Debian side. That said, let me summarize the arguments and see if we can figure out a permanent solution: 1) It appears that the consensus of upstream developers is that any distributed binary should only be linked against libraries that the bitcoin developers have tested and audited since any compatibility bug is a risk to both the user and the network. Response: Is there a way to certify the Debian libraries? Debian bitcoind/bitcoin-qt runs the compile test during all architectures. MIPS has been failing recently, but no one has looked into it yet. Perhaps it's not worth developers efforts yet, but at some point the technology should reach a point it can be redistributed. 2) Bitcoin is new technology, so any patches have the ability of harming both the network and user. Response: I, and I'm sure everyone else working on it, totally agrees. All patches are public [1], you can see that the patches are only to the build system (except for one adding a debug message). Is there a specific patch or bug that is problematic? This seems to be reiterating (1) above: don't patch the build system to use libraries that were not audited by the developers. The two solutions are: (1) no one besides the upstream developers compiles and distributes binaries, ever, or (2) upstream comes up with a system where someone besides them can compile working binaries for distribution. Most likely the best solution is to do (1) until upstream sets up a system to allow (2). I'm curious as to other's opinions. Thanks, Scott [1] http://anonscm.debian.org/gitweb/?p=collab-maint/bitcoin.git;a=tree;f=debian/patches;h=ba576f9f3ddad47a2f85dcbfb7a0b3482834f19f;hb=HEAD -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tuesday, July 23, 2013 10:02:28 PM Scott Howard wrote: 1) It appears that the consensus of upstream developers is that any distributed binary should only be linked against libraries that the bitcoin developers have tested and audited since any compatibility bug is a risk to both the user and the network. Response: Is there a way to certify the Debian libraries? Debian bitcoind/bitcoin-qt runs the compile test during all architectures. It doesn't need to be audited by any given person or team, just someone who understands the issues and can dedicate the time to doing a competent audit. Testing bitcoind/bitcoin-qt is not sufficient: you must guarantee that your libraries (especially LevelDB) are bug-for-bug compatible with the ones used by everyone else. And not only the current versions, but every future version to ever hit the repository. This means a lot of additional work for the maintainers of the library packages, and the security team; for example, the security team must understand that they *cannot* deploy a critical security bugfix to LevelDB until someone competent has reviewed that it is behaviourally (including bug behaviours!) compatible with the versions in use everywhere else/previously. I think it is likely all this additional work/delays will be considered unacceptable to your library/security teams, thus using the bundled/embedded LevelDB is probably the best solution. MIPS has been failing recently, but no one has looked into it yet. Perhaps it's not worth developers efforts yet, but at some point the technology should reach a point it can be redistributed. MIPS (and any other big endian architecture) has *always* failed on the Satoshi codebase, and will not be supported until someone has time to dedicate to fixing the numerous little-endian assumptions in the code. I have an incomplete port, but it isn't very high on my priority list to figure out what else it's missing. 2) Bitcoin is new technology, so any patches have the ability of harming both the network and user. Response: I, and I'm sure everyone else working on it, totally agrees. All patches are public [1], you can see that the patches are only to the build system (except for one adding a debug message). Is there a specific patch or bug that is problematic? This seems to be reiterating (1) above: don't patch the build system to use libraries that were not audited by the developers. The two solutions are: (1) no one besides the upstream developers compiles and distributes binaries, ever, or (2) upstream comes up with a system where someone besides them can compile working binaries for distribution. Most likely the best solution is to do (1) until upstream sets up a system to allow (2). Debian could probably get away with packaging Bitcoin-Qt and bitcoind as-is with no modifications, and not have any problems. It's only when you begin making modifications that it becomes a problem. There are also some concerns that it puts a much larger price on compromising Debian's build servers and/or repositories (suddenly the attacker can steal a LOT of money). The official binaries are not simply built by upstream developers: using Gitian, *anyone* can produce bit-for-bit identical binaries. Official releases are only published after 3 or more people have done an independent compile and signed the output. It would be excellent if the whole of Debian could work toward achieving this level of security eventually, which would make distributing Bitcoin node software much safer as well. Luke signature.asc Description: This is a digitally signed message part. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tue, Jul 23, 2013 at 10:01:55PM +0200, Mike Hearn wrote: The trigger for this is the discovery that Debian bitcoind's got split out of the consensus some time in April, for reasons that nobody yet figured out but is presumably related to a patch (eg it uses system leveldb). Just to make sure there are no misunderstandings, as far as I know there is no reason to assume the reported problem (comment on #2726) is: 1) a fork (it's an indeterministic and avoidable database corruption, it seems) 2) related to leveldb 3) reproducible by more than one person 4) debian's fault. That said, I think reaching out to packagers to educate them about the risks is a good idea - but let's not blame people before we understand our own problems. -- Pieter -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
I find it interesting that this is a linux packaging letter. How much of this applies to pkgsrc, FreeBSD ports, OpenBSD ports, and other non-Linux packaging systems (pkgsrc supports Linux as on of 20 operating systems, but is not a Linux packaging system)? Is the repeatable build infrastructure portable (to any reasonable mostly-POSIX-compliant system, with gcc or clang)? I have the vague impression it's Ubuntu only, but I am very unclear on this point. How does this repeatableness interact with building for multiple operating systems and cpu types (say 20 OS, with typically 3 versions of the OS for each, with 1-20 cpu types per OS, for a cross-product of perhaps 200 combinations)? Requiring precise library depdendencies is quite awkward. Certainly requiring new enough to avoid known bugs is understandable, but that should be caught at configure time and fail. Synchronous updates of multiple packages is difficult, and runs into A wants only n of C, while B wants only m. So if you are talking about running regression tests with the set of versions of a dependency that are considered reasonable, and there's therefore a solution to the multiple-package constraint problem, that seems ok. It seems like a bug that the package will build on BE systems and then fail tests. If it's known not to be ok, it seems that absent some configure-time flag the build should fail. Asking people not to patch should mean willingnesss to make accomodation in the master sources for build issues for multiple packaging systems. I haven't gotten around to packaging this for pkgsrc - so far I only have the energy to lurk (due to too many things on the todo list). But I often find that some changes are needed. If you're willing (in theory) to add in configure flags to control build behavior (in a way that you can audit and decide is safe), that's great, and of course we can discuss an actual situation when one gets figured out. Greg -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tue, Jul 23, 2013 at 4:23 PM, Greg Troxel g...@work.lexort.com wrote: Is the repeatable build infrastructure portable (to any reasonable mostly-POSIX-compliant system, with gcc or clang)? I have the vague It's portable to anything that can run the relevant VMs. Uh provided you don't mind cross compiling everything from an unbuntu VM. It certainly would be nice if the trusted-computing-base for gitian were a bit smaller, thats an area for long term improvement for sure. It may need some massaging. The tor project is beginning to use the same infrastructure, so this could be usefully conserved work. Likewise expanding the supported output targets would be great— though in the case of Bitcoin this is bounded by resources to adequately QA builds on alternative targets. Requiring precise library depdendencies is quite awkward. Certainly requiring new enough to avoid known bugs is understandable, but that should be caught at configure time and fail. In some cases packages solving bugs is problematic for Bitcoin. This is something that it seems to take a whiteboard to explain, so I apologize for the opacity of simple email here. From a technical perspective Bitcoin's whole purpose is getting a whole bunch of computers world wide to reach a bit identical agreement on the content of a database, subject to a whole pile of rules, in the face of potentially malicious input, without any trusted parties at all (even the guy you got the software from, assuming you have the resources to audit it). I'll walk through a simple example: Say Bitcoin used a backing database which had an unknown a bug where any item with a key that begins with 0xDEADBEEF returns not found when queried, even if its in the DB. Once discovered, any database library would want to fix that quickly and they'd fix it in a point release without reservation. They might not even release note that particular fix it if went along with some others, it could even be fixed accidentally. Now say that we have a state where half the Bitcoin network is running the old buggy version, and half is running the fixed version. Someone creates a transaction with ID 0xDEADBEEF... and then subsequently spends the output of that transaction. This could be by pure chance or it could be a malicious act. To half the network that spending transaction looks like someone spending coin from nowhere, a violation of the rules. The consensus would then fork, effectively partitioning the network. On each fork any coin could be spent twice, and the fork will only be resolvable by one side or the other abandoning their state (generally the more permissive side would need to be abandoned because the permissive one is tolerant of the restrictive one's behavior) by manually downgrading or patching software. As a result of this parties who believed some of their transactions were safely settled would find them reversed by people who exploited the inconsistent consensus. To deploy such a fix in Bitcoin without creating a risk for participants we need to make a staged revision of the network protocol rules: There would be a protocol update that fixed the database bug _and_ explicitly rejected 0xDEADBEEF transactions until either some far out future date or until triggered by quorum sensing (or both). The users of Bitcoin would all be advised that they had to apply fixes/workaround by the switchover point or be left out of service or vulnerable. At designated time / quorum nodes would simultaneously switch to the new behavior. (Or in some cases, we'd just move the 'bug' into the Bitcoin code so that it can be fixed in the database, and we'd then just keep it forever, depending on how harmful it was to Bitcoin, a one if 4 billion chance of having to rewrite a transaction wouldn't be a big deal) We've done these organized updates to solve problems (as various flaws in Bitcoin itself can present similar consensus risks) several times with great success, typical time horizons spanning for months to years. But it cannot work if the behavior is changed out from under the software. Fortunately, if the number of users running with an uncontrolled consensus relevant inconsistent behavior is small the danger is only to themselves (and, perhaps, their customers). I'm not happy to see anyone get harmed, but it's better if its few people than many. This is part of the reason that it's a linux packaging letter, since for Bitcoin the combination of uncoordinated patching and non-trivial userbases appears to be currently unique to GNU/Linux systems. Though indeed, the concerns do apply more broadly. multiple packages is difficult, and runs into A wants only n of C, while B wants only m. My understanding is that gentoo is actually able to handle this (and does, for Bitcoin)— and really I presume just about everything else could with enough effort. I certainly wouldn't ask anyone else to do that. If you're really getting into the rathole of building separate libraries
Re: [Bitcoin-development] Linux packaging letter
Honestly, until I read the quoted part of your response, I actually wasn't in favor of this whole thing since in general the types of issues being mentioned are, in large part, the types of issues that maintainers deal with all the time. On Jul 23, 2013, at 3:02 PM, Scott Howard showard...@gmail.com wrote: Response: Is there a way to certify the Debian libraries? Debian bitcoind/bitcoin-qt runs the compile test during all architectures. MIPS has been failing recently, but no one has looked into it yet. Perhaps it's not worth developers efforts yet, but at some point the technology should reach a point it can be redistributed. The fact that you're even trying to package and/or at some point have packaged and shipped big endian binaries is straight up *NEGLIGENT.* Stop that. Now. It won't work. Thanks for showing that this *is* necessary, I guess. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tue, Jul 23, 2013 at 9:45 PM, Douglas Huff dh...@jrbobdobbs.org wrote: Honestly, until I read the quoted part of your response, I actually wasn't in favor of this whole thing since in general the types of issues being mentioned are, in large part, the types of issues that maintainers deal with all the time. On Jul 23, 2013, at 3:02 PM, Scott Howard showard...@gmail.com wrote: Response: Is there a way to certify the Debian libraries? Debian bitcoind/bitcoin-qt runs the compile test during all architectures. MIPS has been failing recently, but no one has looked into it yet. Perhaps it's not worth developers efforts yet, but at some point the technology should reach a point it can be redistributed. The fact that you're even trying to package and/or at some point have packaged and shipped big endian binaries is straight up *NEGLIGENT.* Stop that. Now. It won't work. Thanks for showing that this *is* necessary, I guess. before people get too upset, I'm talking about little-endian MIPS (debian-mipsel) http://www.debian.org/ports/mips/ -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Linux packaging letter
On Tue, Jul 23, 2013 at 6:26 PM, Luke-Jr l...@dashjr.org wrote: This means a lot of additional work for the maintainers of the library packages, and the security team; for example, the security team must understand that they *cannot* deploy a critical security bugfix to LevelDB until someone competent has reviewed that it is behaviourally (including bug behaviours!) compatible with the versions in use everywhere else/previously. I think it is likely all this additional work/delays will be considered unacceptable to your library/security teams, thus using the bundled/embedded LevelDB is probably the best solution. The above is a key point, lukejr addressed it well I think it is likely all this additional work/delays will be considered unacceptable to your library/security teams, thus using the bundled/embedded LevelDB is probably the best solution. MIPS has been failing recently, but no one has looked into it yet. Perhaps it's not worth developers efforts yet, but at some point the technology should reach a point it can be redistributed. MIPS (and any other big endian architecture) has *always* failed on the Satoshi codebase, and will not be supported until someone has time to dedicate to fixing the numerous little-endian assumptions in the code. I have an incomplete port, but it isn't very high on my priority list to figure out what else it's missing. To be clear, bitcoind/bitcoin-qt is only built on little endian machines https://buildd.debian.org/status/package.php?p=bitcoin Debian could probably get away with packaging Bitcoin-Qt and bitcoind as-is with no modifications, and not have any problems. It's only when you begin making modifications that it becomes a problem. There are also some concerns that it puts a much larger price on compromising Debian's build servers and/or repositories (suddenly the attacker can steal a LOT of money). The only current modification is to use system leveldb instead of the packaged leveldb. (There is also a patch porting libmemenv.a to several other architectures, but that is only used in test suites - so it shouldn't pose a risk to users). The official binaries are not simply built by upstream developers: using Gitian, *anyone* can produce bit-for-bit identical binaries. Official releases are only published after 3 or more people have done an independent compile and signed the output. It would be excellent if the whole of Debian could work toward achieving this level of security eventually, which would make distributing Bitcoin node software much safer as well. Ironically, debian (in general) doesn't trust upstream security maintenance of third part libraries - that's why they typically get dropped in favor of use system libraries. In this case, upstream doesn't trust (rightfully) that some future debian security team bug fix to a stable library won't be tested properly against bitcoin, causing problems for users (since bitcoin might expect buggy library behavior). I'm not the original packager or maintainer - I just came across the package in really bad shape and helped bring it to something reasonable and have done the most recent uploads (since 0.8, I believe). Since updated libraries could pose a security risk because bitcoin may expect buggy behavior, I think that is a good argument for debian to use the included library. However, I'm just a recent helper - I still want to hear what people who have been doing this for longer think. ~Scott -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
