Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
On Monday, June 18, 2012 3:27:52 AM grarpamp wrote: > So I get that github/master is the obvious top of things. > But in looking at where the tags are between repositories, > it's still not clear to me what the workflow is. Workflow is all new development takes place in master during release windows. Eventually, those windows close and master is cleaned up and bugfix'd for the next 0.x release. Occasionally, when 0.N.0 has some problem before the next release window opens, Gavin will use it to roll a 0.N.1 (and recently even a 0.N.2 and 0.N.2.2). Once the release window for the next 0.N version opens, I import the (last bugfix-only commit after the final 0.N.M release made in master) into the stable repository as the 0.N.x branch, and begin applying backports. When there's significant backports, I'll tag another 0.N.M from the branch and possibly release Windows binaries. Usually this happens around the same time as master becomes the next 0.N.0 release. > Example... > > There are these release tarballs on sourceforge, which all have > tags in github, yet no tags in gitorious. There are no 'x' branches > on github, yet there is one release applicable branch on gitorious. > > I guess I'd expect to see, that if as hinted by Luke that gitorious > has the stable trees, that there would be release tags there, laid > down at some comfy point in time on the 'x' stable branches there. > (The stable branches inheriting new code from master). But there > are no such tags. I guess I've been neglecting to update the stable repo with releases tagged in master. It should be fixed now. Luke -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
> be sure to have good backups that never touched the new code... > We have at various times had bugs in master that would corrupt > wallets Sorry, that's to be expected, I shouldn't have asked. > It would be very helpful if anyone offering bitcoin services would > setup parallel toy versions of your sites on testnet... Good point. > we don't currently have enough testing activity on master. I usually test compile / report current and stable of things I use. So I get that github/master is the obvious top of things. But in looking at where the tags are between repositories, it's still not clear to me what the workflow is. Example... There are these release tarballs on sourceforge, which all have tags in github, yet no tags in gitorious. There are no 'x' branches on github, yet there is one release applicable branch on gitorious. I guess I'd expect to see, that if as hinted by Luke that gitorious has the stable trees, that there would be release tags there, laid down at some comfy point in time on the 'x' stable branches there. (The stable branches inheriting new code from master). But there are no such tags. And the releases/tags seem to magically appear from nowhere on github :) Again, I'm trying to extricate myself from CVS here. # sourceforge tarballs 0.6.0 0.6.1 0.6.2 0.6.2.2 # github branches remotes/origin/master432d28d Merge pull request #1477 from gmaxwell/master remotes/origin/0.6.2 40fd689 Bump version to 0.6.2.2 for osx-special build # github tags v0.6.0 v0.6.1 v0.6.2 v0.6.2.1 v0.6.2.2 # gitorius branches remotes/origin/0.6.0.x d354f94 Merge branch '0.5.x' into 0.6.0.x remotes/origin/0.6.x 5e322a7 Merge branch '0.6.0.x' into 0.6.x # gitorious tags v0.6.0.7 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposed new P2P command and response: getcmds, cmdlist
Sorry for the duplication Amir, I meant to send this to everyone: BitTorrent might be an example to look to here. It's a peer-to-peer network that has undergone many significant protocol upgrades over the years while maintaining compatibility. More recent clients have had the ability to expose the capabilities of connected peers and modify behavior accordingly, and overall it has worked very well. Capability-based systems do work, and provide an excellent means of trying out new algorithms, adding new features for upgraded clients, and when necessary reverting protocol changes (by depreciating or removing extensions). The problem with OpenGL was and continues to be that the two superpowers of that industry develop and maintain competing proposals for similar functionality, which are thrust upon developers which must support both if they want access to the latest and greatest features, until such time that the ARB arbitrarily choses one to standardize upon (in the process creating yet another extension of the form ARB_* that may be different and must be explicitly supported by developers). I think the BitTorrent example shows that a loosely organized, open-source community *can* maintain a capability-based extension system without falling into capability-hell. Mark On Sun, Jun 17, 2012 at 9:30 AM, Amir Taaki wrote: > As the only person to have created and maintaining a full reimplementation > of the Bitcoin protocol/standard, I do think Bitcoin is filled with > arbitrary endianness and magic numbers. However it is a tiny and simple > protocol. > > The big problem is not implementing the Bitcoin protocol, but the fact > that once you have created a codebase, you want to perfect and fine-tune > the design. During the meantime, the Bitcoin protocol is being changed. > Change to the Bitcoin protocol is far more damaging to people that want to > implement the protocol than any issues with the current protocol. > > That's not to say, I disagree with changes to the protocol. I think > changes should be a lot more conservative and have a longer time frame than > they do currently. Usually changes suddenly get added to the Satoshi client > and I notice them in the commit log or announcements. Then it's like "oh I > have to add this" and I spend a week working to implement the change > without proper consideration or reflection which ends up with me having to > compromise on design choices. That is to remain compatible with the > protocol. > > However it is not my intent to slow down progress so I usually try to > hedge against that kind of feeling towards conservatism. > > > > - Original Message - > From: Jeff Garzik > To: Wladimir > Cc: bitcoin-development@lists.sourceforge.net > Sent: Sunday, June 17, 2012 5:19 PM > Subject: Re: [Bitcoin-development] Proposed new P2P command and response: > getcmds, cmdlist > > On Sat, Jun 16, 2012 at 4:42 AM, Wladimir wrote: > > Which is a perfectly reasonable requirement. However, one could simply > > standardize what a 'thin client' and what a 'thick client' does and > offers > > (at a certain version level), without having to explicitly enumerate > > everything over the protocol. > > > > This also makes it easier to deprecate (lack of) certain features later > on. > > You can simply drop support for protocol versions before a certain number > > (which has happened before). With the extension system this is much > harder, > > which likely means you keep certain workarounds forever. > > > > Letting the node know of each others capabilities at connection time > helps > > somewhat. It'd allow refusing clients that do not implement a certain > > feature. Then again, to me it's unclear what this wins compared to > > incremental protocol versions with clear requirements. > > > > I'm just afraid that the currently simple P2P protocol will turn into a > zoo > > of complicated (and potentially buggy/insecure) interactions. > > What is missing here is some perspective on the current situation. It > is -very- easy to make a protocol change and bump PROTOCOL_VERSION in > the Satoshi client. > > But for anyone maintaining a non-Satoshi codebase, the P2P protocol is > already filled with all sorts of magic numbers, arbitrarily versioned > binary data structures.. already an unfriendly zoo of complicated and > potentially buggy interactions. There is scant, incomplete > documentation on the wiki -- the Satoshi source code is really the > only true reference. > > I see these problems personally, trying to keep ArtForz' half-a-node > running on mainnet (distributed as 'blkmond' with pushpool). > > In an era of HTTP and JSON, NFS and iSCSI, bitcoin's P2P protocol is > woefully backwards, fragile, limited and inflexible when it comes to > parameter/extension exchange and negotiation. Even iSCSI, that which > is implemented on hard drive firmware, has the ability to exchange > key=value parameters between local and remote sides of the RPC > connection. > > Calling the curre
[Bitcoin-development] Block preview for faster relaying
Right now we're seeing cases where block propagation is sometimes taking minutes. This doesn't cause much of a problem for general Bitcoin users but for miners its problematic because it potentially increases the risk for orphaning. There are probably many contributing factors which can be improved here but one of the most obvious is that nodes fully validate blocks before relaying them. The validation is IO intensive and can currently take a minute alone on sufficiently slow nodes with sufficiently large blocks and larger blocks require more data to be transmitted. Because this slowness is proportional to the size of the block this risks creating mismatched incentives where miners are better off not mining (many) transactions in order to maximize their income. The validation speed can and should be improved but there is at least one short term improvement that can be made at the protocol level: Make it possible to relay blocks to other nodes before fully validating them. This can be reasonable secure because basic validation (such as the difficulty, previous block identity, and timestamps) can be done first so an attacker would need to burn enormous amounts of computing power just to make very modest trouble with it... and it's a change which would be beneficial even after any other performance improvements were made. Luke has been working on a patch for this: https://github.com/luke-jr/bitcoin/commit/0ce6f590dc2b9cbb46ceecd7320220f55d814bca One aspect of it that I wanted to see more comments on was the use of a new message for the preview-blocks instead of just announcing them like normal. The reason for this is two-fold: To prevent existing full nodes from blacklisting nodes sending a bad preview block due to the existing misbehavior checks, otherwise an attacker could burn one block to partition the network, and also so that SPV nodes which aren't able to fully validate the block themselves can opt-out or at least know that the data is not yet validated by the peer. I don't see any better way to address this but I thought other people might have comments. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
On Sun, Jun 17, 2012 at 7:04 PM, grarpamp wrote: > Presumably the github/0.6.2 branch is safe for production? 0.6.2 is very widely used, more so than the other acceptably updated backports. > What degree of caution about wallet eating should be > made for those using github/master? I can't speak for anyone but myself: I don't run master on wallets with large amounts of (non-testnet) coin in them, except for a few times when I needed access to this feature or that or just in a isolated capacity for testing. In any use with real wallets I'd be sure to have good backups that never touched the new code. We have at various times had bugs in master that would corrupt wallets (though IIRC not too severely) and have bugs that would burn coin both in mining and in transactions (though again, I think not too severely). My caution is not due to the risk being exceptionally great but just because there is probably no remedy if things go wrong, this caution is magnified by the fact that we don't currently have enough testing activity on master. Testnet exists so that people can test without fear of losing a lot of funds and with the 0.7.0(git master) testnet reboot it should be more usable than it has been. It would be very helpful if anyone offering bitcoin services would setup parallel toy versions of your sites on testnet— it would bring more attention to your real services, it would give you an opportunity to get more testing done of your real services, it would show some more commitment to software quality, and it would let you take a more active role in advancing bitcoin development by doing a little testing yourself that you couldn't do on your production systems. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
On Sunday, June 17, 2012 11:04:42 PM grarpamp wrote: > >> https://github.com/bitcoin/bitcoin > >> https://git.gitorious.org/bitcoin/bitcoind-stable > > > > The latter is Luke's backports of security and stability fixes to > > otherwise unmaintained old versions. > > Ah ok, coming from cvs/svn, it's a bit different to find things. > There's something to be said for maintenance of pior branches. > Though I see some things I can use in github and my work would > be more useful there, so maybe I'll stwitch to that from gitorius/0.6.x. > > Presumably the github/0.6.2 branch is safe for production? No, that was a temporary branch of what became the stable 0.6.x branch. GitHub/master is bleeding edge. For production, you usually want the stable branches/releases (which are on Gitorious). The fix to -detachdb's position in -help was just merged to master, and should be backported sometime in the next few days. Luke -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Ultimate Blockchain Compression w/ trust-free lite nodes
Hi Alberto, Your thread was part of the inspiration for the idea that I proposed. But as I read it more, I see that I originally misunderstood it (mistaking it for a simpler unspent-TxOut tree idea). Even after reading it, I'm not entirely clear how your proposal would work, but I see that you proposed something very similar. I just want to clarify that there are two, major orthogonal pieces to both proposals: (1) The method for creating unspent-TxOut-tree roots/fingerprints for verification (2) Using an alternate blockchain to maintain and distribute those fingerprints There are multiple ways to do both of those. You proposed a different tree structure (which I haven't entirely figured out, yet), and putting those "fingerprints" in the main chain header. In my proposal, (2) is to avoid inducing a blockchain fork, or even changing the protocol at all. By using a separate blockchain, it can be done non-disruptively, and could even be thrown out and re-worked if we were to find an issue with it later. The availability of merged mining makes it possible to get [almost] the same security as changing the protocol, but without the disruption of hard-forking. (I expect that if there's not too much computational overhead and the software is already written, most miners would sign on) I'll read into your page a little more. I don't want to take credit away from you, since you clearly had a comparable idea developed long before me :) -Alan On 06/17/2012 06:46 PM, Alberto Torres wrote: > Hi, > > I did describe a very similar thing back in January (also illustrated, > and, if I'm not mistaken, more simple and efficient to recalculate), > and I wanted to do a prototype, but I have been very busy with other > projects since then. > > https://en.bitcoin.it/wiki/User:DiThi/MTUT > > I just saw Gavin left a comment in the talk page, I'm sorry I haven't > seen it earlier. > > I think armory is the perfect client to implement such an idea. I sort > of waited it to be able to run in my laptop with 2 GB of RAM before > being sucked into other projects. I even lost track of its > development. > > I hope this gets developed. I will be able to help after summer if > this is still not done. > > DiThi > > P.S: Sorry Peter, I've sent you the message privately by mistake. > Also, I don't quite understand your concern of "unbalancing" the tree. > > 2012/6/17 Peter Todd: >> On Sun, Jun 17, 2012 at 02:39:28PM -0400, Alan Reiner wrote: >>> All, >>> >>> With the flurry of discussion about blockchain compression, I >>> thought it was time to put forward my final, most-advanced idea, >>> into a single, well-thought-out, *illustrated*, forum post. >>> Please check it out: https://bitcointalk.org/index.php?topic=88208.0 >>> >>> This is a huge undertaking, but it has some pretty huge benefits. >>> And it's actually feasible because it can be implemented without >>> disrupting the main network. I'm sure there's lots of issues with >>> it, but I'm putting it out there to see how it might be improved and >>> actually executed. >>> >>> >>> *Summary: >>> >>> */Use a special tree data structure to organize all unspent-TxOuts >>> on the network, and use the root of this tree to communicate its >>> "signature" between nodes. The leaves of this tree actually >>> correspond to addresses/scripts, and the data at the leaf is >>> actually a root of the unspent-TxOut list for that address/script. >>> To maintain security of the tree signatures, it will be included in >>> the header of an alternate blockchain, which will be secured by >>> merged mining. >>> >>> This provides the same compression as the simpler unspent-TxOut >>> merkle tree, but also gives nodes a way to download just the >>> unspent-TxOut list for each address in their wallet, and verify that >>> list directly against the blockheaders. Therefore, even lightweight >>> nodes can get full address information, from any untrusted peer, and >>> with only a tiny amount of downloaded data (a few kB). /* >> How are you going to prevent people from delibrately unbalancing the >> tree with addresses with chosen hashes? >> >> One idea that comes to mind, which unfortunately would make for a >> pseudo-network rule, is to simply say that any *new* address whose hash >> happens to be deeper in the tree than, say, 10*log(n), indicating it was >> probably chosen to be unbalanced, gets discarded. The "new address" part >> of the rule would be required, or else you could use the rule to get >> other people's addresses discarded. >> >> Having said that, such a rule just means that anyone playing games will >> find they can't spend *their* money, and only with pruning clients. >> Unrelated people will not be effected. The coins can also always be >> spent with a non-pruning client to an acceptable address, which can >> later re-spend on a pruning client. >> >> >> It also comes to mind is that with the popularity of firstbits it may be >> a good idea to use a comparison fu
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
>> https://github.com/bitcoin/bitcoin >> https://git.gitorious.org/bitcoin/bitcoind-stable > > The latter is Luke's backports of security and stability fixes to > otherwise unmaintained old versions. Ah ok, coming from cvs/svn, it's a bit different to find things. There's something to be said for maintenance of pior branches. Though I see some things I can use in github and my work would be more useful there, so maybe I'll stwitch to that from gitorius/0.6.x. Presumably the github/0.6.2 branch is safe for production? What degree of caution about wallet eating should be made for those using github/master? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Ultimate Blockchain Compression w/ trust-free lite nodes
Hi, I did describe a very similar thing back in January (also illustrated, and, if I'm not mistaken, more simple and efficient to recalculate), and I wanted to do a prototype, but I have been very busy with other projects since then. https://en.bitcoin.it/wiki/User:DiThi/MTUT I just saw Gavin left a comment in the talk page, I'm sorry I haven't seen it earlier. I think armory is the perfect client to implement such an idea. I sort of waited it to be able to run in my laptop with 2 GB of RAM before being sucked into other projects. I even lost track of its development. I hope this gets developed. I will be able to help after summer if this is still not done. DiThi P.S: Sorry Peter, I've sent you the message privately by mistake. Also, I don't quite understand your concern of "unbalancing" the tree. 2012/6/17 Peter Todd : > On Sun, Jun 17, 2012 at 02:39:28PM -0400, Alan Reiner wrote: >> All, >> >> With the flurry of discussion about blockchain compression, I >> thought it was time to put forward my final, most-advanced idea, >> into a single, well-thought-out, *illustrated*, forum post. >> Please check it out: https://bitcointalk.org/index.php?topic=88208.0 >> >> This is a huge undertaking, but it has some pretty huge benefits. >> And it's actually feasible because it can be implemented without >> disrupting the main network. I'm sure there's lots of issues with >> it, but I'm putting it out there to see how it might be improved and >> actually executed. >> >> >> *Summary: >> >> */Use a special tree data structure to organize all unspent-TxOuts >> on the network, and use the root of this tree to communicate its >> "signature" between nodes. The leaves of this tree actually >> correspond to addresses/scripts, and the data at the leaf is >> actually a root of the unspent-TxOut list for that address/script. >> To maintain security of the tree signatures, it will be included in >> the header of an alternate blockchain, which will be secured by >> merged mining. >> >> This provides the same compression as the simpler unspent-TxOut >> merkle tree, but also gives nodes a way to download just the >> unspent-TxOut list for each address in their wallet, and verify that >> list directly against the blockheaders. Therefore, even lightweight >> nodes can get full address information, from any untrusted peer, and >> with only a tiny amount of downloaded data (a few kB). /* > > How are you going to prevent people from delibrately unbalancing the > tree with addresses with chosen hashes? > > One idea that comes to mind, which unfortunately would make for a > pseudo-network rule, is to simply say that any *new* address whose hash > happens to be deeper in the tree than, say, 10*log(n), indicating it was > probably chosen to be unbalanced, gets discarded. The "new address" part > of the rule would be required, or else you could use the rule to get > other people's addresses discarded. > > Having said that, such a rule just means that anyone playing games will > find they can't spend *their* money, and only with pruning clients. > Unrelated people will not be effected. The coins can also always be > spent with a non-pruning client to an acceptable address, which can > later re-spend on a pruning client. > > > It also comes to mind is that with the popularity of firstbits it may be > a good idea to use a comparison function that works last bit first... > > > It's merkles all the way down... > > -- > 'peter'[:-1]@petertodd.org > > -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > ___ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
On Sun, Jun 17, 2012 at 5:35 PM, grarpamp wrote: >> It isn't inside the ifdef in bitcoin git master. > > Oh, hmm, well then, what is the difference or usage > between these two repositories in regards to the project? > Which one are the formal releases tagged (tbz's cut) in? > > Which one has the branches with the commits that will > make it into the next formal release? ie: tracking along > 0.5.x, 0.6.x, HEAD/master (to be branched for 0.7.x). > > https://github.com/bitcoin/bitcoin > https://git.gitorious.org/bitcoin/bitcoind-stable The latter is Luke's backports of security and stability fixes to otherwise unmaintained old versions. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
> It isn't inside the ifdef in bitcoin git master. Oh, hmm, well then, what is the difference or usage between these two repositories in regards to the project? Which one are the formal releases tagged (tbz's cut) in? Which one has the branches with the commits that will make it into the next formal release? ie: tracking along 0.5.x, 0.6.x, HEAD/master (to be branched for 0.7.x). https://github.com/bitcoin/bitcoin https://git.gitorious.org/bitcoin/bitcoind-stable I seem to be seeing more tags in the former, and more maintained branches in the latter? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Ultimate Blockchain Compression w/ trust-free lite nodes
On Sun, Jun 17, 2012 at 02:39:28PM -0400, Alan Reiner wrote: > All, > > With the flurry of discussion about blockchain compression, I > thought it was time to put forward my final, most-advanced idea, > into a single, well-thought-out, *illustrated*, forum post. > Please check it out: https://bitcointalk.org/index.php?topic=88208.0 > > This is a huge undertaking, but it has some pretty huge benefits. > And it's actually feasible because it can be implemented without > disrupting the main network. I'm sure there's lots of issues with > it, but I'm putting it out there to see how it might be improved and > actually executed. > > > *Summary: > > */Use a special tree data structure to organize all unspent-TxOuts > on the network, and use the root of this tree to communicate its > "signature" between nodes. The leaves of this tree actually > correspond to addresses/scripts, and the data at the leaf is > actually a root of the unspent-TxOut list for that address/script. > To maintain security of the tree signatures, it will be included in > the header of an alternate blockchain, which will be secured by > merged mining. > > This provides the same compression as the simpler unspent-TxOut > merkle tree, but also gives nodes a way to download just the > unspent-TxOut list for each address in their wallet, and verify that > list directly against the blockheaders. Therefore, even lightweight > nodes can get full address information, from any untrusted peer, and > with only a tiny amount of downloaded data (a few kB). /* How are you going to prevent people from delibrately unbalancing the tree with addresses with chosen hashes? One idea that comes to mind, which unfortunately would make for a pseudo-network rule, is to simply say that any *new* address whose hash happens to be deeper in the tree than, say, 10*log(n), indicating it was probably chosen to be unbalanced, gets discarded. The "new address" part of the rule would be required, or else you could use the rule to get other people's addresses discarded. Having said that, such a rule just means that anyone playing games will find they can't spend *their* money, and only with pruning clients. Unrelated people will not be effected. The coins can also always be spent with a non-pruning client to an acceptable address, which can later re-spend on a pruning client. It also comes to mind is that with the popularity of firstbits it may be a good idea to use a comparison function that works last bit first... It's merkles all the way down... -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Ultimate Blockchain Compression w/ trust-free lite nodes
All, With the flurry of discussion about blockchain compression, I thought it was time to put forward my final, most-advanced idea, into a single, well-thought-out, *illustrated*, forum post. Please check it out: https://bitcointalk.org/index.php?topic=88208.0 This is a huge undertaking, but it has some pretty huge benefits. And it's actually feasible because it can be implemented without disrupting the main network. I'm sure there's lots of issues with it, but I'm putting it out there to see how it might be improved and actually executed. *Summary: */Use a special tree data structure to organize all unspent-TxOuts on the network, and use the root of this tree to communicate its "signature" between nodes. The leaves of this tree actually correspond to addresses/scripts, and the data at the leaf is actually a root of the unspent-TxOut list for that address/script. To maintain security of the tree signatures, it will be included in the header of an alternate blockchain, which will be secured by merged mining. This provides the same compression as the simpler unspent-TxOut merkle tree, but also gives nodes a way to download just the unspent-TxOut list for each address in their wallet, and verify that list directly against the blockheaders. Therefore, even lightweight nodes can get full address information, from any untrusted peer, and with only a tiny amount of downloaded data (a few kB). /* * Alright, tear it up! -Alan -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposed new P2P command and response: getcmds, cmdlist
As the only person to have created and maintaining a full reimplementation of the Bitcoin protocol/standard, I do think Bitcoin is filled with arbitrary endianness and magic numbers. However it is a tiny and simple protocol. The big problem is not implementing the Bitcoin protocol, but the fact that once you have created a codebase, you want to perfect and fine-tune the design. During the meantime, the Bitcoin protocol is being changed. Change to the Bitcoin protocol is far more damaging to people that want to implement the protocol than any issues with the current protocol. That's not to say, I disagree with changes to the protocol. I think changes should be a lot more conservative and have a longer time frame than they do currently. Usually changes suddenly get added to the Satoshi client and I notice them in the commit log or announcements. Then it's like "oh I have to add this" and I spend a week working to implement the change without proper consideration or reflection which ends up with me having to compromise on design choices. That is to remain compatible with the protocol. However it is not my intent to slow down progress so I usually try to hedge against that kind of feeling towards conservatism. - Original Message - From: Jeff Garzik To: Wladimir Cc: bitcoin-development@lists.sourceforge.net Sent: Sunday, June 17, 2012 5:19 PM Subject: Re: [Bitcoin-development] Proposed new P2P command and response: getcmds, cmdlist On Sat, Jun 16, 2012 at 4:42 AM, Wladimir wrote: > Which is a perfectly reasonable requirement. However, one could simply > standardize what a 'thin client' and what a 'thick client' does and offers > (at a certain version level), without having to explicitly enumerate > everything over the protocol. > > This also makes it easier to deprecate (lack of) certain features later on. > You can simply drop support for protocol versions before a certain number > (which has happened before). With the extension system this is much harder, > which likely means you keep certain workarounds forever. > > Letting the node know of each others capabilities at connection time helps > somewhat. It'd allow refusing clients that do not implement a certain > feature. Then again, to me it's unclear what this wins compared to > incremental protocol versions with clear requirements. > > I'm just afraid that the currently simple P2P protocol will turn into a zoo > of complicated (and potentially buggy/insecure) interactions. What is missing here is some perspective on the current situation. It is -very- easy to make a protocol change and bump PROTOCOL_VERSION in the Satoshi client. But for anyone maintaining a non-Satoshi codebase, the P2P protocol is already filled with all sorts of magic numbers, arbitrarily versioned binary data structures.. already an unfriendly zoo of complicated and potentially buggy interactions. There is scant, incomplete documentation on the wiki -- the Satoshi source code is really the only true reference. I see these problems personally, trying to keep ArtForz' half-a-node running on mainnet (distributed as 'blkmond' with pushpool). In an era of HTTP and JSON, NFS and iSCSI, bitcoin's P2P protocol is woefully backwards, fragile, limited and inflexible when it comes to parameter/extension exchange and negotiation. Even iSCSI, that which is implemented on hard drive firmware, has the ability to exchange key=value parameters between local and remote sides of the RPC connection. Calling the current P2P protocol "simple" belies all the implementation details you absolutely -must- get right, to run on mainnet today. Satoshi client devs almost never see the fragility and complexity inherent in the current legacy codebase, built up over time. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposed new P2P command and response: getcmds, cmdlist
On Sat, Jun 16, 2012 at 4:42 AM, Wladimir wrote: > Which is a perfectly reasonable requirement. However, one could simply > standardize what a 'thin client' and what a 'thick client' does and offers > (at a certain version level), without having to explicitly enumerate > everything over the protocol. > > This also makes it easier to deprecate (lack of) certain features later on. > You can simply drop support for protocol versions before a certain number > (which has happened before). With the extension system this is much harder, > which likely means you keep certain workarounds forever. > > Letting the node know of each others capabilities at connection time helps > somewhat. It'd allow refusing clients that do not implement a certain > feature. Then again, to me it's unclear what this wins compared to > incremental protocol versions with clear requirements. > > I'm just afraid that the currently simple P2P protocol will turn into a zoo > of complicated (and potentially buggy/insecure) interactions. What is missing here is some perspective on the current situation. It is -very- easy to make a protocol change and bump PROTOCOL_VERSION in the Satoshi client. But for anyone maintaining a non-Satoshi codebase, the P2P protocol is already filled with all sorts of magic numbers, arbitrarily versioned binary data structures.. already an unfriendly zoo of complicated and potentially buggy interactions. There is scant, incomplete documentation on the wiki -- the Satoshi source code is really the only true reference. I see these problems personally, trying to keep ArtForz' half-a-node running on mainnet (distributed as 'blkmond' with pushpool). In an era of HTTP and JSON, NFS and iSCSI, bitcoin's P2P protocol is woefully backwards, fragile, limited and inflexible when it comes to parameter/extension exchange and negotiation. Even iSCSI, that which is implemented on hard drive firmware, has the ability to exchange key=value parameters between local and remote sides of the RPC connection. Calling the current P2P protocol "simple" belies all the implementation details you absolutely -must- get right, to run on mainnet today. Satoshi client devs almost never see the fragility and complexity inherent in the current legacy codebase, built up over time. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] After compressed pubkeys: hybrid pubkeys
On Sun, Jun 17, 2012 at 2:04 PM, Pieter Wuille wrote: > On Sun, Jun 17, 2012 at 01:01:12PM +0200, Mike Hearn wrote: > > > * 0x04 [32-byte X coord] [32-byte Y coord]: uncompressed format > > > * 0x06 [32-byte X coord] [32-byte Y coord]: hybrid format for even Y > coords > > > * 0x07 [32-byte X coord] [32-byte Y coord]: hybrid format for odd Y > coords > > > > So what's the actual difference in format? Is there any at all, or > > it's just the first number that's different? > > >From what I understand, that is indeed the only difference. > > To prevent surprises in the future, in case OpenSSL decides to add more, can we disable all other key formats in advance? Wladimir -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] After compressed pubkeys: hybrid pubkeys
On Sun, Jun 17, 2012 at 01:01:12PM +0200, Mike Hearn wrote: > > * 0x04 [32-byte X coord] [32-byte Y coord]: uncompressed format > > * 0x06 [32-byte X coord] [32-byte Y coord]: hybrid format for even Y coords > > * 0x07 [32-byte X coord] [32-byte Y coord]: hybrid format for odd Y coords > > So what's the actual difference in format? Is there any at all, or > it's just the first number that's different? >From what I understand, that is indeed the only difference. -- Pieter -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] After compressed pubkeys: hybrid pubkeys
> * 0x04 [32-byte X coord] [32-byte Y coord]: uncompressed format > * 0x06 [32-byte X coord] [32-byte Y coord]: hybrid format for even Y coords > * 0x07 [32-byte X coord] [32-byte Y coord]: hybrid format for odd Y coords So what's the actual difference in format? Is there any at all, or it's just the first number that's different? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.6.x - detachdb in wrong place
On Sun, Jun 17, 2012 at 5:22 AM, grarpamp wrote: > Well, detachdb doesn't appear in the -\? help > because it's stuffed under pnp, which is not set > in my build. please fix for people, tx :) It isn't inside the ifdef in bitcoin git master. (For future reference this sort of request is probably best opened as an issue in the github issue tracker instead of posted to the list). -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] 0.6.x - detachdb in wrong place
Well, detachdb doesn't appear in the -\? help
because it's stuffed under pnp, which is not set
in my build. please fix for people, tx :)
#ifdef USE_UPNP
#if USE_UPNP
" -upnp\t " + _("Use Universal Plug and
Play to map the listening port (default: 1)") + "\n" +
#else
" -upnp\t " + _("Use Universal Plug and
Play to map the listening port (default: 0)") + "\n" +
#endif
" -detachdb\t " + _("Detach block and address
databases. Increases shutdown time (default: 0)") + "\n" +
#endif
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
