Re: [Bitcoin-development] Long-term mining incentives
Hello Mike, Are you aware of my proposal for network assurance contracts? Yes I am aware of that; sorry for not mentioning it. I think it is an interesting proposal, but I would not rely on it today, because it includes a large share of unproven social experiment. (Bitcoin too is a social experiment, but so far it has been working) But I agree with Gavin that attempting to plan for 20 years from now is ambitious at best. Bitcoin might not even exist 20 years from now, or might be an abandoned backwater a la USENET. I agree with that, but I don't think it can be used as a way to justify how decisions are made today. The opposition to block size increase comes from two things: (1) The perceived risk of increased centralization. (2) Long-term considerations on the need for fee pressure. I believe you and Gavin have properly addressed (1). Concerning (2), I think the belief that miners can eventually be funded by a fee market is wishful thinking. Thus, I am not against the proposed block size increase. However, the issue of long-term mining incentives remains. So far, the only proven method to incentivize mining has been direct block reward. The easiest solution to ensure long-term viability of Bitcoin would be to put an end to the original block halving schedule, and to keep the block reward constant (this is what Monero does, btw). That solution is inflationary, but in practice, users happen to lose private keys all the time. The rate of coins loss would eventually converge to whatever rate of emission is chosen, because the care people take of their coins depends on their value. Another solution, that does not break Bitcoin's social contract, would be to keep the original block halving schedule, but to allow miners to also redeem lost coins (defined as: coins that have not moved for a fixed number of years. Some time averaging of the lost coins may be needed in order to prevent non-productive miner strategies). That solution would create less uncertainty on the actual money supply, and better acceptability. I do not expect such a solution to be adopted before miner incentives become a problem. Neither am I attempting to predict the future; a completely different solution might be found before the problem arises, or Bitcoin might stop to exist for some other reason. However, if I had to decide today, I would choose such a solution, instead of relying on completely unproven mechanisms. More important, since we need to decide about block size today, I want to make it clear that my support is motivated by that long-term possibility. I believe that the we will need fee pressure argument can reasonably be dismissed, not because we don't know how Bitcoin will work in 20 years, but because we know how it works today, and it is not thanks to fee pressure. Thomas -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Long-term mining incentives
Le 12/05/2015 18:10, Gavin Andresen a écrit : Added back the list, I didn't mean to reply privately: Fair enough, I'll try to find time in the next month or three to write up four plausible future scenarios for how mining incentives might work: 1) Fee-supported with very large blocks containing lots of tiny-fee transactions 2) Proof-of-idle supported (I wish Tadge Dryja would publish his proof-of-idle idea) 3) Fees purely as transaction-spam-prevention measure, chain security via alternative consensus algorithm (in this scenario there is very little mining). 4) Fee supported with small blocks containing high-fee transactions moving coins to/from sidechains. Would that be helpful, or do you have some reason for thinking that we should pick just one and focus all of our efforts on making that one scenario happen? I always think it is better, when possible, not to bet on one horse. Sorry if I did not make myself clear. It is not about betting on one single horse, or about making one particular scenario happen. It is not about predicting whether something else will replace PoW in the future, and I am in no way asking you to focus your efforts in one particular direction at the expenses of others. Various directions will be explored by various people, and that's great. I am talking about what we know today. I would like an answer to the following question: Do we have a reason to believe that Bitcoin can work in the long run, without involving technologies that have not been invented yet? Is there a single scenario that we know could work? Exotic and unproven technologies are not an answer to that question. The reference scenario should be as boring as possible, and as verifiable as possible. I am not asking what you think is the most likely to happen, but what is the most likely to work, given the knowledge we have today. If I was asking: Can we send humans to the moon by 2100?, I guess your answer would be: Yes we can, because it has been done in the past with chemical rockets, and we know how to build them. You would probably not use a space elevator in your answer. The reason I am asking that is, there seems to be no consensus among core developers on how Bitcoin can work without miner subsidy. How it *will* work is another question. -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Long-term mining incentives
Thank you for your answer. I agree that a lot of things will change, and I am not asking for a prediction of technological developments; prediction is certainly impossible. What I would like to have is some sort of reference scenario for the future of Bitcoin. Something a bit like the Standard Model in Physics. The reference scenario should not be a prediction of the future, that's not the point. In fact, it will have to be updated everytime technological evolutions or code changes render it obsolete. However, the reference scenario should be a workable path through the future, using today's technologies and today's knowlegde, and including all planned code changes. It should be, as much as possible, amenable to quantitative analysis. It could be used to justify controversial decisions such as a hard fork. Your proposal of a block size increase would be much stronger if it came with such a scenario. It would show that you know where you are going. Le 11/05/2015 19:29, Gavin Andresen a écrit : I think long-term the chain will not be secured purely by proof-of-work. I think when the Bitcoin network was tiny running solely on people's home computers proof-of-work was the right way to secure the chain, and the only fair way to both secure the chain and distribute the coins. See https://gist.github.com/gavinandresen/630d4a6c24ac6144482a for some half-baked thoughts along those lines. I don't think proof-of-work is the last word in distributed consensus (I also don't think any alternatives are anywhere near ready to deploy, but they might be in ten years). I also think it is premature to worry about what will happen in twenty or thirty years when the block subsidy is insignificant. A lot will happen in the next twenty years. I could spin a vision of what will secure the chain in twenty years, but I'd put a low probability on that vision actually turning out to be correct. That is why I keep saying Bitcoin is an experiment. But I also believe that the incentives are correct, and there are a lot of very motivated, smart, hard-working people who will make it work. When you're talking about trying to predict what will happen decades from now, I think that is the best you can (honestly) do. -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Long-term mining incentives
The discussion on block size increase has brought some attention to the other elephant in the room: Long-term mining incentives. Bitcoin derives its current market value from the assumption that a stable, steady-state regime will be reached in the future, where miners have an incentive to keep mining to protect the network. Such a steady state regime does not exist today, because miners get most of their reward from the block subsidy, which will progressively be removed. Thus, today's 3 billion USD question is the following: Will a steady state regime be reached in the future? Can such a regime exist? What are the necessary conditions for its existence? Satoshi's paper suggests that this may be achieved through miner fees. Quite a few people seem to take this for granted, and are working to make it happen (developing cpfp and replace-by-fee). This explains part of the opposition to raising the block size limit; some people would like to see some fee pressure building up first, in order to get closer to a regime where miners are incentivised by transaction fees instead of block subsidy. Indeed, the emergence of a working fee market would be extremely reassuring for the long-term viability of bitcoin. So, the thinking goes, by raising the block size limit, we would be postponing a crucial reality check. We would be buying time, at the expenses of Bitcoin's decentralization. OTOH, proponents of a block size increase have a very good point: if the block size is not raised soon, Bitcoin is going to enter a new, unknown and potentially harmful regime. In the current regime, almost all transaction get confirmed quickly, and fee pressure does not exist. Mike Hearn suggested that, when blocks reach full capacity and users start to experience confirmation delays and confirmation uncertainty, users will simply go away and stop using Bitcoin. To me, that outcome sounds very plausible indeed. Thus, proponents of the block size increase are conservative; they are trying to preserve the current regime, which is known to work, instead of letting the network enter uncharted territory. My problem is that this seems to lacks a vision. If the maximal block size is increased only to buy time, or because some people think that 7 tps is not enough to compete with VISA, then I guess it would be healthier to try and develop off-chain infrastructure first, such as the Lightning network. OTOH, I also fail to see evidence that a limited block capacity will lead to a functional fee market, able to sustain a steady state. A functional market requires well-informed participants who make rational choices and accept the outcomes of their choices. That is not the case today, and to believe that it will magically happen because blocks start to reach full capacity sounds a lot like like wishful thinking. So here is my question, to both proponents and opponents of a block size increase: What steady-state regime do you envision for Bitcoin, and what is is your plan to get there? More specifically, how will the steady-state regime look like? Will users experience fee pressure and delays, or will it look more like a scaled up version of what we enjoy today? Should fee pressure be increased jointly with subsidy decrease, or as soon as possible, or never? What incentives will exist for miners once the subsidy is gone? Will miners have an incentive to permanently fork off the last block and capture its fees? Do you expect Bitcoin to work because miners are altruistic/selfish/honest/caring? A clear vision would be welcome. -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposed alternatives to the 20MB step function
Le 11/05/2015 00:31, Mark Friedenbach a écrit : I'm on my phone today so I'm somewhat constrained in my reply, but the key takeaway is that the proposal is a mechanism for miners to trade subsidy for the increased fees of a larger block. Necessarily it only makes sense to do so when the marginal fee per KB exceeds the subsidy fee per KB. It correspondingly makes sense to use a smaller block size if fees are less than subsidy, but note that fees are not uniform and as the block shrinks the marginal fee rate goes up.. Oh I see, you expect the sign of the dE/dx to change depending on whether fees exceed the subsidy. This is possible, but instead of the linear identity, you have to increase the block size twice as fast as the difficulty. In that case we would get (using the notations of my previous email): D' = D(1+x) F' = F(1+2x) and thus: E' - E = x/(1+x)P(F-S) The presence of the (F-S) factor means that the sign reversal occurs when fees exceed subsidy. Limits on both the relative and absolute amount a miner can trade subsidy for block size prevent incentive edge cases as well as prevent a sharp shock to the current fee-poor economy (by disallowing adjustment below 1MB). Also the identity transform was used only for didactic purposes. I fully expect there to be other, more interesting functions to use. -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposed alternatives to the 20MB step function
Le 08/05/2015 22:33, Mark Friedenbach a écrit : * For each block, the miner is allowed to select a different difficulty (nBits) within a certain range, e.g. +/- 25% of the expected difficulty, and this miner-selected difficulty is used for the proof of work check. In addition to adjusting the hashcash target, selecting a different difficulty also raises or lowers the maximum block size for that block by a function of the difference in difficulty. So increasing the difficulty of the block by an additional 25% raises the block limit for that block from 100% of the current limit to 125%, and lowering the difficulty by 10% would also lower the maximum block size for that block from 100% to 90% of the current limit. For simplicity I will assume a linear identity transform as the function, but a quadratic or other function with compounding marginal cost may be preferred. Sorry but I fail to see how a linear identity transform between block size and difficulty would work. The miner's reward for finding a block is the sum of subsidy and fees: R = S + F The probability that the miner will find a block over a time interval is inversely proportional to the difficulty D: P = K / D where K is a constant that depends on the miner's hashrate. The expected reward of the miner is: E = P * R Consider that the miner chooses a new difficulty: D' = D(1 + x). With a linear identity transform between block size and difficulty, the miner will be allowed to collect fees from a block of size: S'=S(1+x) In the best case, collected will be proportional to block size: F' = F(1+x) Thus we get: E' = P' * R' = K/(D(1+x)) * (S + F(1+x)) E' = E - x/(1+x) * S * K / D So with this linear identity transform, increasing block size never increases the miners gain. As long as the subsidy exists, the best strategy for miners is to reduce block size (i.e. to choose x0). -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum 2.0 has been tagged
Hi Andreas, I don't think it's a problem that BIP43 is tied to BIP32. What I don't like is that you have to explore branches of the derivation tree, in order to know if there is a wallet. As a result, it is not possible for the software to give a negative answer, like this wallet is empty, because you do not know if you have explored all the possible derivations; a new one may have been added after the software was written. With a version number, you can answer sorry this seed is not recognized by me, and you do not need to be online to do that. If you are online, you can answer this wallet is empty after exploring it. Le 11/03/2015 16:31, Andreas Schildbach a écrit : Thanks Thomas, for sharing your experience! I'd like know why you think it's a problem that BIP43 is tied to BIP32? I understand we all agreed at least on the BIP32-derivation spec (excluding the BIP32-hierarchy spec)? -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum 2.0 has been tagged
Thanks Mike, and sorry to answer a bit late; it has been a busy couple of weeks. You are correct, a BIP39 seed phrase will not work in Electrum, and vice versa. It is indeed unfortunate. However, I believe BIP39 should not be followed, because it reproduces two mistakes I did when I designed the older Electrum seed system. Let me explain. The first problem I have with BIP39 is that the seed phrase does not include a version number. Wallet development is still in an exploratory phase, and we should expect even more innovation in this domain. In this context, it is unwise to make decisions that prevent future innovation. However, when we give a seed phrase to users, we have a moral obligation to keep supporting this seed phrase in future versions. We cannot simply announce to Electrum users that their old seed phrase is not supported anymore, because we created a new version of the software that uses a different derivation. This could lead to financial losses for users who are unaware of these technicalities. Well, at least, that is how I feel about it. BIP39 and Electrum v2 have a very different ways of handling future innovation. Electrum v2 seed phrases include an explicit version number, that indicates how the wallet addresses should be derived. In contrast, BIP39 seed phrases do not include a version number at all. BIP39 is meant to be combined with BIP43, which stipulates that the wallet structure should depend on the BIP32 derivation path used for the wallet (although BIP43 is not followed by all BIP39 compatible wallets). Thus, innovation in BIP43 is allowed only within the framework of BIP32. In addition, having to explore the branches of the BIP32 tree in order to determine the type of wallet attached to a seed might be somewhat inefficient. The second problem I see with BIP39 is that it requires a fixed wordlist. Of course, this forbids innovation in the wordlist itself, but that's not the main problem. When you write a new standard, it is important to keep this standard minimal, given the goal you want to achieve. I believe BIP39 could (and should) have been written without including the wordlist in the standard. There are two ways to derive a master key from a mnemonic phrase: 1. A bidirectional mapping between words and numbers, as in old Electrum versions. Pros: bidirectional means that you can do Shamir secret sharing of your seed. Cons: It requires a fixed wordlist. 2. Use a hash of the seed phrase (pbkdf). Pros: a fixed wordlist is not required. Cons: the mapping isn't bidirectional. Electrum v1 uses (1). Electrum v2 uses (2). Early versions of BIP39 used (1), and later they switched to (2). However, BIP39 uses (2) only in order to derive the wallet keys, not for its checksum. The BIP39 checksum uses (1), and it does requires a fixed wordlist. This is just plainly inconsistent. As a result, you have neither wordlist flexibility, nor Shamir secret sharing. Having a fixed wordlist is very unfortunate. First, it means that BIP39 will probably never leave the 'draft' stage, until all languages of the world have been added. Second, once you add a wordlist for a new language, you cannot change it anymore, because it will break existing seed phrases; therefore you have to be extremely careful in the way you design these wordlists. Third, languages often have words in common. When you add a new language to the list, you should not use words already used by existing wordlists, in order to ensure that the language can be detected. It leads to a first come first served situation, that might not be sustainable in the future. In order to support the old Electrum v1 seeds, all future versions of Electrum will have to include the old wordlist. In addition, when generating new seed phrases, Electrum now has to avoid collisions with old seed phrases, because the old ones did not have a version number. This is painful enough, I will not repeat the same errors twice. Electrum v2 derives both its private keys and its checksum/version number using a hash of the seed phrase. This means that wordlists can be added and modified in the future, without breaking existing seed phrases. It also means that it will be very easy for other wallets to support Electrum seedphrases: it requires about 20 lines of code, and no wordlist is required. Thomas Le 02/03/2015 16:37, Mike Hearn a écrit : Congrats Thomas! Glad to see Electrum 2 finally launch. * New seed derivation method (not compatible with BIP39). Does this mean a 12 words wallet created by Electrum won't work if imported into some other wallet that supports BIP39? Vice versa? This seems unfortunate. I guess if seeds are being represented with 12 words consistently, people will expect them to work everywhere. -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your
[Bitcoin-development] Electrum 2.0 has been tagged
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Bitcoin devs, I just tagged version 2.0 of Electrum: https://github.com/spesmilo/electrum/tree/2.0 The electrum.org website will be updated later today. The release notes are a bit dense, due to the large amount of changes and new features in this release. In the coming weeks we will be adding more detailed documentation to the wiki and to the website. There has been a very long hiatus in Electrum releases, because it took me a lot of time to decide about the new seed derivation method and wallet structure. Now that this part is done, I hope that we will resume to a faster release pace. I would like to thank all the people who contributed to this release, developers, beta testers, but also people from this list who provided useful feedback. Cheers, Thomas _ RELEASE-NOTES # Release 2.0 * Before you upgrade, make sure you have saved your wallet seed on paper. * Documentation is now hosted on a wiki: http://electrum.orain.org * New seed derivation method (not compatible with BIP39). The seed phrase includes a version number, that refers to the wallet structure. The version number also serves as a checksum, and it will prevent the import of seeds from incompatible wallets. Old Electrum seeds are still supported. * New address derivation (BIP32). Standard wallets are single account and use a gap limit of 20. * Support for Multisig wallets using parallel BIP32 derivations and P2SH addresses (2 of 2, 2 of 3). * Compact serialization format for unsigned or partially signed transactions, that includes the BIP32 master public key and derivation needed to sign inputs. Serialized transactions can be sent to cosigners or to cold storage using QR codes (using Andreas Schildbach's base 43 idea). * Support for BIP70 payment requests: - - Verification of the chain of signatures uses tlslite. - - In the GUI, payment requests are shown in the 'Invoices' tab. * Support for hardware wallets: Trezor (Satoshilabs) and Btchip (Ledger). * Two-factor authentication service by TrustedCoin. This service uses 2 of 3 multisig wallets and Google Authenticator. Note that wallets protected by this service can be deterministically restored from seed, without Trustedcoin's server. * Cosigner Pool plugin: encrypted communication channel for multisig wallets, to send and receive partially signed transactions. * Audio Modem plugin: send and receive transactions by sound. * OpenAlias plugin: send bitcoins to aliases verified using DNSSEC. * New 'Receive' tab in the GUI: - - create and manage payment requests, with QR Codes - - the former 'Receive' tab was renamed to 'Addresses' - - the former Point of Sale plugin is replaced by a resizeable window that pops up if you click on the QR code * The 'Send' tab in the Qt GUI supports transactions with multiple outputs, and raw hexadecimal scripts. * The GUI can connect to the Electrum daemon: electrum -d will start the daemon if it is not already running, and the GUI will connect to it. The daemon can serve several clients. It times out if no client uses if for more than 5 minutes. * The install wizard can be used to import addresses or private keys. A watching-only wallet is created by entering a list of addresses in the wizard dialog. * New file format: Wallets files are saved as JSON. Note that new wallet files cannot be read by older versions of Electrum. Old wallet files will be converted to the new format; this operation may take some time, because public keys will be derived for each address of your wallet. * The client accepts servers with a CA-signed SSL certificate. * ECIES encrypt/decrypt methods, availabe in the GUI and using the command line: encrypt pubkey message decrypt pubkey message * The Android GUI has received various updates and it is much more stable. Another script was added to Android, called Authenticator, that works completely offline: it reads an unsigned transaction shown as QR code, signs it and shows the result as a QR code. -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJU8y7fAAoJECvVgkt/lHDm78oP/2uIqCyOwLsAJGkAI3CPFxtw WssFJlnCnFiA4tPv5pd7HdOgxQkTaPbUHftexfdd/lpfmFvxZVoHcA/32IIKFH63 BU2bnEyYOaW1A4XfNDQH6VG7eT2er1HOlHCtIgzRl0KJNmVggU6DnXnHkUs1PVvg pyEIR7Xv3GiK7rcS4qCS/9COroqQGFOXJAiLnOaQP5KszT1bMUdoL7mBPTfavnla LM+2MgKJOWv+JpHQCDp3XwAXX62LLsS2BjdK1Jt6OpGA6IuVQGBSaTIn5K81S+Yh M6RDKbP3kObYQ+bzLvtWrzgUD3sdht/V8L5ZPS3+Jibvmhae2zRrm/YpJZ77Yjd4 7QliCFGH0+Gwle72yOempFGWULwq7p6yo4dVZXpj1G3XmbZXuvFg4jYeC/usCx+T kQgMBPWME2m80fCzhJew1pRChSs/lzVreB0Lh6Tm/5Pibmy721J4oUr6oLkaR9Uy NMrYqnSy0+tCEOXHrpCYhqogyzzdjOlv0gWKqB2uSkO5TkEHv2eyHeiZttAn11qO sb85q/k0kYQBZZEvKJ9022eyKHjejDhQjKsCVIHhb81BJ1QYnZFIxBiKkVMxf0u5 sT2TTi18eOrYCUGD2WJ+ALyI1zN1sHO0/sn5+XzlC0jg+1KUXoo0j8NYnzmHb0Yx 5lbdlcaw0Uo7iWkFdMYT =IGGP -END PGP SIGNATURE- -- Dive into the World of Parallel Programming The Go
Re: [Bitcoin-development] Plans to separate wallet from core
Le 24/06/2014 11:44, Wladimir a écrit : But IMO this is a passed stage. SPV wallets w/ Bloom filtering can work without any special servers, so why require a 'close binding' to a trusted bitcoin core? To clarify (and not piss off ThomasV :-): I do not think the idea of having servers with a reputation of their own is a passed stage. There are many things that cannot be done at SPV level security with just the P2P protocol yet. So having fewer but more trusted Electrum servers is a reasonable compromise. Thanks for that :) Note that my goal is to make the Electrum servers as trustless as possible, and not to rely on some sort of 'reputation'. Thomas -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure for P2SH multisig wallets
Le 26/04/2014 11:43, Mike Hearn a écrit : I'm not sure I understand why you need any special structure for this at all. The way I'd do it is just use regular HD wallets for everyone, of the regular form, and then swap the watching keys. Why do people need to be given a cosigner index at all, given that they all have unique root keys anyway? I agree with that. Perhaps the only thing that needs to be standardized is the order of public keys in the redeem script: I think they should be sorted, so that the p2sh address does not depend on the order of pubkeys. -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP32 wallet structure in use? Remove it?
I totally agree with gmaxwell here. The cost of interoperability is too high. It would force us to freeze all features, and to require a broad consensus everytime we want to add something new. In addition, some partial level of compatibility would probably lead to users not able to recover all their funds when they enter their seed in another wallet. That is not acceptable, and should be avoided. Le 25/04/2014 17:46, Gregory Maxwell a écrit : I don't believe that wallet interoperability at this level is possible in general except as an explicit compatibility feature. I also don't believe that it is a huge loss that it is so. The structure of the derivation defines and constrains functionality. You cannot be structure compatible unless you have the same features and behavior with respect to key management. To that extent that wallets have the same features, I agree its better if they are compatible— but unless they are dead software they likely won't keep the same features for long. Even if their key management were compatible there are many other things that go into making a wallet portable between systems; the handling of private keys is just one part: a complete wallet will have other (again, functionality specific) metadata. I agree that it would be it would be possible to support a compatibility mode where a wallet has just a subset of features which works when loaded into different systems, but I'm somewhat doubtful that it would be widely used. The decision to use that mode comes at the wrong time— when you start, not when you need the features you chose to disable or when you want to switch programs. But the obvious thing to do there is to just specify that a linear chain with no further branching is that mode: then that will be the same mode you use when someone gives you a master public key and asks you to use it for reoccurring changes— so at least the software will get used. Compatibility for something like a recovery tool is another matter, and BIP32 probably defines enough there that with a bit of extra data about how the real wallet worked that recovery can be successful. Calling it vendor lock in sounds overblown to me. If someone wants to change wallets they can transfer the funds— manual handling of private keys is seldom advisable, and as is they're going to lose their metadata in any case. No one expects to switch banks and to keep their account records at the new bank. And while less than perfect, the price of heavily constraining functionality in order to get another result is just too high. -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 24/04/2014 09:10, Pieter Wuille a écrit : To clarify: BIP64 has a much stricter definition for accounts than BIP32. In BIP32, it is not well specified what accounts are used for. They can be used for subwallets, receive accounts (as in bitcoind's account feature), recurring payments, part of a chain used as multisig addresses, ... determined individually for each index. In BIP64, they are strictly used for subwallets, and can't be used by anything else. yes, I saw that. In particular, bip64 stipulates that the wallet never mixes coins across different accounts. This is not what Electrum does currently. The UI allows you to chose between two modes: activate a single account (and the wallet will only use UTXOs from that acccount), or activate all accounts (and spend from all of them simultaneously). Concerning multisig addresses, I have changed my mind: Electrum will use parallel BIP32 trees. A wallet will not mix standard and multisig accounts. I think that is better in terms of UX. I agree with Mike Hearn's view that wallets with multiple accounts are probably too difficult to deal with for most users. If a user feels the need to have different accounting identities, they will probably create different wallet files, instead of creating bip32 subwallets. However, since multiple subwallets have been asked by many users, Electrum will propose them. But this should not be the default. More important, multiple accounts should never be required (in my previous implementation, they were required for multisig, because the wallet was creating multisig addresses in dedicated multisig accounts) Thomas -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 24/04/2014 09:21, Gregory Maxwell a écrit : It doesn't appear to me that reoccurring payments, receive accounts, multisig addresses, etc can be used with this proposal, but instead you must use a different purpose code and another BIP and are not compatible with the draft here. Am I misunderstanding it? Will Electrum be limiting itself in this way? I'd consider it a unfortunate loss of functionality if wallets couldn't implement reoccurring payment chains without making users generate entirely different wallets (which they couldn't share funds across) since addresses for recurring payments was one of the main motivations in BIP32. No, Electrum will not be limiting itself in this way. I believe that we are only at the beginning of exploring the different possibilities opened by HD wallets. It will probably take years until we have clear ideas on what users need, what choices they make, and how to organize everything. Therefore it is too early to take decisions that might limit future functionality. I can see that it is very difficult today to find a consensus on wallet structure between wallet developers. In addition, I changed my mind several times on these questions, so I guess I will probably need to change things again in the future. This is why I decided to include a version number in Electrum seeds. The version number will be updated everytime the wallet structure changes. I know many developers do not follow me on this, but that is something I am quite sure Electrum needs, despite all the other things I am not sure about :) I think it is too early to aim for inter-wallet compatibility today. I guess we should postpone this goal, and move on with software releases. As Andreas pointed out, we should just make sure that we do not import an incompatible seed in another wallet, because not recovering all your bitcoins would be a terrible user experience; the version number built in the seed will ensure that for Electrum. -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] bits: Unit of account
Let me make a sacrilegious proposal: keep using the name bitcoin, and shift the decimal point. There would be a short adaption period, where people will need to talk about new bitcoins and old bitcoins in order to disambiguate them. However, Bitcoin users are techies, so I don't think that the ambiguity will be a big issue. I don't think lots of people will mistakenly send 1000 times more than the amount they intended. The name bitcoin has a huge advantage over any other proposal, because it is already established. No marketing is needed. This kind of renaming has already taken place many times in history, because the currency was debased. Bitcoin would be the first time it happens in the other direction. -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based Open Source Intranet - Social, Extensible, Cloud Ready Get Started Now And Turn Your Intranet Into A Collaboration Platform http://p.sf.net/sfu/ExoPlatform ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
Le 09/04/2014 17:54, Gregory Maxwell a écrit : Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think that increasing the resource utilization of the node is a good way to go there for the purposes expressed here. (not that electrum couldn't be used here, but not unmodified without the resource usage increasing route) Electrum uses two large indexes: address - utxo (patricia tree, aka ultimate blockchain compression, see thread started by Alan Reiner in the bitcointalk forum) address - spent history The first index is not going to grow larger than what bitcoind already needs to store, because bitcoind will always need to store utxos. The second index threatens to become large. However, Electrum servers do not keep the full histories, they prune older entries. Without adapting Electrum clients, it would even be possible to keep only one bit per address (to know whether that address has been used or not), and that information is only used to restore wallets from seed, not during normal operations. If the first index (patricia tree) was implemented in bitcoind, that would obviously be a big relief for electrum servers. and that it might be an easier way to support SPV clients than creating a new API in bitcoind for it since Stratum itself already relies on bitcoind to provide it's services. Bitcoin's own P2P protocol is already the API for a ordinary SPV client. So I don't believe any new API would be require, except perhaps for some process management stuff (which also isn't provided for Electrum). -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
+1 I would prefer that solution... Le 08/04/2014 15:53, Pieter Wuille a écrit : I see the cause of our disagreement now. You actually want to share a single BIP32 tree across different currency types, but do it in a way that guarantees that they never use the same keys. I would have expected that different chains would use independent chains, and have serializations encode which chain they belong to. Let me offer an alternative suggestion, which is compatible with the original default BIP32 structure: * You can use one seed across different chains, but the master nodes are separate. * To derive the master node from the seed, the key string Bitcoin seed is replaced by something chain-specific. * Every encoded node (including master nodes) has a chain-specific serialization magic. This is in practice almost the same as your suggestion, except that the m/cointype' in m/cointype'/account'/change/n is replaced by different masters. The only disadvantage I see is that you do not have a way to encode the super master that is the parent of all chain-specific masters. You can - and with the same security properties - encode the seed, though. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 27/03/2014 00:37, Andreas Schildbach a écrit : Thanks for starting the discussion on finding a better structure. For me, the most important thing is either we're 100% interoperable or 0%. There should not be anything inbetween, as users will delete seeds without knowing there is still money in them on another implementation. I believe you have a good point here: we should not advertise wallets as compatible if they are not 100% compatible. One issue that I have is bandwidth: Electrum (and mycelium) cannot watch as many addresses as they want, because this will create too much traffic on the servers. (especially when servers send utxo merkle proofs for each address, which is not the case yet, but is planned) For this reason Electrum imposes a constraint on the number of virgin addresses provided to the user. Although the current strategy used by Electrum can certainly be improved, it will not scale up to having every client watching thousands of addresses. This constraint is not so important for bloom-filter clients. So I guess that it makes sense for Multibit to provide hundreds, or even thousands of virgin addresses to the user, regardless of how they are used. Such a wallet will in general not be recoverable in Electrum, unless the user helps the recovery procedure. (or the seed has metadata telling the software that this is a Multibit wallet). So we have a problem here, if we advertise these wallets as compatible. My opinion, as far as Electrum is concerned, is that merchant accounts should behave differently from regular user accounts: While merchants need to generate an unlimited number of receiving addresses, it is also acceptable for them to have a slightly more complex wallet recovery procedure (for example, the wallet might show an option to search for more addresses, and it might not need to watch old addresses anymore) OTOH, I don't think we can ask regular users to do this, not only because it adds complexity to the wallet recovery procedure (which makes it scarier), but also because we want fully automated synchronization between different instances of a wallet, using only no other source of information than the blockchain. The first versions of Electrum allowed users to set the gap limit parameter in their GUI preferences, but I removed it from GUI after I realized it was a bad idea (users messed with it and did not understand what happened..) With bloom filter clients I guess the distinction between these two use cases is not really necessary, because watching addresses is cheap. So it would be good to hear what you and Mike think about this problem. If you decide to let the user create hundreds of unused addresses (and I think it perfectly makes sense for you), then I guess it would be better for Electrum to give up on compatibility, rather than running the risk of seeing only a subset of addresses. Another option is to handle these seeds as merchant accounts. I heard from multiple sources that using this standard some wallets will only see a subset of the addresses/keys of some other wallets. Implementation differences can always happen (and should addresses as bugs), but I think its unacceptable that this source of issues is by design. I suggest we agree on an even simpler least common denominator and wallets that want to implement some feature on top of that can do but are encouraged to pick a totally different cointype. I guess that would mean removing reserved and account. I'm still thinking it might be a good idea to have a separate chain for refunds. Refunds will be rarely used and thus need a much slower moving window than receiving addresses or change. On 03/26/2014 09:49 PM, Mike Hearn wrote: Myself, Thomas V (Electrum) and Marek (Trezor) got together to make sure our BIP32 wallet structures would be compatible - and I discovered that only I was planning to use the default structure. Because I'm hopeful that we can get a lot of interoperability between wallets with regards to importing 12-words paper wallets, we brainstormed to find a structure acceptable to everyone and ended up with: /m/cointype/reserved'/account'/change/n The extra levels require some explanation: * cointype: This is zero for Bitcoin. This is here to support two things, one is supporting alt coins based off the same root seed. Right now nobody seemed very bothered about alt coins but sometimes feature requests do come in for this. Arguably there is no need and alt coins could just use the same keys as Bitcoin, but it may help avoid confusion if they don't. More usefully, cointype can distinguish between keys intended for things like multisig outputs, e.g. for watchdog services. This means if your wallet does not know about the extra protocol layers involved in this, it can still import the raw money and it will just ignore/not see the keys used in more complex transactions.
Re: [Bitcoin-development] New BIP32 structure
Le 26/03/2014 21:49, Mike Hearn a écrit : * reserved is for other stuff. I actually don't recall why we ended up with this. It may have been intended to split out multisig outputs etc from cointype. Marek, Thomas? yes, this was intended to create multisig addresses from the same seed. cointype was proposed after that. -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 27/03/2014 12:30, Marek Palatinus a écrit : Ah, I forget to two things, which should be into the BIP as well: a) Gap factor for addresses; as Thomas mentioned, although some software can watch almost unlimited amount of unused addresses, this is serious concern for lightweight or server-based wallets like Electrum or myTREZOR. myTREZOR currently uses gap factor 10, which is (from my experience so far) quite sane for most of users. Yes, I was planning to increase the number of available unused addresses to 10 or 20 in the bip32 version of Electrum. Related to this, here is another idea I would like to submit: Instead of using a gap limit (maximal number of consecutive unused addresses), I think we should get rid of the topology, and simply count the number of unused addresses since the beginning of the sequence. Indeed, the topology of the sequence of addresses is of no interest to the user. Users often misinterpret gap limit as the number of unused addresses available, so I think we should just give them what they want :) This is easier to understand, and it makes things more predictable, because the wallet will always display the same number of unused addresses (except when it is waiting for confirmations). -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 27/03/2014 12:39, Mike Hearn a écrit : One issue that I have is bandwidth: Electrum (and mycelium) cannot watch as many addresses as they want, because this will create too much traffic on the servers. (especially when servers send utxo merkle proofs for each address, which is not the case yet, but is planned) This is surprising and the first time I've heard about this. Surely your constraint is CPU or disk seeks? Addresses are small, I find it hard to believe that clients uploading them is a big drain, and mostly addresses that are in the lookahead region won't have any hits and so won't result in any downloads? To be honest, I have not carried out a comprehensive examination of server performance. What I can see is that Electrum servers are often slowed down when a wallet with a large number (thousands) of addresses shows up, and this is caused by disk seeks (especially on my slow VPS). The master branch of electrum-server is also quite wasteful in terms of CPU, because it uses client threads. I have another branch that uses a socket poller, but that branch is not widely deployed yet. I reckon that I might have been a bit too conservative, in setting the number of unused receiving addresses watched by Electrum clients (until now, the default gap limit has always been 5). The reason is that, if I increase that number, then there is no way to go back to a smaller value, because it needs to be compatible with all previously released versions. However, Electrum servers performance has improved over time, so I guess it could safely be raised to 20 (see previous post to slush). In terms of bandwidth, I am referring to my Android version of Electrum. When it runs on a 3G connection, it sometimes takes up to 1 minute to synchronize (with a wallet that has hundreds of addresses). However, I have not checked if this was caused by addresses or block headers. This constraint is not so important for bloom-filter clients. Bloom filters are a neat way to encode addresses and keys but they don't magically let clients save bandwidth. A smaller filter results in less upload bandwidth but more download (from the wallets perspective). So I'm worried if you think this will be an issue for your clients: I haven't investigated bandwidth usage deeply yet, perhaps I should. FWIW the current bitcoinj HDW alpha preview pre-gens 100 addresses on both receive and change branches. But I'm not sure what the right setting is. Heh, may I suggest 20 in the receive branch? For the change branch, there is no need to watch a large number of unused addresses, because the wallet should try to fill all the gaps in the sequence of change. (Electrum does that. It also watches 3 unused addresses at the end of that sequence, in order to cope with possible blockchain reorgs causing gaps. As an extra safety, it also waits for 3 confirmations before using a new change address, which sometimes results in address reuse, but I guess a smarter strategy could avoid that). We also have to consider latency. The simplest implementation from a wallets POV is to step through each transaction in the block chain one at a time, and each time you see an address that is yours, calculate the next ones in the chain. But that would be fantastically slow, so we must instead pre-generate a larger lookahead region and request more data in one batch. Then you have to recover if that batch ends up using all the pre-genned addresses. It's just painful. My opinion, as far as Electrum is concerned, is that merchant accounts should behave differently from regular user accounts: While merchants need to generate an unlimited number of receiving addresses, it is also acceptable for them to have a slightly more complex wallet recovery procedure Maybe. I dislike any distinction between users and merchants though. I don't think it's really safe to assume merchants are more sophisticated than end users. well, it depends what we mean by merchant. I was thinking more of a website running a script, rather than a brick and mortar ice cream seller. :) but also because we want fully automated synchronization between different instances of a wallet, using only no other source of information than the blockchain. I think such synchronization won't be possible as we keep adding features, because the block chain cannot sync all the relevant data. For instance Electrum already has a label sync feature. Other wallets need to compete with that, somehow, so we need to build a way to do cross-device wallet sync with non-chain data. Oh, I was not referring to label sync, but only to the synchronization of the list of addresses in the wallet. Label sync is an Electrum plugin that relies on a centralized server. Using a third party server is acceptable in that case, IMO, because you will not lose your coins if the server fails.
Re: [Bitcoin-development] New BIP32 structure
Le 27/03/2014 13:49, Mike Hearn a écrit : Ah, BIP32 allows for a range of entropy sizes and it so happens that they picked 256 bits instead of 128 bits. I'd have thought that there is a right answer for this. 2^128 should not be brute forceable, and longer sizes have a cost in terms of making the seeds harder to write down on paper. So should this be a degree of freedom? Here is what I understand: 2^128 iterations is not brute forcable today, and will not be for the foreseeable future. An EC pubkey of length n can be forced in approximately 2^(n/2) iterations (see http://ecc-challenge.info/) Thus, Bitcoin pubkeys, which are 256 bits, would require 2^128 iterations. This is why unused addresses (160 bits hash) are better protected than already used ones. However, people tend to believe that a public key of size n requires 2^n iterations. This belief might have been spread by this popular image: https://bitcointalk.org/index.php?topic=508880.msg5616146#msg5616146 -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New BIP32 structure
Le 27/03/2014 14:44, Pavol Rusnak a écrit : On 03/27/2014 01:06 PM, Thomas Voegtlin wrote: Yes, I was planning to increase the number of available unused addresses to 10 or 20 in the bip32 version of Electrum. Also, we'd need to specify a gap limit for accounts as well. In TREZOR we currently use 0, which means that the scan will stop as soon as we hit first account with no transaction history (meaning that its first X=10 addresses are unused). I agree with that. I was planning to do the same (no gap) Note: Maybe we could just look at the first address of each account, instead of the first 10 ? -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Electrum 1.9.8 release
I am happy to announce the release of Electrum 1.9.8. This release includes some features initially planned for version 2.0. Packages are available on https://electrum.org/download.html (signed by me) Binaries for windows and mac will be available in the coming days enjoy Thomas --- RELEASE NOTES # Release 1.9.8 (This release includes features initially planned for version 2.0) * Electrum servers were upgraded to version 0.9. The new server stores a Patrica tree of all UTXOs, an idea proposed by Alan Reiner in the bitcointalk forum. This property allows the client to directly request the balance of any address. The new commands are: 1. getaddressbalance address 2. getaddressunspent address 3. getutxoaddress txid pos * In addition, two commands for message encryption were added: 1. encrypt pubkey message 2. decrypt pubkey message The encryption algorithm is ECIES, and code was was borrowed from https://github.com/jackjack-jj/jeeq. In order to know the public key corresponding to a Bitcoin address in your wallet, you can use the 'getpubkeys' command. The 'decrypt' command assumes that the wallet has the private key corresponding to the public key passed as argument. * The encrypt and decrypt functions are available in the Qt GUI (from the menubar, or right click on one of your addresses if you want to use its public key). * Command-line commands that require a connection to the network spawn a daemon, that remains connected and handles subsequent commands. The daemon terminates itself if it remains unused for more than one minute. The purpose of this is to make scripting more efficient. For example, a bash script using many electrum commands will open only one connection. -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum 1.9.8 release
thanks for your feedback! I was not aware that that implementation was flawed. I will see how I can fix that code and get back to you. Thomas Le 16/03/2014 14:54, Gregory Maxwell a écrit : On Sun, Mar 16, 2014 at 6:24 AM, Thomas Voegtlin thoma...@gmx.de wrote: The encryption algorithm is ECIES, and code was was borrowed from https://github.com/jackjack-jj/jeeq. In order to know the public key corresponding to a Bitcoin address in your wallet, you can use the 'getpubkeys' command. The 'decrypt' command assumes that the wallet has the private key corresponding to the public key passed as argument. The cryptosystem in that repository appears to be insecure in several ways and is not actually implementing ECIES. The most important of which is that instead of using a cryptographically strong mac tied to the ephemeral secret it uses a trivial 16 bit check value. This means that that I can decode an arbitrary message encrypted to a third person if they allow me to make no more than 65536 queries to a decryption oracle to decrypt some other message. Also, in the event that a random query to a decryption oracle yields a result (1:2^16 times) the result directly reveals the ECDH value because it is only additively combined with the message value. If the implementation does not check if the nonce point is on the curve (an easy implementation mistake) the result can yield a point on the twist instead of the curve which is far more vulnerable to recovery of the private key. ECIES uses a KDF instead of using the ECDH result directly to avoid this. There may be other problems (or mitigating factors) as it was very hard for me to follow what it was actually doing. (The particular implementation has a number of other issues, like apparently not using a cryptographically strong RNG for its EC nonce.. but I assume you didn't copy that particular flaw) -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum 1.9.8 release
Thanks again for having a look. Given these problems, I have decided to remove the encryption methods from the current release. I retagged 1.9.8 and updated the packages. Thomas Le 16/03/2014 15:39, Gregory Maxwell a écrit : On Sun, Mar 16, 2014 at 7:31 AM, Thomas Voegtlin thoma...@gmx.de wrote: thanks for your feedback! I was not aware that that implementation was flawed. I will see how I can fix that code and get back to you. It also leaks on the order of 7 bits of data about the message per message chunk. I'm also think it's likely that there are some messages which are just incorrectly decrypted. ... it's really screwy and suspect. -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Multisign payment protocol?
Trezor and Electrum may be earlier than this. Sorry for not joining the discussion earlier. I have postponed the release of bip32 features in Electrum due to ongoing discussions with Trezor and bitcoinj developers. I planned to post a summary in a separate thread, but this info is also relevant for this thread, so I'm posting here. (sorry if this is a bit offtopic, though) I plan to create a 2-factor authentication service that uses p2sh addresses in Electrum. All addresses are derived from the wallet root seed, and should be recoverable from it. (of course this departs from scenarios where master keys are generated independently; my opinion is that both should be possible) So, when the user activates 2fa protection, the root private key is deleted from their hard drive, as well as the master private key of one of the branches used to create p2sh addresses (which is sent to a remote server). See this (fairly old) description here for more details: https://bitcointalk.org/index.php?topic=274182.0 Since I still want to be able to generate 1of1 accounts after the 2fa protection is activated, 1of 1 accounts should not be generated directly from the root of the tree. Thus, an extra level must be inserted in the tree. For example, 1of1 addresses can be derived as follows: m/reserved'/n' where n is the account index, and reserved is an index that indicates the type of address. (0 would be reserved for 1of1 addresses) slush suggested that another layer of derivation would be useful, in order to use wallets with altcoins on the same seed. This lead to this type of derivation: m/coin'/reserved'/n' where coin would be 0 for Bitcoin, and reserved would be 0 for 1of1 addresses Thomas -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039: Final call
Le 24/01/2014 10:05, Peter Todd a écrit : On Tue, Jan 21, 2014 at 01:00:43AM +0100, Thomas Voegtlin wrote: Hi slush, Thank you for your new proposal; it seems to be a compromise. @Christophe Biocca: If the wordlist becomes part of the standard, then we will run into problems of collisions once users ask for wordlists in every language. IMO the right approach is to implement checksums that do not depend on the wordlist (eg the 'brute force' method, Hash(mnemonic||1) mod 2^k == 0 ) this would also allow us to implement sipa's variable stretching proposal. I understand this is not possible because of the computational requirements of devices such as trezor. Is it? Surely the trezor can bruteforce, say, 8 bits == 0. How many SHA256/sec can the trezor hardware do? Generating your seed is a one-time thing after all - that taking 10-30s doesn't seem like a big deal to me. Even a 1/256th checksum will really cut down on the number of mistakes made and money lost. slush, correct me if I'm wrong, but I don't think that's the only reason: They want to generate a seed by combining entropy from the trezor device and from the user's computer; In addition, they want the computer to be able to check that the seed actually was derived from the entropy it provided, using only a master public key (the computer does not have access to the seed) This is why they designed bip39 that way. I think the new bip39 proposal could be used in Electrum as an option for trezor, but I am reluctant to make it default, because it imposes its own dictionary. -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039: Final call
Hi slush, Thank you for your new proposal; it seems to be a compromise. @Christophe Biocca: If the wordlist becomes part of the standard, then we will run into problems of collisions once users ask for wordlists in every language. IMO the right approach is to implement checksums that do not depend on the wordlist (eg the 'brute force' method, Hash(mnemonic||1) mod 2^k == 0 ) this would also allow us to implement sipa's variable stretching proposal. I understand this is not possible because of the computational requirements of devices such as trezor. I am leaning toward considering these devices as a nonstandard case, instead of enforcing a given wordlist in the standard. Thomas Le 21/01/2014 00:18, slush a écrit : On Tue, Jan 21, 2014 at 12:06 AM, Christophe Biocca christophe.bio...@gmail.com mailto:christophe.bio...@gmail.com wrote: I remember the wordlist choice getting bikeshedded to death a month ago. I would just include the wordlist as part of the standard (as a recommendation) so that fully compliant implementations can correct a user's typos regardless of the original generator. That's exactly our attitude. We realized that have a community-wide agreement on the wordlist itself is simply imposible, so to reach at least some consensus we split the proposal to two parts - one what is essential to call itself a bip39 compatible, i.e. converting the mnemonic to bip32 node and second which is optional, including our proposed wordlist, which has some advanced features like checksums etc. Now it is up to client developers to decide if they really insist on their superior wordlist or if they'll implement checksums following the full specification. Those who don't like it will have to deal with the compatibility concerns themselves, or get an alternate wordlist approved as a BIP. Odds are no one will go that route. At least Trezor and bitcoinj (Multibit) seems to be going in this way, which is 100% of clients which expressed interest in bip39 :-). slush -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP proposal: Authenticated prefix trees
Le 07/01/2014 01:21, Mark Friedenbach a écrit : -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/06/2014 10:13 AM, Peter Todd wrote: On Sun, Jan 05, 2014 at 07:43:58PM +0100, Thomas Voegtlin wrote: I have written a Python-levelDB implementation of this UTXO hashtree, which is currently being tested, and will be added to Electrum servers. Along the lines of my recent post on blockchain data: Is it going to be possible to do partial prefix queries on that tree? There's really two tree structures being talked about here. Correct me if I'm wrong Thomas, but last time I looked at your code it was still implementing a 256-way PATRICIA trie, correct? This structure lends itself to indexing either scriptPubKey or H(scriptPubKey) with approximately the same performance characteristics, and in the Ultimate blockchain compression thread there is much debate about which to use. You are right. The 256-way branching follows from the fact that the tree was implemented using a key-value database operating with byte strings (leveldb). With this implementation constraint, a different branching would probably be possible but wasteful. My recent code creates one leaf per unspent, and uses 56-byte keys built as: H(scriptPubKey) + txid + txpos (This is not pushed yet, it needs cleanup. Previous code created one leaf per address) Partial prefix queries are possible with database iterators. In the process of experimentation I've since moved from a 256-way PATRICIA trie to a bitwise, non-level-compressed trie structure - what I call proof-updatable trees in the BIP. These have the advantage of allowing stateless application of one proof to another, and as consequence enable mining mempool operations without access to the UTXO set, so long as proofs are initially provided in the transaction block wire format. I see the advantage of doing that, but this looks really far-fetched.. My understanding is that it would require a complete change in the way clients and miners work. Could such a change be brought iteratively? -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP proposal: Authenticated prefix trees
Hello and happy new year to this mailing list! Thank you Mark for the incredible work you've been doing on this. I am following this very closely, because it is of primary importance for Electrum. I have written a Python-levelDB implementation of this UTXO hashtree, which is currently being tested, and will be added to Electrum servers. My implementation follows Alan Reiner's idea to store the tree as items in a key-value database. I believe that a C++ implementation like yours will be at least an order of magnitude faster, and I am looking forward to it. I too believe that BIPs should define interoperability points, but probably not implementation details. For the UTXO hashtree, this means that a BIP should at least specify how the root hash is constructed. This might be the only thing that needs to be specified. However, I see no pressing issue with writing a BIP; it might be preferable to implement and test different options first, and learn from that. Thomas Le 20/12/2013 02:47, Mark Friedenbach a écrit : -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello fellow bitcoin developers. Included below is the first draft of a BIP for a new Merkle-compressed data structure. The need for this data structure arose out of the misnamed Ultimate blockchain compression project, but it has since been recognized to have many other applications. In addition to this BIP I am preparing three additional BIPs describing the use of this data structure in stateless validation mining, the UBC address index for SPV+ operating modes, document timestamping and merged mining. A Python implementation of this data structure is available here: https://github.com/monetizeio/python-bitcoin A C++ implementation is being worked on. As per the BIP-1 procedure, I am submitting this rough draft to the community for discussion. I welcome all comments and criticisms of both form and content. - -Mark ==Abstract== This BIP describes a [http://en.wikipedia.org/wiki/Hash_tree Merkle hash tree] variant of the [http://en.wikipedia.org/wiki/Trie prefix-tree data structure], ideally suited for encoding key-value indices which support memory-efficient proofs. ==Motivation== There are a number of applications which would benefit from having a data structure with the following properties: * '''Arbitrary mapping of keys to values.''' A ''key'' can be any bytestring, and its ''value'' any other bytestring. * '''Duplicate keys disallowed.''' Every key has one, and only one value associated with it. Some applications demand assurance that no key value is reused, and that this constraint can be checked without requiring access to the entire data structure. * '''Efficient look-up by key.''' The data structure should support sub-linear lookup operations with respect to the number of keys in the mapping. Logarithmic time or linear with respect to the length of the key should be achievable and would be sufficient for realistic applications. * '''Merkle compression of mapping structure.''' It should be possible to produce a reduced description of the tree consisting of a single root hash value which is deterministically calculated from the mapping structure. * '''Efficient proofs of inclusion.''' It should be possible to extract a proof of key/value mapping which is limited in size and verification time by the length of the key in the worst case. * '''Computation of updates using local information.''' Given a set of inclusion proofs, it should be possible to calculate adjustments to the local mapping structure (update or deletion of included mappings, or insertion between two included mappings which are adjacent in the global structure). Such applications include committed validation indices which enable stateless mining nodes, committed wallet indices which enable trust-less querying of the unspent transaction output set by codescriptPubKey/code, efficient document time-stamping, and secure efficient merged mining. This BIP describes an authenticated prefix tree which has the above properties, but leaves the myriad applications to be formalized in future BIPs. ==Data structure== This BIP defines a binary prefix tree. Such a structure provides a mapping of bitstrings (the ''keys'') to bytestrings (the ''values''). It is an acyclic binary tree which implicitly encodes keys within the traversal path -- a left branch is a 0, and a right branch is a 1. Each node is reachable by only one unique path, and reading off the branches taken (0 for each left, 1 for each right) as one follows the path from root to target yields the node's key. The particular binary prefix tree defined by this BIP is a hybrid PATRICIA / de la Brandais tree structure. [http://en.wikipedia.org/wiki/Radix_tree PATRICIA trees] compress a long sequence of non-branching nodes into a single interior node with a per-branch ''skip prefix''. This achieves significant savings in storage space, root
Re: [Bitcoin-development] Proposal to replace BIP0039
Le 03/11/2013 07:41, Timo Hanke a écrit : No. You mean the computer would use B for this check? (k,K) could be rigged by Trezor, who computes b as k-a. Timo I was just asking a question, in order to understand how this device works, and what are its requirements. if you think you can help, please explain. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
Le 03/11/2013 08:40, Timo Hanke a écrit : I think the communication would have to go the other way around. Trezor has to commit to a value First. Like this: Trezor picks random s and sends S=s*G to computer, keeping s secret. Computer picks random t and sends t to Trezor. Trezor makes r := s+t its internal master private key with corresponding master public key R := (s+t)*G. Since R = S+t*G, the computer can verify the master public key. As you say, the computer can then store R and can later verify for each derived pubkey that it was indeed derived from R, hence from his own entropy t. I'm not sure how this differs from what I wrote... However, if this is how it works, then my question remains: The computer has no proof to know that pubkeys derived through bip32's private derivations are derived from its own entropy... This verification would only work for public (aka type2) derivations. .. but maybe Trezor works in a different way? I think an explanation from slush would be needed. However, Trezor could not use straight bip32 out of the box. The chaincode would have to be something like SHA(R). And the seed (that gets translated to mnemonic) would be r itself, making it 256 bit instead of only 128 bit. If the longer seed is bearable then this is a good way to do it. One question remains: if you only write down the mnemonic how can you be sure that it is correct and corresponds to the secret in Trezor? You cannot verify that on paper. You would have to restore it on some device, eg another empty Trezor, and see if it brings up the same master pubkey. Right? I guess you have to trust Trezor that it derives R from r -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
Le 31/10/2013 12:18, slush a écrit : Oh, I forgot to one practical aspect; the way how the mnemonic is mined in Thomas proposal prevents usage in embedded devices, because difficulty of generating proper mnemonic is simply too high for embedded microcontrollers. Maybe this can be solved somehow by modifying the proposal, but right now it is a showstopper for us. even if metadata is only 8 bits ? (that's about 256 hashes) -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
Indeed, I want to include a version number in the seed phrase because there are multiple ways to define the tree structure used with BIP32. It is certainly too early to make final decisions on that, or to achieve a common standard. Also, I can imagine that bip32 itself might be superseeded in the future. I understand that encapsulating a version number in the seed phrase might not be as important for other wallets as it is for Electrum. So it is probably not necessary to propose another BIP for that. I will simply implement it for Electrum, and I will try to do it in such a way that other wallets can use the same format. The other question we might be solving is strenghtening (your proposal). I consider that this is not a strong requirement for Electrum, because it does not let the user choose their seed phrase. However, if a few bits of the seed phrase are allocated for metadata, then I guess strenghtening can be part of it. That's another good reason to have a version number encapsulated in the seed. I too wonder why the transformation needs to be bidirectional in bip39. Le 26/10/2013 23:30, Pieter Wuille a écrit : Let's first try to agree on what we are solving. It seems that Thomas wants - in addition to the cryptographic data - to encode the tree structure, or at least version information about what features are used in it, inside the seed. I'm not sure whether we're ready to standardize on something like that yet, not having established best practices regarding different wallet structures. I do think we first need to see what possibilities and developments are possible related to it. In addition, information about the wallet structure is strictly less secret than the key data - it is even less confidential than address book data, transaction annotations, labels and comments and bookkeeping information. It could be backed up anywhere and everywhere without any repercussions, as far as I can see. I understand that in the case of Electrum, there is a strong reason to want this encapsulated together with the seed, but it's not really a requirement for most wallets. (if really needed, any key derivation scheme that starts from random strings can be augmented with metadata by enforcing property-bits on a hash of the string (so not of the output), as this data doesn't need protection from brute-forcing). Regarding other requirements, I wonder why we want the transformation to be bidirectional? If it is just about generating master seeds, one direction should be enough, and allows far nicer properties w.r.t. security. If we do settle on a standard for 'brainwallets', I would strongly prefer if it had at least some strengthening built-in, to decrease the impact of worst-case situations. If the reason is backward-compatibility, I think that any client that supports seeds already can just keep supporting whatever they supported before. Only if it matches both encoding schemes (as mentioned before) there is a potential collision (and in that case, the user could just be asked). -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
slush wrote : Two years ago I proposed exactly this and you refused to add extra information to mnemonic, because it isn't necessary and it makes it longer to mnemonization. What changed since then? I was wrong, and I fully acknowledge it. My concern was that adding extra information would make the mnemonic longer than 12 words. In addition, you proposed to allocate these extra bits for a checksum, not for metadata. However, a checksum does not really add any information, because Electrum checks the existence of a wallet directly from the blockchain. So, my feeling at that time was that adding extra bits would increase the risks (a longer seed is harder to memorize, increases the probability of mistakes, etc), and did not bring any real benefit. However, you showed since then how to solve this by using a slightly longer dictionary, and I do like your solution, I find it absolutely brilliant. In addition, I realize now that metadata (ie a version number) is crucially needed, for the reasons mentioned in my previous post. Hm, what exactly do you need to store about wallet structure? I lived in opinion that everything is able to recover using CKD function to generate new addresses and blockchain lookups for their balances. BIP32 gives a lot of freedom to wallet developers: it does not specify which branches of the HD tree shall be used for which purpose. However, if you want to recover a wallet from its mnemonic (a requirement for Electrum), then you need to know which branches to explore. In Electrum 1.9 I had to make some choices about branch allocation. However, the decisions that I made are certainly not final, so it is important to be able to change them in the future. Thus, this metadata needs to be added to the mnemonic. Yes, that's true. It isn't possible to make everybody 100% happy. At least I wanted to be constructive and asked you to replace the most problematic words. No pull request from you so far. The solution I propose is very different from BIP39, and it does not require to predefine a dictionary. My proposal is actually somewhat similar to Pieter Wuille's proposal, which I discovered after his recent post. ( https://bitcointalk.org/index.php?topic=102349.0 ) Yes, it was original idea. So far I don't think this is a problem. Of course some words may have some meaning across languages, but it should be easy to avoid them. There are tens of thousands words in every language and we need to pick only 2048 words to wordlist. ... Are your worries about overlapping words across languages a real issue? No, there are not so many words that are frequent enough. Overlapping will be an issue, especially if we go for a 4096 words dictionary. If I understand this well, it is basically one-way algorithm mnemonic - seed, right? Seed cannot be printed out as mnemonic, because there's hashing involved, but the bi-directionality has been the original requirement for such algorithm (at least in Electrum and bip39). You are right, this encoding is not symmetric. Bi-directionality has never been a requirement for Electrum. May I ask why you need bi-directionality in Trezor? (the only reason I can think of is if you want to export a bip32 branch into another wallet, but this would create a very long mnemonic string) Then, how is this different to picking 12 random words from dictionary and hashing them together? I don't see any benefit in that mining part of the proposal (except that it is lowering the entropy for given length of mnemonic). it makes it possible to hash a utf8 string, and to retrieve the metadata from the hash. Thus we don't need to spend ages arguing about the best choice of a dictionary, and to set it in stone. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
