Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
On 23/10/13 23:07, Pieter Wuille wrote: In short, consistency is more important than correctness. That's a nice and concise way to put it and any potential protocol documentation should start with a statement like that. However, I do not think that making it hard to find information about the details of the system is the way to go. Alternate implementations are likely inevitable, and in the long run probably a win for the ecosystem. If effort is put into accurately describing the rules, it should indeed carry a strong notice about it being descriptive rather than normative. One interesting question is whather alternative implementations are more likely to get it wrong because the protocol description is wrong or because the authors misunderstood the reference implementation source code. Extensive documentation of the source code, a la Knuth's literate programming, may be some kind of a middle ground. If someone is willing to work on that, I am (and likely many people in #bitcoin-dev are) available for any questions about the protocol and its semantics. Ok. Several people expressed an interest in the topic, so I'll give it a try and see how it fares. Martin -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Core Development Update #5
https://bitcoinfoundation.org/blog/?p=290 Very excited about this, particularly the 80 bytes embeddable message. I do believe satoshi mentioned he wanted to add short messages, at some point. Great work Gavin all! -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
Yes. I had pointed people in IRC to Knuth's literate programming, as an example of how we might document bitcoin. On Thu, Oct 24, 2013 at 3:03 AM, Martin Sustrik sust...@250bpm.com wrote: On 23/10/13 23:07, Pieter Wuille wrote: In short, consistency is more important than correctness. That's a nice and concise way to put it and any potential protocol documentation should start with a statement like that. However, I do not think that making it hard to find information about the details of the system is the way to go. Alternate implementations are likely inevitable, and in the long run probably a win for the ecosystem. If effort is put into accurately describing the rules, it should indeed carry a strong notice about it being descriptive rather than normative. One interesting question is whather alternative implementations are more likely to get it wrong because the protocol description is wrong or because the authors misunderstood the reference implementation source code. Extensive documentation of the source code, a la Knuth's literate programming, may be some kind of a middle ground. If someone is willing to work on that, I am (and likely many people in #bitcoin-dev are) available for any questions about the protocol and its semantics. Ok. Several people expressed an interest in the topic, so I'll give it a try and see how it fares. Martin -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
I'd like to add some historical background about how the protocol specification came to be in the first place. A bit over three years [1] ago I started an attempt to document the network protocol, by reverse engineering it from the satoshi client. My goal, back then, was to enable like-minded engineers to create alternative clients and move away from the client-monoculture that is still predominant today. It was clear from the beginning that it would merely be a reverse engineering effort, and that it would likely lag a bit behind the changes in the main client. It was meant as a help for engineers that are not well versed in C/C++ to enable them to contribute by creating new clients, but the satoshi client would always be the de-facto standard. With the move from Google Code to the Bitcoin.it wiki somehow this notion of it being a reverse engineering effort was lost and people started assuming that if the behavior of the satoshi client did not match the protocol description it was a bug on the client side. Instead it is because the reverse engineering of the protocol is incorrect or simply missing some details. Although the protocol description is far more complete than it was back when we started, I still don't feel comfortable giving it the name specification. I still believe that a client monoculture is bad for the system as a whole, because a single bug might bring down the whole network. Giving people the necessary tools to implement new clients brings stability. I do understand the criticism that writing a specification might hinder future development as it restricts the possible changes to the protocol, but isn't this already the case as long as we have legacy versions of the client participating in the network? I would also argue that having a specification allows an application independent review of the protocol to identify possible improvements and bugs. I think the protocol description has an important place in the development of Bitcoin, so much so that we pushed a long time ago to separate protocol version from the client version. I would love to see the protocol specification becoming official part of the bitcoin github repository, which would ideally be maintained alongside the satoshi client to keep it up to date. Regards, Christian Decker [1] https://bitcointalk.org/index.php?topic=231 -- Christian Decker On Thu, Oct 24, 2013 at 9:03 AM, Martin Sustrik sust...@250bpm.com wrote: On 23/10/13 23:07, Pieter Wuille wrote: In short, consistency is more important than correctness. That's a nice and concise way to put it and any potential protocol documentation should start with a statement like that. However, I do not think that making it hard to find information about the details of the system is the way to go. Alternate implementations are likely inevitable, and in the long run probably a win for the ecosystem. If effort is put into accurately describing the rules, it should indeed carry a strong notice about it being descriptive rather than normative. One interesting question is whather alternative implementations are more likely to get it wrong because the protocol description is wrong or because the authors misunderstood the reference implementation source code. Extensive documentation of the source code, a la Knuth's literate programming, may be some kind of a middle ground. If someone is willing to work on that, I am (and likely many people in #bitcoin-dev are) available for any questions about the protocol and its semantics. Ok. Several people expressed an interest in the topic, so I'll give it a try and see how it fares. Martin -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP39 word list
I've just pushed updated wordlist which is filtered to similar characters
taken from this matrix.
BIP39 now consider following character pairs as similar:
similar = (
('a', 'c'), ('a', 'e'), ('a', 'o'),
('b', 'd'), ('b', 'h'), ('b', 'p'), ('b', 'q'), ('b', 'r'),
('c', 'e'), ('c', 'g'), ('c', 'n'), ('c', 'o'), ('c', 'q'),
('c', 'u'),
('d', 'g'), ('d', 'h'), ('d', 'o'), ('d', 'p'), ('d', 'q'),
('e', 'f'), ('e', 'o'),
('f', 'i'), ('f', 'j'), ('f', 'l'), ('f', 'p'), ('f', 't'),
('g', 'j'), ('g', 'o'), ('g', 'p'), ('g', 'q'), ('g', 'y'),
('h', 'k'), ('h', 'l'), ('h', 'm'), ('h', 'n'), ('h', 'r'),
('i', 'j'), ('i', 'l'), ('i', 't'), ('i', 'y'),
('j', 'l'), ('j', 'p'), ('j', 'q'), ('j', 'y'),
('k', 'x'),
('l', 't'),
('m', 'n'), ('m', 'w'),
('n', 'u'), ('n', 'z'),
('o', 'p'), ('o', 'q'), ('o', 'u'), ('o', 'v'),
('p', 'q'), ('p', 'r'),
('q', 'y'),
('s', 'z'),
('u', 'v'), ('u', 'w'), ('u', 'y'),
('v', 'w'), ('v', 'y')
)
Feel free to review and comment current wordlist, but I think we're slowly
moving forward final list.
slush
On Sat, Oct 19, 2013 at 1:58 AM, Gregory Maxwell gmaxw...@gmail.com wrote:
some fairly old wordlist solver code of mine:
https://people.xiph.org/~greg/wordlist.visual.py
it has a 52x52 letter visual similarity matrix in it (along with a
citation)
On Fri, Oct 18, 2013 at 4:52 PM, jan jan.mare...@gmail.com wrote:
The words 'public', 'private' and 'secret' could be confusing when
encoding public and private keys. eg. a private key that begins with
the word 'public'.
I think avoiding words that could look similar when written down would
be a good idea aswell. I searched for words that only differ by the
letters c e, g y, u v and found the following:
car ear
cat eat
gear year
value valve
Other combinations could potentially be problematic depending on the
handwriting style: ft, ao, ij, vy, possibly even lt and il?
I've included the search utility I used below.
#include stdbool.h
#include string.h
#include stdio.h
char *similar_char_pairs[] = { ce, gy, uv, NULL };
bool is_similar_char(char c1, char c2)
{
char **pairs = similar_char_pairs;
do {
char *p = *pairs;
if ((c1 == p[0] c2 == p[1]) ||
(c1 == p[1] c2 == p[0]))
return true;
} while (*++pairs);
return false;
}
bool print_words_if_similar(char *word1, char *word2)
{
/* reject words of different lengths */
if (strlen(word1) != strlen(word2))
return false;
size_t i, similarcount = 0;
for (i = 0; i strlen(word1); i++) {
/* skip identical letters */
if (word1[i] == word2[i])
continue;
/* reject words that don't match */
if (is_similar_char(word1[i], word2[i]) == false)
return false;
similarcount++;
}
/* reject words with more than 1 different letter */
//if (similarcount 1)
// return false;
printf(%s %s\n, word1, word2);
return true;
}
int main(void)
{
/* english.txt is assumed to exist in the working directory
download from:
https://github.com/trezor/python-mnemonic/blob/master/mnemonic/wordlist/english.txt*/
FILE* f = fopen(english.txt, r);
if (!f) {
fprintf(stderr, failed to open english.txt\n);
return 1;
}
/* read in word list, assumes one word per line */
#define MAXWORD 16
char wordlist[2048][MAXWORD];
int word = 0;
while (fgets(wordlist[word], MAXWORD, f)) {
/* strip trailing whitespace, assumes no leading whitespace */
char *ch = strpbrk(wordlist[word], \n\t);
if (ch)
*ch = '\0';
word++;
}
if (word != 2048) {
fprintf(stderr, word list incorrect length\n);
return 1;
}
/* check each word for similarity against every other word */
int i, j, count = 0;
for (i = 0; i 2048; i++) {
for (j = i+1; j 2048; j++) {
if (print_words_if_similar(wordlist[i], wordlist[j]))
count++;
}
}
printf(%d matches\n, count);
return 0;
}
--
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
from
the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Fix available for startup issues for git HEAD nodes
DisconnectBlock() : outputs still spent? database corrupted If you can't get your node up and running and see the above in your debug.log there's a potential fix available: https://github.com/bitcoin/bitcoin/pull/3143 Be warned: highly rushed and poorly tested, so you're best to ask a dev in IRC for more details. -- 'peter'[:-1]@petertodd.org bf7bcf3da1b3b228216b72fefccbed84becaaba6fcc6aff2 signature.asc Description: Digital signature -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Thu, Oct 24, 2013 at 4:30 PM, Peter Todd p...@petertodd.org wrote: Quick thought on how to make blockchain-based fee estimates work better in the context of out-of-band mining contracts: have miners advertise in their coinbase's what fees were actually paid, as opposed to appear to have been paid. This is interesting, but I suppose some miners may have business models that can't be easily summed up as a fee - like all-you-can-eat deals with certain providers, or preference to certain kinds of transactions etc. For the concern that estimation might force fees down too far if miners include private transactions, I thought the estimates were calculated only on broadcast transactions, so transactions that just appear in a block won't ever influence the estimate? -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Thu, Oct 24, 2013 at 04:46:41PM +0200, Mike Hearn wrote: Well, miners are all supposed to be more or less equivalent - modulo differences in tx acceptance policies - so I'd hope that having out of bad fee mechanisms yet still broadcasting the TX isn't that common. If it was broadcasted, it should get mined in short order, otherwise things are going wrong. Eligius has contracts to do transaction mining, and it's currently 10% of the hashing power. As I said elsewhere, a good use-case for OOB fee payment is for merchants who use the payment protocol, and want to get their customers transactions mined as efficiently and cheaply as possible. (child-pays-for-parent has more blockchain bloat and thus extra expense) On Thu, Oct 24, 2013 at 4:43 PM, Peter Todd p...@petertodd.org wrote: Anyway, in what circumstance would a customer want an exclusive contract with a miner? I was thinking for transactions that aren't standard so have to be submitted to miners directly. Sure, but even then there's no harm in letting more than one miner know about it. There's even an existing form of this: P2Pool lets shares be accompanied by up to 50KB worth of transactions of any form. -- 'peter'[:-1]@petertodd.org 000d2860c825ea223b805c60a33b26b9b70616698033d360b066 signature.asc Description: Digital signature -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Proposal to replace BIP0039
I would like to propose a new BIP, that replaces BIP0039. My initial problem was that BIP0039 is not backward compatible with Electrum. When trying to solve that, I realized that the seed encoding used in Electrum does not help, because it does not contain a version number information. However, BIP0039 suffers the same shortcoming: it does nothing to help a future replacement, it wants to be final. My first recommendation is to allocate a few bits of the mnemonic, in order to encode a version number along with the checksum bits. The second problem is the wallet structure. There are multiple ways to use a BIP32 tree, and each client will certainly handle this differently. For Electrum, it is important to be able to recover an entire wallet from its mnemonic, using no extra information. Thus, the client needs to know which branches of the BIP32 tree are populated by default. This means that the version number I mentioned will not only be about the seed encoding, but it should also give some information about the wallet structure, at least in the case of Electrum. The third problem is the dictionary. I do not like the dictionary proposed in BIP0039, because it contains too many short words, which are bad for memorization (I explained here how I designed the dictionary used by Electrum: https://bitcointalk.org/index.php?topic=153990.msg2167909#msg2167909). I had some discussions with slush about this, but I do not think it will ever be possible to find a consensus on that topic. BIP0039 also suggests to use localized dictionaries, with non-colliding word lists, but it is not clear how that will be achieved; it seems to be difficult, because languages often have words in common. It looks like a first-come-first-served aproach will be used. For these reasons, I believe that we need a dictionary-independent solution. This will allow developers to use the dictionary they like, and localization will be easy. I would like to suggest the following solution: 1. Define a target of k bits: this target contains the metadata (version number), plus some extra bits for the checksum. For example, with k=16, we can allocate 8 bits for the version number, and 8 bits for checksum. 2. Pick a random number of length n+k bits, where n is the desired entropy of the seed, and k is the number of bits needed for the metadata (checksum, version number) 3. Translate this random number to a mnemonic string, using a dictionary. 4. Compute a hash of the mnemonic string (utf8 encoded). 5. Repeat steps 2, 3 and 4 until the k first bits of the hash are equal to the target defined in 1. 6. Use the final hash as input for bip32 (as the master seed) This means that we mine for the seed, until the desired metadata is obtained in the hash. This mining also adds a bit of difficulty to the process of finding a seed (on average, it will require 2^k iterations). The entropy of the final hash is n, the number of unconstrained bits. This solution makes it possible for developers to define new dictionaries, localized or adapted to a particular need. The resulting mnemonics will always be usable with other clients, even if they do not know the dictionary. I am willing to write a new BIP where this proposal is specified in detail. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
On Thu, Oct 24, 2013 at 7:29 PM, thoma...@gmx.de wrote: My initial problem was that BIP0039 is not backward compatible with Electrum. When trying to solve that, I realized that the seed encoding used in Electrum does not help, because it does not contain a version number information. However, BIP0039 suffers the same shortcoming: it does nothing to help a future replacement, it wants to be final. My first recommendation is to allocate a few bits of the mnemonic, in order to encode a version number along with the checksum bits. Two years ago I proposed exactly this and you refused to add extra information to mnemonic, because it isn't necessary and it makes it longer to mnemonization. What changed since then? The second problem is the wallet structure. There are multiple ways to use a BIP32 tree, and each client will certainly handle this differently. For Electrum, it is important to be able to recover an entire wallet from its mnemonic, using no extra information. Thus, the client needs to know which branches of the BIP32 tree are populated by default. This means that the version number I mentioned will not only be about the seed encoding, but it should also give some information about the wallet structure, at least in the case of Electrum. Hm, what exactly do you need to store about wallet structure? I lived in opinion that everything is able to recover using CKD function to generate new addresses and blockchain lookups for their balances. The third problem is the dictionary. I do not like the dictionary proposed in BIP0039, because it contains too many short words, which are bad for memorization (I explained here how I designed the dictionary used by Electrum: https://bitcointalk.org/index.php?topic=153990.msg2167909#msg2167909). I had some discussions with slush about this, but I do not think it will ever be possible to find a consensus on that topic. Yes, that's true. It isn't possible to make everybody 100% happy. At least I wanted to be constructive and asked you to replace the most problematic words. No pull request from you so far. BIP0039 also suggests to use localized dictionaries, with non-colliding word lists, but it is not clear how that will be achieved; it seems to be difficult, because languages often have words in common. It looks like a first-come-first-served aproach will be used. Yes, it was original idea. So far I don't think this is a problem. Of course some words may have some meaning across languages, but it should be easy to avoid them. There are tens of thousands words in every language and we need to pick only 2048 words to wordlist. For these reasons, I believe that we need a dictionary-independent solution. This will allow developers to use the dictionary they like, and localization will be easy. I would like to suggest the following solution: If I understand this well, it is basically one-way algorithm mnemonic - seed, right? Seed cannot be printed out as mnemonic, because there's hashing involved, but the bi-directionality has been the original requirement for such algorithm (at least in Electrum and bip39). Then, how is this different to picking 12 random words from dictionary and hashing them together? I don't see any benefit in that mining part of the proposal (except that it is lowering the entropy for given length of mnemonic). This solution makes it possible for developers to define new dictionaries, localized or adapted to a particular need. Are your worries about overlapping words across languages a real issue? slush -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039 Mnemonic code for generating deterministic keys
We've reflected many comments about BIP39 wordlist from the community and I think the wordlist is much better now. Specifically we removed many of theoretically offensive words as well as we implemented algorithm for detecting words with similar characters (cat/eat) and we resolved these duplicities. I'm now quite happy with the wordlist and I want to ask you for next (final?) round of comments. From other features, we added password protection of seed and seed hardening (against bruteforcing) using Rijndael cipher. This has been chosen because its blocksize can be 128, 192 or 256 bits, so it fits length of desired seeds. Also there are Rijndael implementations in every language. Btw password protection has one interesting feature - plausible deniability. It allows user to have one mnemonic and by using it with different passwords, it will generate different BIP32 wallets (wink wink) I want to be pretty clear that we need to close this topic somehow, because we want to use such algorithm in Trezor (which deadline is coming quick) and also other wallet developers want to implement such algorithm into clients to be compatible with Trezor. There were quite strict requirements for such algorithm (like the possibility to convert mnemonic to seed as well as seed to mnemonic) and I think we found a good solution. I'm wildly asking you for constructive comments, but saying it's a crap, I don't like it won't help anything. Thanks, slush On Thu, Sep 12, 2013 at 6:02 PM, Matthew Mitchell matthewmitch...@godofgod.co.uk wrote: I removed some more but I haven't added enough back in. It was taking far longer than expected so I gave up, but maybe someone else can try to add some more: https://github.com/MatthewLM/python-mnemonic/blob/master/mnemonic/wordlist/english.txt On 12 Sep 2013, at 13:11, Pavol Rusnak st...@gk2.sk wrote: On 10/09/13 23:03, Matthew Mitchell wrote: Maybe it would have been better without the aggressive words? I revisited the wordlist and replaced around 67 words that can be found offensive in some context. -- Best Regards / S pozdravom, Pavol Rusnak st...@gk2.sk -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039 Mnemonic code for generating deterministic keys
This is probably too late in the discussion, and I certainly don't want to derail any standard being formed. But if it is controversial, I want to offer my own suggestion. This is a proposal I wrote a year ago, but never spent enough work to push it as a standard: https://bitcointalk.org/index.php?topic=102349.0 It needs some work, but I believe it may be a base for a superior system than what is being proposed here. As the scheme linked above has built-in configurable difficulty and checksums, the word set being used doesn't need to function for checking anymore. You could use any dictionary/language/text generator, and feed it into the system - the software on the other side doesn't need to use the same dictionary. The disadvantage is of course that it cannot encode arbitrary data - it can only be used to generate a random seed. It does have some theoretical advantages, though (see link). -- Pieter On Thu, Oct 24, 2013 at 8:26 PM, slush sl...@centrum.cz wrote: We've reflected many comments about BIP39 wordlist from the community and I think the wordlist is much better now. Specifically we removed many of theoretically offensive words as well as we implemented algorithm for detecting words with similar characters (cat/eat) and we resolved these duplicities. I'm now quite happy with the wordlist and I want to ask you for next (final?) round of comments. From other features, we added password protection of seed and seed hardening (against bruteforcing) using Rijndael cipher. This has been chosen because its blocksize can be 128, 192 or 256 bits, so it fits length of desired seeds. Also there are Rijndael implementations in every language. Btw password protection has one interesting feature - plausible deniability. It allows user to have one mnemonic and by using it with different passwords, it will generate different BIP32 wallets (wink wink) I want to be pretty clear that we need to close this topic somehow, because we want to use such algorithm in Trezor (which deadline is coming quick) and also other wallet developers want to implement such algorithm into clients to be compatible with Trezor. There were quite strict requirements for such algorithm (like the possibility to convert mnemonic to seed as well as seed to mnemonic) and I think we found a good solution. I'm wildly asking you for constructive comments, but saying it's a crap, I don't like it won't help anything. Thanks, slush On Thu, Sep 12, 2013 at 6:02 PM, Matthew Mitchell matthewmitch...@godofgod.co.uk wrote: I removed some more but I haven't added enough back in. It was taking far longer than expected so I gave up, but maybe someone else can try to add some more: https://github.com/MatthewLM/python-mnemonic/blob/master/mnemonic/wordlist/english.txt On 12 Sep 2013, at 13:11, Pavol Rusnak st...@gk2.sk wrote: On 10/09/13 23:03, Matthew Mitchell wrote: Maybe it would have been better without the aggressive words? I revisited the wordlist and replaced around 67 words that can be found offensive in some context. -- Best Regards / S pozdravom, Pavol Rusnak st...@gk2.sk -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039 Mnemonic code for generating deterministic keys
This will probably sound stupid to most of you, but I'll say it anyway. The aim of mnemonics is to easily remember, isn't it? But the approach of removing offensive words is probably counterproductive to achieving that end. These words cause a greater emotional impact in our human moral psyches. If we were willing to use that fact in our advantage to optimize the maximum unforgettableness criterion, we should actually prefer the most generally offensive words in that list. Specially if they can combine with each other to produce more offensive results, basically the opposite of what we're doing. Isn't legalize murder dirty jew much easier to remember for most people than sandwich house yellow cauliflower? I guess that even if I'm right, this will be hard to explain to users and I'm not offering myself to do it. So I completely understand if the people working on this BIP simply ignore this unforgettable wordlist proposal like if it was just a bad taste joke. Using the sub-optimal (in terms of human memory) politically correct wordlist probably won't be that much worse. On 10/24/13, slush sl...@centrum.cz wrote: We've reflected many comments about BIP39 wordlist from the community and I think the wordlist is much better now. Specifically we removed many of theoretically offensive words as well as we implemented algorithm for detecting words with similar characters (cat/eat) and we resolved these duplicities. I'm now quite happy with the wordlist and I want to ask you for next (final?) round of comments. From other features, we added password protection of seed and seed hardening (against bruteforcing) using Rijndael cipher. This has been chosen because its blocksize can be 128, 192 or 256 bits, so it fits length of desired seeds. Also there are Rijndael implementations in every language. Btw password protection has one interesting feature - plausible deniability. It allows user to have one mnemonic and by using it with different passwords, it will generate different BIP32 wallets (wink wink) I want to be pretty clear that we need to close this topic somehow, because we want to use such algorithm in Trezor (which deadline is coming quick) and also other wallet developers want to implement such algorithm into clients to be compatible with Trezor. There were quite strict requirements for such algorithm (like the possibility to convert mnemonic to seed as well as seed to mnemonic) and I think we found a good solution. I'm wildly asking you for constructive comments, but saying it's a crap, I don't like it won't help anything. Thanks, slush On Thu, Sep 12, 2013 at 6:02 PM, Matthew Mitchell matthewmitch...@godofgod.co.uk wrote: I removed some more but I haven't added enough back in. It was taking far longer than expected so I gave up, but maybe someone else can try to add some more: https://github.com/MatthewLM/python-mnemonic/blob/master/mnemonic/wordlist/english.txt On 12 Sep 2013, at 13:11, Pavol Rusnak st...@gk2.sk wrote: On 10/09/13 23:03, Matthew Mitchell wrote: Maybe it would have been better without the aggressive words? I revisited the wordlist and replaced around 67 words that can be found offensive in some context. -- Best Regards / S pozdravom, Pavol Rusnak st...@gk2.sk -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=5127iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jorge Timón http://freico.in/ -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk
Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
Thanks Christian, this is a really interesting bit of history. My own personal experience from when I wrote my own client and BCCAPI-ish server was that the protocol specification on the Wiki was hugely valuable, and rarely sent me astray. Supplement that with the occasional questions on #bitcoin-dev, and then just coding, coding, coding and getting unit tests to pass. Nothing compares (IMO) to stepping through your own code watching the unit tests run, scripts evaluate, calculating transaction hashes for the different SIGHASH modes, and finally getting your first transaction into the block chain. I really appreciated the .json files holding the unit test data, which were easy to load into my own test harness, the tables on the Wiki showing what the stack should look like at each point in a script execution, and the diagrams showing transaction signing. Bitcoin takes some time to grok when you first approach; more than a day, less than a month, and really no amount of reading documentation or specs will get you to that ah ha moment. When the fog lifts and the blockchain, scripting, signing, and wallet handling really click, suddenly the bitcoind code (and many other great public sources in just about any language you could want) actually does starts to feel fairly simple and obvious. But it certainly doesn't start out that way on day one. I think the majority of client code development is actually people writing 'agents' not end-user P2P wallets, and they tend to be written to connect to a single bitcoind acting as a proxy to the network. Even some end-user wallets work this way! As such, I spent very little time in my own client writing P2P protocol code, no peer discovery code, no anti-DoS, etc. Clients like this also don't pose much systemic risk, because they don't mine, they don't connect directly to external nodes, etc. They can certainly be used to cause trouble though, but so can 'sendrawtransaction'. I chose to speak the P2P protocol to bitcoind versus using some of the other options like ZeroMQ, but it still didn't take long to get headers, blocks, and transactions downloading. I remember getting stuck on the very first version message, because of missing the checksum and user-agent or something caused the latest bitcoind to just ignore me. A little wireshark capture of the exchange between two working bitcoind instances cleared it right up. I didn't mind the leg work, I don't think everything needs to be spoon fed, and it's certainly not purposefully obfuscated. Maybe one exception is the mix-matched endianness will throw you off, especially if you are developing on LE! Anyway, I have huge respect for how much effort it takes to keep even small bits of documentation up-to-date. For as slow as bitcoin moves, it's actually moving incredibly fast. Finally, the bitcoind console and debug logs, as well sites like blockchain.info and blockexplorer.com are hugely helpful for debugging raw and live transactions for when you get stuck. There's a surprisingly large tooling and support ecosystem out there. Moral of the story, I think, is everything you need is there. No, it's not all in one place. Yes, it takes time to find it and assimilate all that knowledge. It also really helps that the community is extremely willing to help and answer technical questions, and point you in the right direction, even when you're working on your own private client code. The IRC channel can certainly be intimidating because it seems like every time I hit enter to send a question, gmaxwell's respond 300ms later would invoke an immediate forehead slap and a groan of shit, I knew/should have known that, now I feel dumb ;-) but if you're working on bitcoin, you better get used to not being the smartest person in the room! The responses I got were never arrogant or disparaging, but they were straight to-the-point and surprisingly high quality. Ain't no slouches in that channel, yes you will have to bring your A-game and you are expected to have tried first before just asking. I have fairly limited experience working on open source projects, but I'm extremely happy with my experience with the Bitcoin community and found writing Bitcoin code hugely enjoyable. The flip side to helping people implement their own clients, agents, or even miners, is helping people to contribute pulls requests, or at the very highest echelon, a BIP. If you haven't written any significant Bitcoin code, you might want to consider investing in that first before submitting a BIP. :-) For a BIP to be valuable, often it requires widespread or even consensus adoption. BIPs are probably not the place to toss just any old 'good idea' because BIPs impose a cost on all active developers. I want to read and understand 'all the BIPs' because for the most part they are actually essential, like, how to handle duplicate transactions in
Re: [Bitcoin-development] BIP0039 Mnemonic code for generating deterministic keys
On Thu, Oct 24, 2013 at 9:23 PM, Pieter Wuille pieter.wui...@gmail.comwrote: This is a proposal I wrote a year ago, but never spent enough work to push it as a standard: https://bitcointalk.org/index.php?topic=102349.0 I think that PoW concept in your proposal is quite smart! However the problem that it isn't bidirectional; it don't allow to convert back and forth between mnemonic and seed, which was one of basic requirement for such algorithm. slush -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP0039 Mnemonic code for generating deterministic keys
On Thu, Oct 24, 2013 at 9:32 PM, Jorge Timón jti...@monetize.io wrote: This will probably sound stupid to most of you, but I'll say it anyway. The aim of mnemonics is to easily remember, isn't it? Well, I would say more retype than remember. I really don't think that common user will memorize it. But of course, it is still an option. But the approach of removing offensive words is probably counterproductive to achieving that end. These words cause a greater emotional impact in our human moral psyches. No, I dont' think it is stupid! Actually it was my concern as well. Unfortunately I don't think it is politically correct to include all bitches, assholes and motherfuckers in end user product :-). If we were willing to use that fact in our advantage to optimize the maximum unforgettableness criterion, we should actually prefer the most generally offensive words in that list. Specially if they can combine with each other to produce more offensive results, basically the opposite of what we're doing. Isn't legalize murder dirty jew much easier to remember for most people than sandwich house yellow cauliflower? Well, bip39 can have more dictionaries and *maybe* swearword dictionary would gain some popularity ;). slush -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Fri, Oct 25, 2013 at 12:54 AM, Peter Todd p...@petertodd.org wrote: Eligius has contracts to do transaction mining, and it's currently 10% of the hashing power. Yes, and I asked Luke what percentage of that 10% is OOB fee payments, and the answer is a small percentage. So: there are multiple layers of reasons why OOB fee payments will not screw up the fee estimation code: + If the transactions are not broadcast, then they have no effect on the estimates. + If the transactions are broadcast but not relayed because their priority and fee are way below current estimates then they will have very close to zero effect on the estimates. + If the OOB transaction is zero-fee, zero-priority (e.g comes from a high-tx-volume service and relies on recently spent outputs) it will have zero effect on the estimates. + If they make up less than about 40% of broadcast transactions they will have very close to zero effect on the fee estimate (because of the distribution of fees and behavior of taking a median) The only case where the estimation code is even slightly likely to get confused is estimating the priority needed to get into a block IF there are a significant number of zero-fee, low-but-not-zero-priority OOB transactions being broadcast. And since priority naturally increases over time, even if that case DOES occur the failure is very mild-- it means your free transactions might have to build up more priority than the code estimates before successfully entering a block. If that gets to be an actual problem, then implementing Pieter's idea of keeping track of memory pool transactions that are NOT getting mined would fix it. But I don't want to waste time on a theoretical problem when it is very possible miners will decide to stop accepting free transactions alltogether. And all of the above is completely orthogonal to child-pays-for-parent and/or replace-with-higher-fee. PS: I would appreciate it if you stop saying things like Regarding the transaction fee estimate code, it's not very well thought out. -- -- Gavin Andresen -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal to replace BIP0039
On Thursday, October 24, 2013 5:29:18 PM thoma...@gmx.de wrote: I would like to propose a new BIP, that replaces BIP0039. BIP 39 is still a draft. Just suggest revisions to the author(s)... -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
