Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-03-24 Thread Jorge Timón
That case is very unlikely IMO, but still you can solve it while keeping hash of the genesis block as the chain id. If a community decides to accept a forking chain with new rules from block N (let's call it bitcoinB), the original chain can maintain the original genesis block and the new

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-03-14 Thread Isidor Zeuner
That was essentially what we did in the end, we replaced the network identifier (main/test) with the genesis block hash. The result is never going to accidentally work with Bitcoin Core (nor vice-versa), but is readily extensible to any other altcoins that want to use the specification

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Wladimir
On Wed, 28 Jan 2015, Nicolas DORIER wrote: I agree that the use protocol buffer and x509 by BIP70 is a poor choice. Well x509 is an international standard in common use, you can't do much better with regard to portability. Your suggestion about HTTPS makes little sense, you do know what TLS

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas DORIER
Sure I know that x509 is international standard. And that HTTPS uses TLS. This is not my point, my point is that when we use HTTPS the developer delegates certificates verification to the plateform he is running on, so developer don't have to bother about it, making the implementation safer and

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas DORIER
I agree that the use protocol buffer and x509 by BIP70 is a poor choice. The choice should have been done to maximize portability, not to maximize efficiency and flexibility. What I ended up doing for having a similar codebase on all plateform is to parse a BIP70 messages with the help of a web

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas DORIER
Sure, But the mobile targets, it is still easier to use Json + HTTPS, especially when you want one code base for everything. And as you said, developers need to think about fetching mozilla store time to time, and check revocations themselves. This is not obvious thing to do, and hard to test

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Jeff Garzik
It is not fear, it is field experience. JSON has proven to be a bug generator for the reasons already stated. JSON does not include type marshalling and input validation. Protobufs/msgpack/etc. engineered those to occur automatically, because that is an area shown by field experience to be a

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Mike Hearn
It is not fear, it is field experience. JSON has proven to be a bug generator for the reasons already stated. To back Jeff up on this point, today we see this story: http://www.theregister.co.uk/2015/01/27/trivial_hole_left_black_phones_open_to_plunder/ The maker of BlackPhone – a mobile

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Mike Hearn
I'm frankly _horrified_ to learn that BitcoinJ ships its own root CA certificates bundle. This means that, if a root CA gets breached and a certificate gets revoked, all BitcoinJ-using software will be vulnerable until BitcoinJ ships an update *and* the software in question pulls in the new

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Mike Hearn
I think we'll just have to agree to disagree on this one. I've implemented BIP70 a couple of times now and didn't find it to be difficult. I know you had odd problems with the C# protobuf implementation you were using but library bugs can happen for any kind of programming. I forgot to mention

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Angel Leon
why not allow both serializations and keep serialization format a parameter, keep everyone happy. http://twitter.com/gubatron On Wed, Jan 28, 2015 at 12:14 PM, Mike Hearn m...@plan99.net wrote: I think we'll just have to agree to disagree on this one. I've implemented BIP70 a couple of times

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Mike Hearn
On the other hand, if you charge the developer (and not the plateform) to check certificate validity, it means that you have to develop a different codebase for all plateform you are targeting, because each plateform store trusted root certificate in a different manner with different APIs,

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas DORIER
My point is not that there is a limitation in BIP70. My point is that you put the burden of certificate verification on developer's shoulder when we can just leverage built in HTTPS support of the platform. This make cross plateform dev a nightmare. Sure I can use a snapshot of moz/apple/msft

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Matt Whitlock
On Wednesday, 28 January 2015, at 5:19 pm, Giuseppe Mazzotta wrote: On 28-01-15 16:42, Mike Hearn wrote: Just as a reminder, there is no obligation to use the OS root store. You can (and quite possibly should) take a snapshot of the Mozilla/Apple/MSFT etc stores and load it in your app. We

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Jeff Garzik
Not to mention the tiresome and error-prone task of writing your own JSON-to-schema marshalling code -- or something equivalent to the protobufs compiler and libs for JSON. protobufs -- and its modern competitors such as msgpack -- natively provide type support in a way that must be hacked into

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Giuseppe Mazzotta
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 28-01-15 16:42, Mike Hearn wrote: Just as a reminder, there is no obligation to use the OS root store. You can (and quite possibly should) take a snapshot of the Mozilla/Apple/MSFT etc stores and load it in your app. We do this in bitcoinj

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas DORIER
For the number of field there is in the spec, I don't consider having a JSON to schama really worthwhile. If you fear it is error prone, then we should provide some testing data for the BIP70. (Which I already did for protobuf, but was rejected, because deemed no useful thanks to the code

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Mike Hearn
My point is not that there is a limitation in BIP70. My point is that you put the burden of certificate verification on developer's shoulder when we can just leverage built in HTTPS support of the platform. Platforms that support HTTPS but not certificate handling are rare - I know HTML5 is

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-28 Thread Nicolas Dorier
Mike, I am not denying it is impossible to do all of that. Just that it is not a trivial stuff to do to make it works everywhere, and I think that it is not a good thing for a client side technology. BIP70 has its use, and I understand why there is case where it is good to ship the certs in the

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-25 Thread Ross Nicoll
That was essentially what we did in the end, we replaced the network identifier (main/test) with the genesis block hash. The result is never going to accidentally work with Bitcoin Core (nor vice-versa), but is readily extensible to any other altcoins that want to use the specification without

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-24 Thread Isidor Zeuner
For what it's worth, there was consideration of replacing protocol buffers when modifying BIP70 to function with the altcoin I work on (changes were required anyway in eliminate any risk that payment requests could not be accidentally applied to the wrong blockchain). Why not serialize some

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Jeff Garzik
None of those listed were in the context of performance. Parsing of binary or text is quite fast these days, and is not really a consideration versus other needs such as a predictable encoding for a single data representation. XML and JSON both can represent the same post-evaluation user data a

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Mike Hearn
The engineers at Google were well aware that ASN.1 existed. I can assure you of that, because I was one of them. The protobuf FAQ has a very polite take on the matter: https://developers.google.com/protocol-buffers/docs/faq This email thread gives more enlightenment:

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 19 January 2015 12:09:13 GMT-07:00, Jeff Garzik jgar...@bitpay.com wrote: Text formats such as XML or JSON are far less deterministic, are more loosely specified, have wide variance in parsing, are not very hash-able, the list goes on.

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Richard Brady
Fair points, although for me the line is blurred between which of those are security considerations vs performance considerations. Richard On 19 January 2015 at 19:09, Jeff Garzik jgar...@bitpay.com wrote: Text formats such as XML or JSON are far less deterministic, are more loosely

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Matt Whitlock
Even if a compact binary encoding is a high priority, there are more standard choices than Google Protocol Buffers. For example, ASN.1 is a very rigorously defined standard that has been around for decades, and ASN.1 even has an XML encoding (XER) that is directly convertible to/from the binary

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Jeff Garzik
Text formats such as XML or JSON are far less deterministic, are more loosely specified, have wide variance in parsing, are not very hash-able, the list goes on. On Mon, Jan 19, 2015 at 2:07 PM, Richard Brady rnbr...@gmail.com wrote: Hi Gavin, Mike and co Is there a strong driver behind the

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Jeff Garzik
ASN.1 is not nearly as flexible when it comes to well-supported libraries, generators, and the ecosystem that surrounds the actual encoding. You don't see ASN.1 compilers + language support packages for [all popular programming languages], as you do with protobufs. Google engineers were well

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Ross Nicoll
For what it's worth, there was consideration of replacing protocol buffers when modifying BIP70 to function with the altcoin I work on (changes were required anyway in eliminate any risk that payment requests could not be accidentally applied to the wrong blockchain). The eventual conclusion was

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Jeff Garzik
Correct. I should have said more likely to be deterministic Bitcoin Core does not *rely* on determinism in BIP70; I was referring to recent upstream efforts to make protobufs usable in a deterministic fashion by default. On Mon, Jan 19, 2015 at 3:03 PM, Alan Reiner etothe...@gmail.com wrote:

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Brian Hoffman
Damn if SPKI had won out we would be parsing S-expressions instead of X.509 certificates. ASN.1 is not fun IMHO. On Jan 19, 2015, at 3:56 PM, Gavin Andresen ga...@bitcoinfoundation.org wrote: On Mon, Jan 19, 2015 at 3:40 PM, Mike Hearn m...@plan99.net wrote: OK, I guess we can boil this

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Alan Reiner
I'm a bit confused. It's been a long time since I looked at protobuf (and will have to dig into it soon), but I seem to recall it doesn't have any of the determinism properties you guys just said. It is intended to allow you to skip details of the on-the-wire representations and just send a

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 That's 100% true: BIP70 passes around serialized protobuf data that it signs directly for this reason; it could just as easily be a byte array with json in it. (not that json/XML/etc. doesn't have other flaws) On 19 January 2015 13:03:32

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Mike Hearn
I'm a bit confused. It's been a long time since I looked at protobuf (and will have to dig into it soon), but I seem to recall it doesn't have any of the determinism properties you guys just said. It's not guaranteed no, which is why we store signed sub-messages as byte arrays instead of

Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding?

2015-01-19 Thread Gavin Andresen
On Mon, Jan 19, 2015 at 3:40 PM, Mike Hearn m...@plan99.net wrote: OK, I guess we can boil this down more simply. BIP 70 uses protocol buffers because I designed it and implemented the original prototype (with lots of input from Gavin and an earlier proposal by sipa). I used protocol buffers