Re: [Bitcoin-development] JSON-RPC is BIP territory or not?
Since several independent clients (I know at least libcoin https://github.com/ceptacle/libcoin/blob/master/src/coinHTTP/RequestHandler.cpp and BitcoinJS https://github.com/bitcoinjs/bitcoinjs-server/tree/master/lib/rpc) aim to implement JSON-RPC APIs which are either a superset of the original client's or have at least some compatible functions, I think you can make a case for including JSON-RPC API calls within the domain of BIPs. In this instance the BIP aims to create a common protocol between different clients, miners, mining proxies and pools. That's a lot of software, so standardization definitely seems like a good idea and I can't think of a reason not to use the BIP process. I have some comments on the content of the BIP, but since this thread is more of a meta-discussion I'll wait until the BIP is officially proposed. On 3/2/2012 7:51 PM, Amir Taaki wrote: Hi, I got sent this BIP: https://en.bitcoin.it/wiki/BIP_DRAFT:_getmemorypool#JSON-RPC_Method:_getmemorypool What is your opinion on this? Is it BIP related? It is a implementation-specific non-bitcoin-protocol proposal. My understanding of BIPs is that they apply across bitcoin implementations and largely focus on the most generic use-cases (like the URIs) and the protocol. Things which affect all clients, and allow the system to function as a united whole. That BIPs especially focus on the protocol, and that something like this is outside the mandate of the BIP process. For instance, we could imagine a future scenario. Bitcoin-Qt is currently based off bitcoind's codebase. However wumpus built the client in mind with an abstraction layer to enable multiple backends (a good design). In our hypothetical situation, there are 3 different backend codebases using Bitcoin-Qt. I do not think a proposal to mandate a changing to Bitcoin-Qt's abstraction layer or a change in the UI placement would be appropriate BIP material. OTOH, many clients do need to make use of URIs and the BIP process is totally correct, as it standardises a behaviour which is needed for interoperability of the network and community. Thoughts? -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
On Saturday, March 03, 2012 9:23:08 AM Stefan Thomas wrote: From what I understand the BIP uses a polling model, e.g. a miner would use getmemorypool to request new work from a pool in intervals. Would it make sense to specify a version of the API supporting long polling? You mean explicitly mention the getwork longpoll protocol as part of the getmemorypool BIP? Probably a good idea. For BitcoinJS specifically, since we also support JSON-RPC over TCP, I'd even be interested in a genuine push API. Something like C-S listenmemorypool, S-C updatememorypool (continually). HTTP and JSON-RPC are a client-server model; there is no way for the server to make calls to the client. It's not practical to expect clients to run their own JSON-RPC server - many cannot listen on WAN ports at all. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
On Saturday, March 03, 2012 10:05:58 AM Gavin Andresen wrote: HTTP and JSON-RPC are a client-server model; there is no way for the server to make calls to the client. It's not practical to expect clients to run their own JSON-RPC server - many cannot listen on WAN ports at all. You're doing that thing where either you say something before you've done adequate research, or you're being needlessly pedantic; I just copied and pasted this from section 2.1 of the JSON-RPC spec: I wasn't aware anyone was considering JSON-RPC over anything other than HTTP. I've updated the draft to include long polling, and remove some assumptions of using HTTP for transport. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Duplicate transactions vulnerability
On Tue, Feb 28, 2012 at 17:48, Pieter Wuille pieter.wui...@gmail.com wrote: Hello all, I've written about it in BIP30[2]. There is a patch for the reference client, which has been tested and verified to make the attack impossible. The change is backward compatible in the same way BIP16 is: if a supermajority of mining power implements it, old clients can continue to function without risk. After getting responses from Deepbit, bitcoin.cz (slush), MtRed, Bitlc and BTCmine, it looks like march 15 is a reasonable deployment date for the security update described in BIP 30. I have created patches for: * git master: https://github.com/sipa/bitcoin/tree/nooverwritetx * v0.4.0: https://github.com/sipa/bitcoin/tree/nooverwritetx_v0.4.0 * v0.3.24: https://github.com/sipa/bitcoin/tree/nooverwritetx_v0.3.24 * v0.3.24+vinced: https://github.com/sipa/bitcoin/tree/nooverwritetx_v0.3.24+vinced * v0.3.19: https://github.com/sipa/bitcoin/tree/nooverwritetx_v0.3.19 I've asked pool operators to upgrade, and confirm when they have done so. If you are a miner or pool operator, and have the ability to upgrade, please do so as well. Thanks, -- Pieter -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
HTTP and JSON-RPC are a client-server model; there is no way for the server to make calls to the client. It's not practical to expect clients to run their own JSON-RPC server - many cannot listen on WAN ports at all. Well, I think what Stefan had in mind was http keep-alive combined with an event system. So similar to the way a web chat application work, just for json-rpc. BitcoinJS already uses this for realtime updating a webwallet. Libcoin is also prepared for this with a quite advanced, non-blocking, http server so I second Stefan that an update function could indeed be of relevance. Btw - question to Stefan as the JavaScript guru - what do you consider the standard/defacto-standard/right/best-practice way of doing S-C json-rpc, what (javascript) library do you use for this? Cheers, Michael -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Fwd: Proposal for a new opcode
On Fri, Mar 2, 2012 at 2:57 PM, Watson Ladd w...@uchicago.edu wrote: Dear all, I am proposing a new opcode for the purposes of anonymous transactions. That's very exciting! I'm eager to read the paper for all of the details, and working out what else would need to be done besides a new opcode to enable strong anonymity (at the very least, I assume we'll need one or more new 'standard' transaction types that clients understand). -- -- Gavin Andresen -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
I've updated the draft to include long polling, and remove some assumptions of using HTTP for transport. Looks good to me. On 3/3/2012 4:44 PM, Luke-Jr wrote: On Saturday, March 03, 2012 10:05:58 AM Gavin Andresen wrote: HTTP and JSON-RPC are a client-server model; there is no way for the server to make calls to the client. It's not practical to expect clients to run their own JSON-RPC server - many cannot listen on WAN ports at all. You're doing that thing where either you say something before you've done adequate research, or you're being needlessly pedantic; I just copied and pasted this from section 2.1 of the JSON-RPC spec: I wasn't aware anyone was considering JSON-RPC over anything other than HTTP. I've updated the draft to include long polling, and remove some assumptions of using HTTP for transport. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
Btw - question to Stefan as the JavaScript guru - what do you consider the standard/defacto-standard/right/best-practice way of doing S-C json-rpc, what (javascript) library do you use for this? As for an explicitly standard way, there is none. The JSON-RPC 1.0 spec http://json-rpc.org/wiki/specification says The specifications do not require a certain transport protocol. The use of TCP/IP socket streams is encouraged. The serialized request and response objects are sent to the peers through the byte streams. The JSON-RPC 2.0 spec http://jsonrpc.org/specification goes out of its way to say It is transport agnostic in that the concepts can be used within the same process, over sockets, over http, or in many various message passing environments. The de-facto standard for bidirectional JSON-RPC is plain TCP sockets. BitcoinJS currently implements this - we detect whether an incoming connection is HTTP or raw JSON-RPC based on the first character. (HTTP must start with an uppercase letter, raw JSON-RPC must start with an opening curly bracket.) There are two things to watch out for with JSON-RPC over plain TCP: 1. Plain TCP sockets (unlike HTTP) have no standardized authentication mechanism, so I added an extra RPC call auth(username, password). 2. The TCP packets may or may not correspond to JSON-RPC messages. You can either use a streaming JSON parser (yajl in ANSI C, Jackson in Java, etc.), or you can just count (non-string-literal) curly braces to detect when a complete message has arrived. Many JSON-RPC libraries come with TCP socket support out of the box: http://json-rpc.org/wiki/implementations We're planning to add more features to our JSON-RPC API in the future, such as: - JSON-RPC over TLS sockets - Challenge-response authentication - TLS client handshake (certificate authentication) As for HTTP Keep-Alive: It works, but I don't think it's very widely supported among client libraries and HTTP isn't really made for this type of thing, so my gut instinct would be to avoid it. That said, it doesn't hurt to offer the option. On 3/3/2012 6:08 PM, Michael Grønager wrote: HTTP and JSON-RPC are a client-server model; there is no way for the server to make calls to the client. It's not practical to expect clients to run their own JSON-RPC server - many cannot listen on WAN ports at all. Well, I think what Stefan had in mind was http keep-alive combined with an event system. So similar to the way a web chat application work, just for json-rpc. BitcoinJS already uses this for realtime updating a webwallet. Libcoin is also prepared for this with a quite advanced, non-blocking, http server so I second Stefan that an update function could indeed be of relevance. Btw - question to Stefan as the JavaScript guru - what do you consider the standard/defacto-standard/right/best-practice way of doing S-C json-rpc, what (javascript) library do you use for this? Cheers, Michael -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
On 28.02.2012 23:06, Luke-Jr wrote: Please review and comment/critique: https://en.bitcoin.it/wiki/BIP_DRAFT:_getmemorypool Looking forward to implementing this in my pool backend and miner. A few comments: transactions add or remove transactions (both of the above; default if transactions omitted) In the above, you may want to specify that the transactions referred to here is the one in the first table (JSON-RPC response object) and not the mutations. For a moment I thought free tx editing was the default. Long polling as currently implemented in pools has a race condition. Does the miner reconnect first or does another block change happen first? Double block changes are common with merged mining and I'm doing all sorts of tricks in my pool backend to reduce this problem. How about another entry longpollid in long poll responses. The last seen longpollid should be included by the client in future long poll requests. This enables the server to see if the client has missed any block changes. The ID could perhaps be submitted in an HTTP header (X-LongPollID?) if we wish to keep the JSON-RPC params empty, or params could hold an object with a key longpollid. Could be a string or number, like workid. Another useful value in the getmemorypool response would be height, so the miner can include the correct height in the coinbase. I would like that in bitcoind as well. One JSON-RPC call instead of two, and no race condition between getmemorypool and getblocknumber. It should be explained how target vs. fulltarget works. Perhaps some things should be optional for a client to implement? I think noncerange is of limited use and there's a good chance of getting the endianness wrong. Regards, Geir Harald Hansen -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] getmemorypool BIP process
On Saturday, March 03, 2012 6:51:34 PM Geir Harald Hansen wrote: Long polling as currently implemented in pools has a race condition. Does the miner reconnect first or does another block change happen first? Double block changes are common with merged mining and I'm doing all sorts of tricks in my pool backend to reduce this problem. How would you suggest addressing this? I presume if a share solves blocks on multiple chains, you just longpoll once when that's successful? How about another entry longpollid in long poll responses. The last seen longpollid should be included by the client in future long poll requests. This enables the server to see if the client has missed any block changes. The ID could perhaps be submitted in an HTTP header (X-LongPollID?) if we wish to keep the JSON-RPC params empty, or params could hold an object with a key longpollid. Could be a string or number, like workid. Hmm, the problem is that adding any parameters to getmemorypool itself breaks compatibility with bitcoind 0.5, and using HTTP headers makes it HTTP-specific again. Any ideas? Another useful value in the getmemorypool response would be height, so the miner can include the correct height in the coinbase. I would like that in bitcoind as well. One JSON-RPC call instead of two, and no race condition between getmemorypool and getblocknumber. Good catch. Should this be required (since it might be necessary for future Bitcoin blocks), or just should for compatibility? It should be explained how target vs. fulltarget works. What is unclear about this? Perhaps some things should be optional for a client to implement? Doing this safely needs some way for clients to communicate capabilities to the server, which has the problem of passing parameters to getmemorypool. I think noncerange is of limited use and there's a good chance of getting the endianness wrong. There is no mining hardware to date that exhausts even half the nonce space, so I'd really prefer to see this as a required feature on the miner side. On the other hand, it's merely an extension for getwork, so I can see the problem so long as we're using getwork proxies. Luke -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
