Re: [Bitcoin-development] Announcement: libcoin
> You will also find the RPC server in libcoin blistering fast compared to the Satoshi client. (It was actually what got me to write libcoin in the first place...). The Satoshi client HTTP server executes all rpc commands in its own thread, but to do so, it needs to stop the thread of the Node, even though the command executed is just a query (i.e. not a SendTo), you hence have two threads blocking each other and when they wait, you wait... In libcoin all the query methods access the blockChain as a const object and they can hence safely query it without intervening the work of the Node thread. The exception are the SendTo methods that first query if a transaction can take place, then pushes it to the work-queue of the Node thread and again exits immediately. The actual execution then follows once the Node has finished its current tasks (e.g. validating a block). Hello Michael, I'm impressed by your refactorings, and hope some of them can make it into the Satoshi codebase. I am however not sure what you've said above is safe. In particular, how do you guarantee that no other thread modifies the blockchain structure while you are performing your query on it? Does the query code operate on a const copy of the structure, or is there guaranteed only one thread accessing it? I've been thinking about moving to read-write locks that allow multiple threads reading the datastructure simultaneously, but removing the locking all together sounds wrong to me. -- Pieter -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] libcoin (HEAD) now supports boost < 1.47 - please test
On Thursday, February 02, 2012 5:43:07 PM Michael Grønager wrote: > Enabling dynamic libs was on my TODO, but on the > Redmond_OS_not_to_be_mentioned you need to : * prepend class definitions > with __declspec(dllexport) when you compile the dll * prepend class > definitions with __declspec(dllimport) when you use the dll I just love > the way they spoil their developers... I hadn't even thought of this. Sounds like a pain :/ The problem I had was related to *using* static libraries; ie, boost. I have libboost*.so, but libboost*.a -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] libcoin (HEAD) now supports boost < 1.47 - please test
Thanks for the patch and the detective work! Enabling dynamic libs was on my TODO, but on the Redmond_OS_not_to_be_mentioned you need to : * prepend class definitions with __declspec(dllexport) when you compile the dll * prepend class definitions with __declspec(dllimport) when you use the dll I just love the way they spoil their developers... I have the framework to automate this with CMake from a former project, but I havn't tested it for libcoin yet, hence the static build. And well, iOS also has this fetich for static libs. I also recall another issue with dll's: If you define a global variable it is shared between all executables using this dll. I have still a handful of globals to clean out, namely those related to logging - I will do so, but it has not yet been a top priority. So, feel free to use dynamic libs on unix'es, but on windows it is .libs for a little while longer. Will fix the Qt stuff in CMake - thanks! Cheers, Michael On 02/02/2012, at 17:30, Luke-Jr wrote: > On Thursday, February 02, 2012 8:46:05 AM Michael Grønager wrote: >> Please test and feed back. > > I found the problem: you are trying to use static libraries. Best practices > are to use shared libraries (except for specific scenarios like universal > "Linux" binaries) and most distros do not have static libraries installed by > default. > > The coinQt stuff was also creating a problem. > > I am able to build with this patch: > diff --git a/CMakeLists.txt b/CMakeLists.txt > index dea37c4..b876881 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -352,7 +352,7 @@ ENDIF(BDB_FOUND) > #Note: We need as a minimum Boost 1.47 to support the signal_set used in > Server. A backup signal_set has been created, though. > SET(Boost_NO_BOOST_CMAKE ON) > SET(Boost_ADDITIONAL_VERSIONS "1.47" "1.47.0" "1.48" "1.48.0") > -SET(Boost_USE_STATIC_LIBSON) > +#SET(Boost_USE_STATIC_LIBSON) > SET(Boost_USE_MULTITHREADED ON) > SET(Boost_USE_STATIC_RUNTIMEOFF) > > @@ -804,9 +804,9 @@ SET(PKGCONFIG_FILES > # libcoin-coinMine > ) > > -IF(QT4_FOUND) > - SET(PKGCONFIG_FILES ${PKGCONFIG_FILES} libcoin-coinQt) > -ENDIF(QT4_FOUND) > +#IF(QT4_FOUND) > +# SET(PKGCONFIG_FILES ${PKGCONFIG_FILES} libcoin-coinQt) > +#ENDIF(QT4_FOUND) > > FOREACH(PKGCONFIG_FILE ${PKGCONFIG_FILES}) > > CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/packaging/pkgconfig/${PKGCONFIG_FILE}.pc.in > diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt > index 0011392..67044d1 100644 > --- a/src/CMakeLists.txt > +++ b/src/CMakeLists.txt > @@ -19,9 +19,9 @@ FOREACH( mylibfolder > > ENDFOREACH() > > -IF (QT4_FOUND AND NOT ANDROID) > -ADD_SUBDIRECTORY(coinQt) > -ENDIF() > +#IF (QT4_FOUND AND NOT ANDROID) > +#ADD_SUBDIRECTORY(coinQt) > +#ENDIF() > > IF(ANDROID) > configure_file("${LIBCOIN_ANDROID_TEMPLATES}/Android.mk.src.in" > "${CMAKE_CURRENT_BINARY_DIR}/Android.mk") Michael Gronager, PhD Director, Ceptacle Jens Juels Gade 33 2100 Copenhagen E Mobile: +45 31 45 14 01 E-mail: grona...@ceptacle.com Web: http://www.ceptacle.com/ -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Announcement: libcoin
On Thu, Feb 2, 2012 at 12:36 PM, Gregory Maxwell wrote: > On Thu, Feb 2, 2012 at 12:12 PM, Gregory Maxwell wrote: >> sync, libbitcoin only made it to height 138k (of course, because the >> time is mostly spent late in the chain 138k is not very far along— I'm >> guessing it's going to take libbitcoin 3x-4x longer all said) > > It ended up taking almost exactly twice as long, FWIW. (and Gah: forgive the autocompletion of my fingers: I'm apparently unable to type the word coin without prefacing it with bit) *libcoin* not libbitcoin. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 20 Rejected, process for BIP 21N
OK - I've added a comment to the pull request. On 2 February 2012 17:39, Matt Corallo wrote: > Not yet, its up to genjix (Amir) to do that. See > https://github.com/genjix/bips/pull/2 > > Matt > > On Thu, 2012-02-02 at 17:07 +, Gary Rowe wrote: > > BlueMatt, did the BIP0021 Wiki entry for "req:" to "req-" get updated? > > I'm looking there now and it seems to be still at "req:" > > > -- > > Keep Your Developer Skills Current with LearnDevNow! > > The most comprehensive online learning library for Microsoft developers > > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > > Metro Style Apps, more. Free future releases when you subscribe now! > > http://p.sf.net/sfu/learndevnow-d2d > > ___ Bitcoin-development > mailing list Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > -- > Keep Your Developer Skills Current with LearnDevNow! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-d2d > ___ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 20 Rejected, process for BIP 21N
Not yet, its up to genjix (Amir) to do that. See https://github.com/genjix/bips/pull/2 Matt On Thu, 2012-02-02 at 17:07 +, Gary Rowe wrote: > BlueMatt, did the BIP0021 Wiki entry for "req:" to "req-" get updated? > I'm looking there now and it seems to be still at "req:" > -- > Keep Your Developer Skills Current with LearnDevNow! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-d2d > ___ Bitcoin-development mailing > list Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Announcement: libcoin
On Thu, Feb 2, 2012 at 12:12 PM, Gregory Maxwell wrote: > sync, libbitcoin only made it to height 138k (of course, because the > time is mostly spent late in the chain 138k is not very far along— I'm > guessing it's going to take libbitcoin 3x-4x longer all said) It ended up taking almost exactly twice as long, FWIW. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Announcement: libcoin
On Wed, Feb 1, 2012 at 9:18 AM, Michael Grønager > The libcoin/bitcoind client downloads the entire block chain 3.5 times faster than the bitcoin/bitcoind client. This is less than 90 minutes on a modern laptop! I'm guessing that you benchmarked this against the version you forked from rather than the current reference client? If so— I suspect your speedup was almost entirely because you removed the secure allocator and as a result fixed the mlock performance bug [https://bitcointalk.org/index.php?topic=56491.0] as a side effect. On some systems the mlock issue makes a very big difference (on other systems not so much). In any case, I finally got libbitcoin built and I'm disappointed to report that in the same time it takes the reference client to fully sync, libbitcoin only made it to height 138k (of course, because the time is mostly spent late in the chain 138k is not very far along— I'm guessing it's going to take libbitcoin 3x-4x longer all said) I assume the reason it's actually slower is because it's CPU bound on ECDSA checks, which are skipped in bitcoin in blocks up to the highest hardcoded checkpoint. Without that difference I suspect libbitcoin would be about the same speed— maybe a little faster because of the other changes you mentioned (though, e.g. lock profiling shows hardly any contention during sync). I don't doubt your rpc performance is a lot better. There is a longstanding pull request for async rpc for the reference client that hasn't been merged. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 20 Rejected, process for BIP 21N
BlueMatt, did the BIP0021 Wiki entry for "req:" to "req-" get updated? I'm looking there now and it seems to be still at "req:" -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] libcoin (HEAD) now supports boost < 1.47 - please test
On Thursday, February 02, 2012 8:46:05 AM Michael Grønager wrote: > Please test and feed back. I found the problem: you are trying to use static libraries. Best practices are to use shared libraries (except for specific scenarios like universal "Linux" binaries) and most distros do not have static libraries installed by default. The coinQt stuff was also creating a problem. I am able to build with this patch: diff --git a/CMakeLists.txt b/CMakeLists.txt index dea37c4..b876881 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -352,7 +352,7 @@ ENDIF(BDB_FOUND) #Note: We need as a minimum Boost 1.47 to support the signal_set used in Server. A backup signal_set has been created, though. SET(Boost_NO_BOOST_CMAKE ON) SET(Boost_ADDITIONAL_VERSIONS "1.47" "1.47.0" "1.48" "1.48.0") -SET(Boost_USE_STATIC_LIBSON) +#SET(Boost_USE_STATIC_LIBSON) SET(Boost_USE_MULTITHREADED ON) SET(Boost_USE_STATIC_RUNTIMEOFF) @@ -804,9 +804,9 @@ SET(PKGCONFIG_FILES # libcoin-coinMine ) -IF(QT4_FOUND) - SET(PKGCONFIG_FILES ${PKGCONFIG_FILES} libcoin-coinQt) -ENDIF(QT4_FOUND) +#IF(QT4_FOUND) +# SET(PKGCONFIG_FILES ${PKGCONFIG_FILES} libcoin-coinQt) +#ENDIF(QT4_FOUND) FOREACH(PKGCONFIG_FILE ${PKGCONFIG_FILES}) CONFIGURE_FILE(${PROJECT_SOURCE_DIR}/packaging/pkgconfig/${PKGCONFIG_FILE}.pc.in diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 0011392..67044d1 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -19,9 +19,9 @@ FOREACH( mylibfolder ENDFOREACH() -IF (QT4_FOUND AND NOT ANDROID) -ADD_SUBDIRECTORY(coinQt) -ENDIF() +#IF (QT4_FOUND AND NOT ANDROID) +#ADD_SUBDIRECTORY(coinQt) +#ENDIF() IF(ANDROID) configure_file("${LIBCOIN_ANDROID_TEMPLATES}/Android.mk.src.in" "${CMAKE_CURRENT_BINARY_DIR}/Android.mk") -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] libcoin (HEAD) now supports boost < 1.47 - please test
I have added a simplified fall back class to the boost::asio::signal_set. This should enable compilation on platforms with less than bleeding edge versions of Boost. Most notably most of the currently deployed Linux'es that use Boost 1.42. I also updated the root CMakeLists.txt to only require 1.42. It works for me, but I also recognize the fact that committers machines has some intrinsic magic that just makes things work, hiding actual errors for the them ;) Please test and feed back. Cheers, Michael Michael Gronager, PhD Director, Ceptacle Jens Juels Gade 33 2100 Copenhagen E Mobile: +45 31 45 14 01 E-mail: grona...@ceptacle.com Web: http://www.ceptacle.com/ -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Announcement: libcoin
On Thu, Feb 02, 2012 at 09:32:24AM +0100, Michael Gr?nager wrote: > >// Register Wallet methods. - note that we don't have any auth, so anyone > (on localhost) can read your balance! >server.registerMethod(method_ptr(new GetBalance(wallet))); >server.registerMethod(method_ptr(new SendToAddress(wallet)), > Auth("username","password")); >server.registerMethod(method_ptr(new ExtraGetBalance(wallet))); >server.registerMethod(method_ptr(new ExtraSendToAddress(wallet)), > Auth("username","password")); >server.run(); This should probably be *server.registerMethod(method_ptr(new ExtraGetBalance(extrawallet))); *server.registerMethod(method_ptr(new ExtraSendToAddress(extrawallet)), Auth("username","password")); right? Easy to miss if both wallets were empty in your quick test... Awesome job with libcoin, BTW! -Craig -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Announcement: libcoin
I agree on your architectural considerations - and with libcoin you can have several wallets in the same application ( and several RPC servers for that matter). And ... they all use the same Node / blockchain. You will also find the RPC server in libcoin blistering fast compared to the Satoshi client. (It was actually what got me to write libcoin in the first place...). The Satoshi client HTTP server executes all rpc commands in its own thread, but to do so, it needs to stop the thread of the Node, even though the command executed is just a query (i.e. not a SendTo), you hence have two threads blocking each other and when they wait, you wait... In libcoin all the query methods access the blockChain as a const object and they can hence safely query it without intervening the work of the Node thread. The exception are the SendTo methods that first query if a transaction can take place, then pushes it to the work-queue of the Node thread and again exits immediately. The actual execution then follows once the Node has finished its current tasks (e.g. validating a block). I have attached the code for a very simple one node, two wallet, libcoin client below (~30 lines), and I have added it to the libcoin source as an example (example name: extrawallets). Once running, you can access your extra wallet using the RPC interface: ./extrawallet extragetbalance And youy normal wallet by: ./extrawallet getbalance I'll leave the generalization to an n-wallet gui application to the reader ;) Cheers, Michael // The derived classes below are only to get other class names (using the auto rpc name feature) // I will put adding a "setName" method to the Method class on the todo. class ExtraGetBalance : public GetBalance { public: ExtraGetBalance(Wallet& wallet) : GetBalance(wallet) {} }; class ExtraSendToAddress : public GetBalance { public: ExtraSendToAddress(Wallet& wallet) : GetBalance(wallet) {} }; int main(int argc, char* argv[]) { logfile = CDB::dataDir(bitcoin.dataDirSuffix()) + "/debug.log"; Node node; // deafult chain is bitcoin Wallet wallet(node, "wallet.dat"); // add the wallet Wallet extra_wallet(node, "extra_wallet.dat"); // add the extra wallet thread nodeThread(&Node::run, &node); // run this as a background thread Server server; // Register Server methods. server.registerMethod(method_ptr(new Stop(server))); // Register Node methods. server.registerMethod(method_ptr(new GetBlockCount(node))); server.registerMethod(method_ptr(new GetConnectionCount(node))); server.registerMethod(method_ptr(new GetDifficulty(node))); server.registerMethod(method_ptr(new GetInfo(node))); // Register Wallet methods. - note that we don't have any auth, so anyone (on localhost) can read your balance! server.registerMethod(method_ptr(new GetBalance(wallet))); server.registerMethod(method_ptr(new SendToAddress(wallet)), Auth("username","password")); server.registerMethod(method_ptr(new ExtraGetBalance(wallet))); server.registerMethod(method_ptr(new ExtraSendToAddress(wallet)), Auth("username","password")); server.run(); node.shutdown(); nodeThread.join(); } On 02/02/2012, at 00:50, grarpamp wrote: >> However, I think perhaps the bitcoin project should be split into a library, >> with a prototype client and the actual clients. This library facilitates >> this. > > I'll be trying your implementation soon. And libbitcoin/subvertx too. > Partly because they're also non-interpreted, and partly to what seems > better architected... > > To the minimal extent of my understanding... I'd like to see wallet > ops completely separated from background chain ops. ie: have > a chain daemon doing it's thing, updating, verifying, etc. The > generator doing it's thing. And a wallet app that can independently > manage separate wallets in parallel, referencing the live chain files > as needed. It seems a library would allow quality focus on the separate > functions and let apps/ui's use the fn's as desired on top. Right now, it > seems I have to run bitcoind and can only deal with one wallet at a time, > having to stop it, deal with state issues, swap in a new wallet, start > it, and repeat till illness ensues :( And when the chain is being processed > hard by the daemon cpuwise, bitcoin RPC takes minutes to respond, if ever > or errors out. If wallet ops or statistical queries on the chain need it for > integrity or reading, a db checkpoint/lock/logroll could be implemented into > the chain demon processes with a client lib api to trigger it as needed. > Don't know, just saying. > > fyi... boost 1.48 and db 4.8.30 work fine with 0.5.2, 0.5.x, and master, > you just need to compile and include it by hand if you want it and > your package manager doesn't have it. Michael Gronager, PhD Director, Ceptacle Jens Juels Gade 33 2100 Copenhagen E Mobile: +45 31 45 14 01 E-mail: grona...@ceptacle.com Web: http://www.ceptacle.com/ -