Bug#578157: and another!

2011-01-25 Thread micah anderson 
On Mon, 17 Jan 2011 10:50:25 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Mon, Jan 17, 2011 at 12:03:53AM -0500, micah anderson wrote:
 On Mon, 10 Jan 2011 21:31:39 +0100, Jonas Smedegaard d...@jones.dk wrote:
  I consider adding a shell wrapper looking something like this:
 
set -e
umask=077
basedir=~/.bitcoin
dbfile=$basedir/DB_CONFIG
cfgfile=$basedir/bitcoin.conf
[ -d ~/.bitcoin ] || mkdir ~/.bitcoin
[ -e $dbfile ] || echo 'set_flags DB_LOG_AUTOREMOVE'  $dbfile
 
 the above scares me a little, because someone might have put their own
 db config into their DB_CONFIG, and the above would overwrite it,
 everytime they start the daemon.
 
[ -e $cfgfile ] || perl -le 
  'printrpcpassword=,map{(a..z,A..Z,0..9)[rand 62]}0..9'  $cfgfile
 
 nice, but again, wont this run each time bitcoind is started, thus
 making a new rpcpassword every time?
 
 The frontmost [] means test for this, and if not, then

Of course, I read that wrong!

 Does it still scare you?

No, I am not frightened now that I am no longer ignorant!

 For fun, I just committed a few things to the collab-maint repository:
 . examples/bitcoin.conf
 . bitcoind(1) and bitcoin.conf(5) man pages
 
 but I am not so sure what the right way to install the man pages are,
 maybe cdbs does it magically? Check it out and please correct it if its
 wrong.
 
 Nice!
 
 Did you hardcode the manpages or generate (at least the bitcoind one) 
 using help2man?

Well, I used help2man to give me a template, but it wasn't very good, so
I mostly did manual work on it.

 Oh well, I'll have a look at it.
 
 Oh, and please pretty please add your name as uploader.  I would looove 
 this to be a teamwork between us - even if you might go busy on me for 
 years at a time ;-)

Will do.

m


pgpbu7B0ezaTb.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-17 Thread Jonas Smedegaard 

On Mon, Jan 17, 2011 at 12:03:53AM -0500, micah anderson wrote:

On Mon, 10 Jan 2011 21:31:39 +0100, Jonas Smedegaard d...@jones.dk wrote:

I consider adding a shell wrapper looking something like this:

  set -e
  umask=077
  basedir=~/.bitcoin
  dbfile=$basedir/DB_CONFIG
  cfgfile=$basedir/bitcoin.conf
  [ -d ~/.bitcoin ] || mkdir ~/.bitcoin
  [ -e $dbfile ] || echo 'set_flags DB_LOG_AUTOREMOVE'  $dbfile


the above scares me a little, because someone might have put their own
db config into their DB_CONFIG, and the above would overwrite it,
everytime they start the daemon.


  [ -e $cfgfile ] || perl -le 'printrpcpassword=,map{(a..z,A..Z,0..9)[rand 62]}0..9' 
 $cfgfile


nice, but again, wont this run each time bitcoind is started, thus
making a new rpcpassword every time?


The frontmost [] means test for this, and if not, then

Does it still scare you?



For fun, I just committed a few things to the collab-maint repository:
   . examples/bitcoin.conf
   . bitcoind(1) and bitcoin.conf(5) man pages

but I am not so sure what the right way to install the man pages are,
maybe cdbs does it magically? Check it out and please correct it if its
wrong.


Nice!

Did you hardcode the manpages or generate (at least the bitcoind one) 
using help2man?


Oh well, I'll have a look at it.

Oh, and please pretty please add your name as uploader.  I would looove 
this to be a teamwork between us - even if you might go busy on me for 
years at a time ;-)



 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-16 Thread micah anderson 
On Mon, 10 Jan 2011 21:31:39 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Mon, Jan 10, 2011 at 02:15:17PM -0500, micah anderson wrote:
  On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
 Transaction logs are great, if you want to do recovery, but I 
 wonder if they are really needed. If I recall, they simply grow, 
 until you rotate them, unless you set specific DB_CONFIG flags.
 
 Database transaction logs are there in the case of crash, you can do a 
 restore by replaying the logs.
 
 Bitcoin stores inside the *database* the entire history of all bitcoin 
 operations. The fact that database transaction logs are on just means 
 that the entire history of all database operations, (which, 
 incidentally aren't exclusively bitcoin trades between users), is also 
 written to the transaction logs.
 
 Ok.  I think I get it now.  Thanks for clarifying.
 
 I consider adding a shell wrapper looking something like this:
 
   set -e
   umask=077
   basedir=~/.bitcoin
   dbfile=$basedir/DB_CONFIG
   cfgfile=$basedir/bitcoin.conf
   [ -d ~/.bitcoin ] || mkdir ~/.bitcoin
   [ -e $dbfile ] || echo 'set_flags DB_LOG_AUTOREMOVE'  $dbfile

the above scares me a little, because someone might have put their own
db config into their DB_CONFIG, and the above would overwrite it,
everytime they start the daemon.

   [ -e $cfgfile ] || perl -le 
 'printrpcpassword=,map{(a..z,A..Z,0..9)[rand 62]}0..9'  $cfgfile

nice, but again, wont this run each time bitcoind is started, thus
making a new rpcpassword every time?

For fun, I just committed a few things to the collab-maint repository: 
. examples/bitcoin.conf
. bitcoind(1) and bitcoin.conf(5) man pages

but I am not so sure what the right way to install the man pages are,
maybe cdbs does it magically? Check it out and please correct it if its
wrong. 

micah


pgpnWWbUXZlcr.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-10 Thread Jonas Smedegaard 

On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
On Fri, 7 Jan 2011 20:17:29 +0100, Jonas Smedegaard d...@jones.dk 
wrote:
So what really is the issue here is that _at_ _each_ _runtime_ 
_environment_ are the db files incompatible.


http://download.oracle.com/docs/cd/E17276_01/html/programmer_reference/changelog_4_8.html#id1655659

Database or Log File On-Disk Format Changes:

  1.  The log file format changed in 4.8.

So it seems that its just the db4.8 logs are not backward compatible 
with 4.7. Transaction logs are great, if you want to do recovery, but I 
wonder if they are really needed. If I recall, they simply grow, until 
you rotate them, unless you set specific DB_CONFIG flags.


According to Helmut Grohne, all Bitcoin users maintain a full history of 
transactions in those logs - so I suspect they shouldn't be rotated but 
just (slowly) grow and grow...




@Micah: Could you please test if upgrading works like this:

   1) Move aside your precious .bitcoin folder
   2) Start the current Debian Bitcoin (or upstream binary)
   3) Let Bitcoin run until its db is fully init'ed (i.e. dir is 
  ~150MB)

   3) Quit Bitcoin
   4) Install packages libdb4.8-util and db4.7-util


you mean db4.8-util and db4.7-util :)


Ah, yes.  Had crept into README.Debian too - corrected now.



   5) run the following commands:


I think that the following commands should only be run after I've
stopped the running bitcoind:


That happened in 3), no?

Any, I am not this detailed in the readme (and not sure why I spelled it 
out to you either - you seem to know usage of this tool better than 
me!).




 db4.7_recover -h ~/.bitcoin
 db4.8_upgrade -h ~/.bitcoin wallet.dat

   6) Start your locally compiled Bitcoin
   7) Tell me if it worked out or not :-)


It seems like it works, here is the log:


[log snipped]

Good.  Thanks for confirming!


I will now compile a new Bitcoin against Berkeley DB 4.8, with a 
README mentioning essential parts of above - blindly hoping that it 
works.


I checked with the bitcoin community, and found there are quite a 
number of people using bdb4.8 on gentoo, redhat, fedora and freebsd.



Please be careful to test against current release, not the version I am
about to release :-)


Yep, I used 0.3.19~dfsg-3~0jones1 for this test.


Thanks again.


- Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-10 Thread micah anderson 
On Mon, 10 Jan 2011 09:46:29 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
 On Fri, 7 Jan 2011 20:17:29 +0100, Jonas Smedegaard d...@jones.dk 
 wrote:
  So what really is the issue here is that _at_ _each_ _runtime_ 
  _environment_ are the db files incompatible.
 
 http://download.oracle.com/docs/cd/E17276_01/html/programmer_reference/changelog_4_8.html#id1655659
 
 Database or Log File On-Disk Format Changes:
 
1.  The log file format changed in 4.8.
 
 So it seems that its just the db4.8 logs are not backward compatible 
 with 4.7. Transaction logs are great, if you want to do recovery, but I 
 wonder if they are really needed. If I recall, they simply grow, until 
 you rotate them, unless you set specific DB_CONFIG flags.
 
 According to Helmut Grohne, all Bitcoin users maintain a full history of 
 transactions in those logs - so I suspect they shouldn't be rotated but 
 just (slowly) grow and grow...

Well sure, that is because they are transaction logs and every database
operation is written to the logs. That is the definition of database
transaction logs... it is nothing particularly unique to
bitcoin. Berkeley DB (and other databases, when configured to do so)
produces transaction logs that can be used to reconstruct changes from a
given point in time.

It is true that bitcoin maintains a full history of all bitcoin
operations (if I used the word 'transactions' here, it would be
confusing because 'transactions' as also a database term, that history
is maintained in the database itself.

  @Micah: Could you please test if upgrading works like this:
 
 1) Move aside your precious .bitcoin folder
 2) Start the current Debian Bitcoin (or upstream binary)
 3) Let Bitcoin run until its db is fully init'ed (i.e. dir is 
~150MB)
 3) Quit Bitcoin
 4) Install packages libdb4.8-util and db4.7-util
 
 you mean db4.8-util and db4.7-util :)
 
 Ah, yes.  Had crept into README.Debian too - corrected now.
 
 
 5) run the following commands:
 
 I think that the following commands should only be run after I've
 stopped the running bitcoind:
 
 That happened in 3), no?

Yes, it did! What didn't happen was you numbered 3 twice, and so my
brain skipped the 3) Quit Bitcoin and thought it didn't exist because
it had already done 3) Let Bitcoin run until its db is fully init'ed
(i.e. dir is ~150MB) and iterated to the next number :)

It took me a good 5 minutes to realize that. Good morning.

 Any, I am not this detailed in the readme (and not sure why I spelled it 
 out to you either - you seem to know usage of this tool better than 
 me!).

I'm not sure I know it better, I just maybe know different parts better,
and you know other parts better. 

thanks for the libdb4.8++ upload, i upgraded to that version and
everything works fine!

micah


pgpSWYPf7vtYy.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-10 Thread Jonas Smedegaard 

On Mon, Jan 10, 2011 at 09:48:06AM -0500, micah anderson wrote:
On Mon, 10 Jan 2011 09:46:29 +0100, Jonas Smedegaard d...@jones.dk 
wrote:

On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
On Fri, 7 Jan 2011 20:17:29 +0100, Jonas Smedegaard d...@jones.dk 
wrote:
 So what really is the issue here is that _at_ _each_ _runtime_ 
 _environment_ are the db files incompatible.


http://download.oracle.com/docs/cd/E17276_01/html/programmer_reference/changelog_4_8.html#id1655659

Database or Log File On-Disk Format Changes:

   1.  The log file format changed in 4.8.

So it seems that its just the db4.8 logs are not backward compatible 
with 4.7. Transaction logs are great, if you want to do recovery, 
but I wonder if they are really needed. If I recall, they simply 
grow, until you rotate them, unless you set specific DB_CONFIG 
flags.


According to Helmut Grohne, all Bitcoin users maintain a full history 
of transactions in those logs - so I suspect they shouldn't be 
rotated but just (slowly) grow and grow...


Well sure, that is because they are transaction logs and every database 
operation is written to the logs. That is the definition of database 
transaction logs... it is nothing particularly unique to bitcoin. 
Berkeley DB (and other databases, when configured to do so) produces 
transaction logs that can be used to reconstruct changes from a given 
point in time.


You wrote wonder if they are really needed.

What I meant to say was that according to Helmut Grohne, each and every 
Bitcoin user _must_ maintain a full history.



It is true that bitcoin maintains a full history of all bitcoin 
operations (if I used the word 'transactions' here, it would be 
confusing because 'transactions' as also a database term, that history 
is maintained in the database itself.


So, what was the point again?  That the storage format need not be in 
log format?


As I understand it, the log format is used only when bootstrapping via 
IRC, but I really don't know these details.


Helmut Grohne seemed to know - he mentioned having reverse-engineerd the 
protocol and reimplemented in Python.  I tried to persuade hime to join 
me in maintaining Bitcoin - until his own implementation perhaps would 
eventually mature and be superior.  No luck, unfortunately. :-(




 @Micah: Could you please test if upgrading works like this:

1) Move aside your precious .bitcoin folder
2) Start the current Debian Bitcoin (or upstream binary)
3) Let Bitcoin run until its db is fully init'ed (i.e. dir is
   ~150MB)
3) Quit Bitcoin
4) Install packages libdb4.8-util and db4.7-util

you mean db4.8-util and db4.7-util :)

Ah, yes.  Had crept into README.Debian too - corrected now.


5) run the following commands:

I think that the following commands should only be run after I've
stopped the running bitcoind:

That happened in 3), no?


Yes, it did! What didn't happen was you numbered 3 twice, and so my 
brain skipped the 3) Quit Bitcoin and thought it didn't exist because 
it had already done 3) Let Bitcoin run until its db is fully init'ed 
(i.e. dir is ~150MB) and iterated to the next number :)


It took me a good 5 minutes to realize that. Good morning.


Ohhh. :-D


Any, I am not this detailed in the readme (and not sure why I spelled 
it out to you either - you seem to know usage of this tool better 
than me!).


I'm not sure I know it better, I just maybe know different parts 
better, and you know other parts better.


...which was my reason for carefully squeezing in usage into above 
sentence ;-)



thanks for the libdb4.8++ upload, i upgraded to that version and 
everything works fine!


:-)


 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-10 Thread micah anderson 
On Mon, 10 Jan 2011 16:13:13 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Mon, Jan 10, 2011 at 09:48:06AM -0500, micah anderson wrote:
 On Mon, 10 Jan 2011 09:46:29 +0100, Jonas Smedegaard d...@jones.dk 
 wrote:
  On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
  On Fri, 7 Jan 2011 20:17:29 +0100, Jonas Smedegaard d...@jones.dk 
  wrote:
   So what really is the issue here is that _at_ _each_ _runtime_ 
   _environment_ are the db files incompatible.
  
  http://download.oracle.com/docs/cd/E17276_01/html/programmer_reference/changelog_4_8.html#id1655659
  
  Database or Log File On-Disk Format Changes:
  
 1.  The log file format changed in 4.8.
  
  So it seems that its just the db4.8 logs are not backward compatible 
  with 4.7. Transaction logs are great, if you want to do recovery, 
  but I wonder if they are really needed. If I recall, they simply 
  grow, until you rotate them, unless you set specific DB_CONFIG 
  flags.
 
  According to Helmut Grohne, all Bitcoin users maintain a full history 
  of transactions in those logs - so I suspect they shouldn't be 
  rotated but just (slowly) grow and grow...
 
 Well sure, that is because they are transaction logs and every database 
 operation is written to the logs. That is the definition of database 
 transaction logs... it is nothing particularly unique to bitcoin. 
 Berkeley DB (and other databases, when configured to do so) produces 
 transaction logs that can be used to reconstruct changes from a given 
 point in time.
 
 You wrote wonder if they are really needed.
 
 What I meant to say was that according to Helmut Grohne, each and every 
 Bitcoin user _must_ maintain a full history.

Yep, that is true.

 It is true that bitcoin maintains a full history of all bitcoin 
 operations (if I used the word 'transactions' here, it would be 
 confusing because 'transactions' as also a database term, that history 
 is maintained in the database itself.
 
 So, what was the point again?  That the storage format need not be in 
 log format?

The storage format is the database. The logs are database transaction
logs. You get those transaction logs with *every* berkeley db
implementation with every software that uses them. In fact, I dont know
a database implementation that doesn't have transaction log capability
(http://en.wikipedia.org/wiki/Transaction_log). A database transaction
log (also commonly called a binary log) is just a record of every
operation that has happened on the database.

They are not unique to bitcoin at all.

Database transaction logs are there in the case of crash, you can do a
restore by replaying the logs.

Bitcoin stores inside the *database* the entire history of all bitcoin
operations. The fact that database transaction logs are on just means
that the entire history of all database operations, (which, incidentally
aren't exclusively bitcoin trades between users), is also written to the
transaction logs.

 As I understand it, the log format is used only when bootstrapping via 
 IRC, but I really don't know these details.

No, the database transaction logs are a property of databases, and are
not unique to bitcoin at all. The bootstrap via IRC basically does this:

1. uses a hard-coded address in the source to connect
2. gets p2p info and blocks from other nodes on irc
3. writes the acquired info into the database

It just so happens that the way database writes are done is by doing
this:

a. writing to the database log file what exactly is going to be written
to the disk, once that has been committed to hard storage it then...
b. does the database operation

databases do this because if there is a crash, the database may not have
properly flushed to the disk 30 different database operations, so they
never were written... but there is storage of the log which has all the
changes that should be there, so those are replayed.

as that wikipedia page says:

If, after a start, the database is found in an inconsistent state or not
been shut down properly, the database management system reviews the
database logs for uncommitted transactions and rolls back the changes
made by these transactions. Additionally, all transactions that are
already committed but whose changes were not yet materialized in the
database are re-applied. Both are done to ensure atomicity and
durability of transactions.

 Helmut Grohne seemed to know - he mentioned having reverse-engineerd the 
 protocol and reimplemented in Python.  I tried to persuade hime to join 
 me in maintaining Bitcoin - until his own implementation perhaps would 
 eventually mature and be superior.  No luck, unfortunately. :-(

haha, awesome, would be nice to have it in python.


pgpYsSHMp4vGi.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-10 Thread Jonas Smedegaard 

On Mon, Jan 10, 2011 at 02:15:17PM -0500, micah anderson wrote:

On Mon, Jan 10, 2011 at 12:54:58AM -0500, micah anderson wrote:
Transaction logs are great, if you want to do recovery, but I 
wonder if they are really needed. If I recall, they simply grow, 
until you rotate them, unless you set specific DB_CONFIG flags.


Database transaction logs are there in the case of crash, you can do a 
restore by replaying the logs.


Bitcoin stores inside the *database* the entire history of all bitcoin 
operations. The fact that database transaction logs are on just means 
that the entire history of all database operations, (which, 
incidentally aren't exclusively bitcoin trades between users), is also 
written to the transaction logs.


Ok.  I think I get it now.  Thanks for clarifying.

I consider adding a shell wrapper looking something like this:

 set -e
 umask=077
 basedir=~/.bitcoin
 dbfile=$basedir/DB_CONFIG
 cfgfile=$basedir/bitcoin.conf
 [ -d ~/.bitcoin ] || mkdir ~/.bitcoin
 [ -e $dbfile ] || echo 'set_flags DB_LOG_AUTOREMOVE'  $dbfile
 [ -e $cfgfile ] || perl -le 'printrpcpassword=,map{(a..z,A..Z,0..9)[rand 62]}0..9'  
$cfgfile
 exec bitcoind $@

What do you think about that?


 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-09 Thread micah anderson 
On Fri, 7 Jan 2011 20:17:29 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Fri, Jan 07, 2011 at 12:06:37PM -0500, micah anderson wrote:
 On Fri, 7 Jan 2011 17:02:20 +0100, Jonas Smedegaard d...@jones.dk 
 wrote:
  On Fri, Jan 07, 2011 at 10:39:11AM -0500, micah anderson wrote:
  I noticed that you are build-depending on libdb4.7++, I dont know 
  that much about the bdb madness, but I do know that the bitcoin that 
  I compiled from source was compiled against libdb4.8++ and when I 
  try and install your packaged version, it can't read the wallet 
  database.
  
  Is there any specific reason you went with 4.7 instead of 4.8,
 
  Yes.
 
  src/build-unix.txt includes this note:
 
  You need Berkeley DB 4.7.  Don't use 4.8, the database/log* 
  files are incompatible.
 
 This is interesting... I am using DB 4.8 just fine. I suspect that this 
 build-unix.txt is out of date, or just wrong. The database/log 
 files are just transaction logs, and while it is true that they are 
 incompatible, this is normal for ABI differences between berkeley db 
 versions.
 
 
 
  Helmuth Grohne, showing up on IRC just before I released the 
  packaging, claims this might actually be a reason to run away 
  screaming from Bitcoin, as it indicates a disastrous design flaw in 
  relying on the network API on the internal BDB format of 
  (little-endian!) v4.7.
 
 talking about irc gave me an idea. I stopped over to #bitcoin-dev, and 
 asked them what the deal was:
 [IRC conversation snipped]
 so it seems there is no particular reason to use 4.7, its just what 
 their pre-compiled binary versions use, and the only reason that they 
 dont move to 4.8 is because nobody has bothered to try it on mac and 
 windows.
 
 As a test, I backed up my .bitcoin/ and then ran your version, with 
 libdb4.7++ and start it up. It creates a wallet and I let it download 
 all the current blocks. I then stop it, run a self-compiled bitcoin, 
 linked against libdb4.8++, it converts the database to 4.8 format, and 
 things continue to work as normal.
 
 Thanks a lot for your investigation!
 
 So what really is the issue here is that _at_ _each_ _runtime_ 
 _environment_ are the db files incompatible.

http://download.oracle.com/docs/cd/E17276_01/html/programmer_reference/changelog_4_8.html#id1655659

Database or Log File On-Disk Format Changes:

   1.  The log file format changed in 4.8.

So it seems that its just the db4.8 logs are not backward compatible
with 4.7. Transaction logs are great, if you want to do recovery, but I
wonder if they are really needed. If I recall, they simply grow, until
you rotate them, unless you set specific DB_CONFIG flags.

 @Micah: Could you please test if upgrading works like this:
 
1) Move aside your precious .bitcoin folder
2) Start the current Debian Bitcoin (or upstream binary)
3) Let Bitcoin run until its db is fully init'ed (i.e. dir is ~150MB)
3) Quit Bitcoin
4) Install packages libdb4.8-util and db4.7-util

you mean db4.8-util and db4.7-util :)

5) run the following commands:

I think that the following commands should only be run after I've
stopped the running bitcoind:

  db4.7_recover -h ~/.bitcoin
  db4.8_upgrade -h ~/.bitcoin wallet.dat
 
6) Start your locally compiled Bitcoin
7) Tell me if it worked out or not :-)

It seems like it works, here is the log:

mi...@algae:~$ mv .bitcoin/ .bitcoin_bak
mi...@algae:~$ /usr/bin/bitcoind 
bitcoin server starting
mi...@algae:~$ Warning: To use bitcoind, you must set rpcpassword=password
in the configuration file: /home/micah/.bitcoin/bitcoin.conf
If the file does not exist, create it with owner-readable-only file permissions.
mi...@algae:~$ cp .bitcoin_bak/bitcoin.conf .bitcoin
mi...@algae:~$ /usr/bin/bitcoind 
bitcoin server starting
...
mi...@algae:~$ du -sch .bitcoin
168M.bitcoin
168Mtotal
mi...@algae:~$ /usr/bin/bitcoind stop
mi...@algae:~$ db4.7_recover -h ~/.bitcoin
mi...@algae:~$ db4.8_upgrade -h ~/.bitcoin wallet.dat
mi...@algae:~$ working/bitcoin-0.3.18/src/bitcoind 
bitcoin server starting
mi...@algae:~$ working/bitcoin-0.3.18/src/bitcoind getinfo
{
version : 31800,
balance : 0.,
blocks : 75297,
connections : 9,
proxy : ,
generate : false,
genproclimit : -1,
difficulty : 511.77353426,
hashespersec : 0,
testnet : false,
keypoololdest : 1294638747,
paytxfee : 0.,
errors : 
}
mi...@algae:~$

 I will now compile a new Bitcoin against Berkeley DB 4.8, with a README 
 mentioning essential parts of above - blindly hoping that it works.

I checked with the bitcoin community, and found there are quite a number
of people using bdb4.8 on gentoo, redhat, fedora and freebsd.

 Please be careful to test against current release, not the version I am 
 about to release :-)

Yep, I used 0.3.19~dfsg-3~0jones1 for this test.

micah


pgpV7KwMyNm3i.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-07 Thread micah anderson 
On Thu, 6 Jan 2011 17:45:57 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Thu, Jan 06, 2011 at 04:46:30PM +0100, Jonas Smedegaard wrote:
 On Thu, Jan 06, 2011 at 10:33:35AM -0500, micah anderson wrote:
 Do you have a repository somewhere with any work done thus far? 
 I've tried out the cli for a while and it seems like a worthwhile 
 package to make without the gui bits.
 
 Sorry - I thought the news was already filed here in the bugreport:
 
 The final official packaging was uploaded ultimo december, now 
 awaiting ftpmaster approval: 
 http://ftp-master.debian.org/new/bitcoin_0.3.19~dfsg-2.html
 
 
 It is also unofficially available using this APT line:
 
   deb http://debian.jones.dk/ unstable freedombox

Great! 

I noticed that you are build-depending on libdb4.7++, I dont know that
much about the bdb madness, but I do know that the bitcoin that I
compiled from source was compiled against libdb4.8++ and when I try and
install your packaged version, it can't read the wallet database.

Is there any specific reason you went with 4.7 instead of 4.8, or was it
just what your build environment had? My guess is that it would be best
to switch to 4.8 right away, before this is accepted into Debian,
because the transition from 4.7 to 4.8 is not going to be trivial
(although, in theory... nuking the transaction logs, and removing the
database environment will make it work, it will be a pain and somewhat
scary for people's wallets). 

micah


pgpbVMCFWuREZ.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-07 Thread Jonas Smedegaard 

On Fri, Jan 07, 2011 at 10:39:11AM -0500, micah anderson wrote:
I noticed that you are build-depending on libdb4.7++, I dont know that 
much about the bdb madness, but I do know that the bitcoin that I 
compiled from source was compiled against libdb4.8++ and when I try and 
install your packaged version, it can't read the wallet database.


Is there any specific reason you went with 4.7 instead of 4.8,


Yes.

src/build-unix.txt includes this note:

You need Berkeley DB 4.7.  Don't use 4.8, the database/log* files 
are incompatible.


Helmuth Grohne, showing up on IRC just before I released the packaging, 
claims this might actually be a reason to run away screaming from 
Bitcoin, as it indicates a disastrous design flaw in relying on the 
network API on the internal BDB format of (little-endian!) v4.7.


In other words: Bitcoin must eternally use BDB v4.7, and cannot work on 
e.g. PowerPC.



 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-07 Thread micah anderson 
On Fri, 7 Jan 2011 17:02:20 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Fri, Jan 07, 2011 at 10:39:11AM -0500, micah anderson wrote:
 I noticed that you are build-depending on libdb4.7++, I dont know that 
 much about the bdb madness, but I do know that the bitcoin that I 
 compiled from source was compiled against libdb4.8++ and when I try and 
 install your packaged version, it can't read the wallet database.
 
 Is there any specific reason you went with 4.7 instead of 4.8,
 
 Yes.
 
 src/build-unix.txt includes this note:
 
 You need Berkeley DB 4.7.  Don't use 4.8, the database/log* files 
 are incompatible.

This is interesting... I am using DB 4.8 just fine. I suspect that this
build-unix.txt is out of date, or just wrong. The database/log files
are just transaction logs, and while it is true that they are
incompatible, this is normal for ABI differences between berkeley db
versions. 


 
 Helmuth Grohne, showing up on IRC just before I released the
 packaging, claims this might actually be a reason to run away
 screaming from Bitcoin, as it indicates a disastrous design flaw in
 relying on the network API on the internal BDB format of
 (little-endian!) v4.7.

talking about irc gave me an idea. I stopped over to #bitcoin-dev, and
asked them what the deal was:

11:46:01  hacim i noticed in src/build-unix.txt, it says to use Berkeley DB 
4.7
11:46:10  hacim You need Berkeley DB 4.7.  Don't use 4.8, the 
database/log* files are incompatible
11:46:15  hacim however, I'm using 4.8 just fine...
11:46:15  ArtForz yes, use bdb 4.7
11:46:33  ArtForz well, now your DB is auto-upgraded to 4.8
11:46:42  hacim the database/log* files are just transaction logs
11:46:51  ArtForz = 4.7 can't read it anymore
11:47:01  hacim they are of course incompatible with 4.7, this is the 
definition of the ABI change :)
11:47:13  gavinandresen As long as you don't reinstall a precompiled bitcoin 
you'll be fine.
11:47:23  ArtForz yep
11:48:03  hacim ok, so there is nothing about the 4.7 libraries, such as 
relying on the network API in the internal bdb format of little-endian 4.7?
11:48:10  ArtForz nope
11:48:29  ArtForz it's just that 4.8 isnt backwards compatible with 4.7 and 
binary builds use 4.7
11:48:31  hacim so, for new installs, why would one chose to use 4.7, when 
you will need to upgrade anyways?
11:48:48  hacim eventually those binary builds will need to use 4.8, and 
people will have to transition
11:48:54  ArtForz why?
11:49:04  hacim if the history of bdb is to be any indicator of the future...
11:49:13  ArtForz so far the binary builds are 4.7, so going to selfcompiled 
w/ 4.8 is a one-way street
11:49:14  hacim do you know anyone using bdb4.1?
11:49:52  ArtForz when binary build switches to 4.8, recommended version for 
compiling will switch to 4.8 (duh)
11:49:59  hacim why would anyone want to use 4.7 right now is beyond me
11:50:14  ArtForz no one WANTS to, everyone using a precompiled binary HAS to
11:50:20  hacim ArtForz: the 'binary build' meaning the binaries offered on 
the website?
11:50:56  gavinandresen hacim: you want to volunteer to organize an effort to 
make sure 4.8 is supported on windows mac and all the linux flavors 
  people are using?
11:52:59  hacim gavinandresen: it works fine on three different linux flavors 
that I've tried this morning
11:53:07  gavinandresen hacim: or, to say it another way:  bitcoin uses 4.7 
because two years ago that is what Satoshi decided to use, and nobody has 
  put in the effort needed to upgrade.  It just hasn't 
been a priority.
11:53:32  gavinandresen hacim: great!  so create a patch and recruit some 
people to see if it works on mac and windows.
11:53:32  hacim gavinandresen: it doesn't bother me that other people use 4.7

so it seems there is no particular reason to use 4.7, its just what
their pre-compiled binary versions use, and the only reason that they
dont move to 4.8 is because nobody has bothered to try it on mac and
windows. 

As a test, I backed up my .bitcoin/ and then ran your version, with
libdb4.7++ and start it up. It creates a wallet and I let it download
all the current blocks. I then stop it, run a self-compiled bitcoin,
linked against libdb4.8++, it converts the database to 4.8 format, and
things continue to work as normal. 

micah


pgpRk8yliHSoA.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-07 Thread Jonas Smedegaard 

On Fri, Jan 07, 2011 at 12:06:37PM -0500, micah anderson wrote:
On Fri, 7 Jan 2011 17:02:20 +0100, Jonas Smedegaard d...@jones.dk 
wrote:

On Fri, Jan 07, 2011 at 10:39:11AM -0500, micah anderson wrote:
I noticed that you are build-depending on libdb4.7++, I dont know 
that much about the bdb madness, but I do know that the bitcoin that 
I compiled from source was compiled against libdb4.8++ and when I 
try and install your packaged version, it can't read the wallet 
database.


Is there any specific reason you went with 4.7 instead of 4.8,

Yes.

src/build-unix.txt includes this note:

You need Berkeley DB 4.7.  Don't use 4.8, the database/log* 
files are incompatible.


This is interesting... I am using DB 4.8 just fine. I suspect that this 
build-unix.txt is out of date, or just wrong. The database/log 
files are just transaction logs, and while it is true that they are 
incompatible, this is normal for ABI differences between berkeley db 
versions.





Helmuth Grohne, showing up on IRC just before I released the 
packaging, claims this might actually be a reason to run away 
screaming from Bitcoin, as it indicates a disastrous design flaw in 
relying on the network API on the internal BDB format of 
(little-endian!) v4.7.


talking about irc gave me an idea. I stopped over to #bitcoin-dev, and 
asked them what the deal was:

[IRC conversation snipped]
so it seems there is no particular reason to use 4.7, its just what 
their pre-compiled binary versions use, and the only reason that they 
dont move to 4.8 is because nobody has bothered to try it on mac and 
windows.


As a test, I backed up my .bitcoin/ and then ran your version, with 
libdb4.7++ and start it up. It creates a wallet and I let it download 
all the current blocks. I then stop it, run a self-compiled bitcoin, 
linked against libdb4.8++, it converts the database to 4.8 format, and 
things continue to work as normal.


Thanks a lot for your investigation!

So what really is the issue here is that _at_ _each_ _runtime_ 
_environment_ are the db files incompatible.


I know that problem already from Netatalk, which also until recently did 
not support seemless upgrading.


I'll immediately rebuild using a newer bdb, and also throw in a README 
based on that used in netatalk until recently - and would appreciate if 
you could verify if manual upgrading/downgrading works using that - to 
prepare for future version bumps.



 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-07 Thread Jonas Smedegaard 

On Fri, Jan 07, 2011 at 08:17:29PM +0100, Jonas Smedegaard wrote:
So what really is the issue here is that _at_ _each_ _runtime_ 
_environment_ are the db files incompatible.


I know that problem already from Netatalk, which also until recently 
did not support seemless upgrading.


I'll immediately rebuild using a newer bdb, and also throw in a README 
based on that used in netatalk until recently - and would appreciate if 
you could verify if manual upgrading/downgrading works using that - to 
prepare for future version bumps.


Looking some more at this upgrade issue, it seems to not be possible to 
use the netatalk scripts.


@Micah: Could you please test if upgrading works like this:

  1) Move aside your precious .bitcoin folder
  2) Start the current Debian Bitcoin (or upstream binary)
  3) Let Bitcoin run until its db is fully init'ed (i.e. dir is ~150MB)
  3) Quit Bitcoin
  4) Install packages libdb4.8-util and db4.7-util
  5) run the following commands:

db4.7_recover -h ~/.bitcoin
db4.8_upgrade -h ~/.bitcoin wallet.dat

  6) Start your locally compiled Bitcoin
  7) Tell me if it worked out or not :-)

I will now compile a new Bitcoin against Berkeley DB 4.8, with a README 
mentioning essential parts of above - blindly hoping that it works.


Please be careful to test against current release, not the version I am 
about to release :-)


 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-06 Thread micah anderson 
On Mon, 13 Dec 2010 14:55:18 +0100, Jonas Smedegaard d...@jones.dk wrote:
 On Sat, Dec 11, 2010 at 08:48:46PM -0500, micah wrote:
 
   Also I noticed that upstream released 0.3.17 this week.
 
 And it looks like 0.3.18 is now there...
 
 Jumping on the bandwagon here, but without any time myself to help in
 the packaging effort :P
 
 Package json-spirit now uploaded to NEW queue targeted unstable.
 
 Package bitcoin now updated to upstream 0.3.18 release.  Still needs a 
 few adjustments to compile - and anyway awaits ftpmaster approval of 
 json-spirit.
 
 Thanks for the encouragements, everyone!

Looks like there is an Ubuntu PPA for bitcoin, version 0.3.19 is up now
with builds for maverick and lucid:

https://launchpad.net/~stretch/+archive/bitcoin

One thing I've noted is that upstream is moving quite fast, bitcoin is
in beta and sometimes there are frequent releases. So, it might require
relatively quick turn over in packaging.

Do you have a repository somewhere with any work done thus far? I've
tried out the cli for a while and it seems like a worthwhile package to
make without the gui bits.

micah



pgpOypYrBwOsy.pgp
Description: PGP signature

Bug#578157: and another!

2011-01-06 Thread Jonas Smedegaard 

On Thu, Jan 06, 2011 at 10:33:35AM -0500, micah anderson wrote:
Do you have a repository somewhere with any work done thus far? I've 
tried out the cli for a while and it seems like a worthwhile package to 
make without the gui bits.


Sorry - I thought the news was already filed here in the bugreport:

The final official packaging was uploaded ultimo december, now awaiting 
ftpmaster approval: 
http://ftp-master.debian.org/new/bitcoin_0.3.19~dfsg-2.html



It is also unofficially available using this APT line:

  deb http://debian.jones.dk/ unstable freedombox

...or manually downloaded here:

  http://debian.jones.dk/pkg/freedombox/bitcoin/

...or git cloned from here:

  git://git.debian.org/git/collab-maint/bitcoin



regards,

  - Jonas

--
  * Jonas Smedegaard - idealist  Internet-arkitekt
  * Tlf.: +45 40843136  Website: http://dr.jones.dk/

  [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2011-01-06 Thread Jonas Smedegaard 

On Thu, Jan 06, 2011 at 04:46:30PM +0100, Jonas Smedegaard wrote:

On Thu, Jan 06, 2011 at 10:33:35AM -0500, micah anderson wrote:
Do you have a repository somewhere with any work done thus far? 
I've tried out the cli for a while and it seems like a worthwhile 
package to make without the gui bits.


Sorry - I thought the news was already filed here in the bugreport:

The final official packaging was uploaded ultimo december, now 
awaiting ftpmaster approval: 
http://ftp-master.debian.org/new/bitcoin_0.3.19~dfsg-2.html



It is also unofficially available using this APT line:

 deb http://debian.jones.dk/ unstable freedombox


...and same place now also backported to squeeze, and built both for 
i486 and amd64 (tell me if anyone needs powerpc or arm as well).



 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2010-12-14 Thread micah anderson 
On Mon, 13 Dec 2010 14:55:18 +0100, Jonas Smedegaard d...@jones.dk wrote:

 Thanks for the encouragements, everyone!

Go Jonas go!!

Maybe we can send you some bitcoins in thanks :)

micah


pgp01uTZviQ7x.pgp
Description: PGP signature

Bug#578157: and another!

2010-12-13 Thread Jonas Smedegaard 

On Sat, Dec 11, 2010 at 08:48:46PM -0500, micah wrote:



 Also I noticed that upstream released 0.3.17 this week.


And it looks like 0.3.18 is now there...

Jumping on the bandwagon here, but without any time myself to help in
the packaging effort :P


Package json-spirit now uploaded to NEW queue targeted unstable.

Package bitcoin now updated to upstream 0.3.18 release.  Still needs a 
few adjustments to compile - and anyway awaits ftpmaster approval of 
json-spirit.


Thanks for the encouragements, everyone!


 - Jonas

--
 * Jonas Smedegaard - idealist  Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: Digital signature

Bug#578157: and another!

2010-12-11 Thread micah 

  Also I noticed that upstream released 0.3.17 this week.

And it looks like 0.3.18 is now there... 

Jumping on the bandwagon here, but without any time myself to help in
the packaging effort :P

micah






-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org