Hi Phil,
I probably should have tried the latest version first, I picked the
older version that matched what was installed on the system out of
an abundance of caution.
I've installed a modified db-4.8.26. There was some progress in the
dump/restore procedure,
opt2326:/fs/pvfs/pvfs/5810ab5d> type db_dump
db_dump is hashed (/usr/local/pvfs-db-4.8.26/bin/db_dump)
opt2326:/fs/pvfs/pvfs/5810ab5d> db_dump -r \
> -f /tmp/dataspace_attributes.out \
> dataspace_attributes.db
opt2326:/fs/pvfs/pvfs/5810ab5d> echo $?
0
opt2326:/fs/pvfs/pvfs/5810ab5d> db_load -f /tmp/dataspace_attributes.out test.db
opt2326:/fs/pvfs/pvfs/5810ab5d> mv dataspace_attributes.db \
dataspace_attributes.db.bad
opt2326:/fs/pvfs/pvfs/5810ab5d> db_verify -o dataspace_attributes.db
opt2326:/fs/pvfs/pvfs/5810ab5d> echo $?
0
However, when attempting to start the pvfs server it exits immediately
with,
[D 03/18 17:02] PVFS2 Server version 2.8.1 starting.
[E 03/18 17:02] dbpf_dspace_iterate_handles_op_svc: Invalid argument
[E 03/18 17:02] Error adding handle range
4099276460824344803-5124095576030431002 to
filesystem pvfs2-fs
[E 03/18 17:02] Error: Could not initialize server interfaces; aborting.
[E 03/18 17:02] Error: Could not initialize server; aborting.
The logs with EventLogging set to 'all' are attached. I've rechecked
the bdb files, and all verify with no errors.
Doug
At Thu, 18 Mar 2010 15:08:52 -0400,
Phil Carns wrote:
>
> Hi Doug,
>
> I haven't seen db_dump fail altogether like that before. I can make
> some suggestions that you may want to try, though. You should probably
> back up the current corrupted db first to make sure things don't get
> worse.
>
> The first thing I would suggest is to try a newer version of berkeley db
> for db_dump, since you are having to build that tool from scratch
> anyway. Newer versions of bdb can read the same format, and maybe there
> is a chance that the verification/recovery has improved since 4.3.
>
> You can also try db_recover as well (perhaps with -c?). That tool is
> really meant to recover transactions (which is not the issue here), but
> maybe it can shed more light on the verification problem.
>
> Finally in db_dump, you can try -R instead of -r. I would save that as
> a last resort, because I think it may recover unwanted (bogus) data as
> well.
>
> -Phil
>
[D 03/19 09:23] PVFS2 Server version 2.8.1 starting.
[D 03/19 09:23] PINT_encode_initialize
[D 03/19 09:23] lebf_initialize
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] PINT_do_request_commit: commit node 0x7fffbf2f6810
[D 03/19 09:23] node stored at 0
[D 03/19 09:23] clearing tree
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] PINT_do_request_commit: commit node 0x7fffbf2f6810
[D 03/19 09:23] node stored at 0
[D 03/19 09:23] clearing tree
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_req_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_req
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] check_resp_size
[D 03/19 09:23] encode_common
[D 03/19 09:23] lebf_encode_resp
[D 03/19 09:23] lebf_encode_rel
[D 03/19 09:23] Passing tcp://opt-ib-2326:3335 as BMI listen address.
[D 03/19 09:23] BMI_tcp_initialize: Initializing TCP/IP module.
[D 03/19 09:23] BMI_tcp_initialize: TCP/IP module successfully initialized.
[D 03/19 09:23] Server using shm key hint: 1477487503
[D 03/19 09:23] [BMI CONTROL]: BMI_set_info: set_info: 0 option: 11
[D 03/19 09:23] Default socket buffers send:16384 receive:87380
[D 03/19 09:23] Setting socket buffer size for send:0 receive:0
[D 03/19 09:23] Reread socket buffers send:16384 receive:87380
[D 03/19 09:23] [BMI CONTROL]: BMI_set_info: set_info: 0 option: 12
[D 03/19 09:23] Default socket buffers send:16384 receive:87380
[D 03/19 09:23] Setting socket buffer size for send:0 receive:0
[D 03/19 09:23] Reread socket buffers send:16384 receive:87380
[D 03/19 09:23] dbpf_thread_initialize: initialized
[D 03/19 09:23] [SYNC_COALESCE]: dbpf_sync_context_init for context 0 called
[D 03/19 09:23] dbpf_collection_lookup of coll: pvfs2-fs
[D 03/19 09:23] dbpf using default db cache size.
[D 03/19 09:23] dbpf using shm key: 2124054726
[D 03/19 09:23] dbpf_thread_function started
[D 03/19 09:23] collection lookup: version is 0.1.4
[D 03/19 09:23] [SYNC_COALESCE]: dbpf_sync_context_init for context 1 called
[D 03/19 09:23] dbpf collection 1477487453 - Setting handle timeout to
360000000 mic
roseconds
[D 03/19 09:23] - set handle re-use timeout to 360 seconds (ret=0)
[D 03/19 09:23] dbpf collection 1477487453 - Setting cache keywords of
attribute cac
he to dh,
[D 03/19 09:23] Setting dbpf_attr_cache keywords to:
dh,
[D 03/19 09:23] dbpf collection 1477487453 - Setting cache size of attribute
cache t
o 511
[D 03/19 09:23] dbpf collection 1477487453 - Setting maximum elements of
attribute c
ache to 1024
[D 03/19 09:23] dbpf collection 1477487453 - Initialize collection attr. cache
[D 03/19 09:23] There are 1 cacheable keywords registered
[D 03/19 09:23] dbpf_attr_cache_initialize: initialized
[D 03/19 09:23] dbpf collection 1477487453 - Setting collection handle ranges
to 409
9276460824344803-5124095576030431002
[D 03/19 09:23] op_queue add: 0x1b7d9750
[D 03/19 09:23] [DBPF THREAD]: STARTING TROVE SERVICE ROUTINE
(DSPACE_ITERATE_HANDLE
S)
[E 03/19 09:23] dbpf_dspace_iterate_handles_op_svc: Invalid argument
[D 03/19 09:23] [DBPF THREAD]: FINISHED TROVE SERVICE ROUTINE
(DSPACE_ITERATE_HANDLE
S) (ret: -1073742095)
[D 03/19 09:23] op_queue add: 0x1b7d9750
[D 03/19 09:23] trove_dspace_iterate_handles failed
[E 03/19 09:23] Error adding handle range
4099276460824344803-5124095576030431002 to
filesystem pvfs2-fs
[E 03/19 09:23] Error: Could not initialize server interfaces; aborting.
[E 03/19 09:23] Error: Could not initialize server; aborting.
[D 03/19 09:23] *** server shutdown in progress ***
[D 03/19 09:23] [+] halting flow interface [ ... ]
[D 03/19 09:23] [SYNC_COALESCE]: dbpf_sync_context_destroy for context 0 called
[D 03/19 09:23] [-] flow interface [ stopped ]
[D 03/19 09:23] [+] halting bmi interface [ ... ]
[D 03/19 09:23] BMI_tcp_finalize: TCP/IP module finalized.
[D 03/19 09:23] [-] bmi interface [ stopped ]
[D 03/19 09:23] [+] halting storage interface [ ... ]
[D 03/19 09:23] dbpf_thread_function ending
[D 03/19 09:23] dbpf_thread_finalize: finalized
[D 03/19 09:23] dbpf_attr_cache_finalized
[D 03/19 09:23] [-] storage interface [ stopped ]
[D 03/19 09:23] [+] halting encoder interface [ ... ]
[D 03/19 09:23] PINT_encode_finalize
[D 03/19 09:23] [-] encoder interface [ stopped ]
[D 03/19 09:23] [+] halting dist interface [ ... ]
[D 03/19 09:23] [-] dist interface [ stopped ]
[D 03/19 09:23] [*] halting logging interface
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
