Hi,
Okay, I have been trying to get PVFS2 2.8.0 up and running, but I am
not having much luck. After creating the conf file and trying to start
the server, the server immediately exits. There is no error message in
the log. However, when I run the server with the -d flag, I get this
output:
r...@msh-new2:/tmp# /usr/local/pvfs2/sbin/pvfs2-server /etc/pvfs2-fs.conf -d
[S 02/18 17:17] PVFS2 Server on node msh-new2 version 2.8.0 starting...
pvfs2-server: src/common/misc/msgpairarray.sm:281: msgpairarray_post:
Assertion `ret < 0 || msg_p->recv_status.error_code != 0' failed.
Aborted
This happens on both my servers. Here is my conf file:
<Defaults>
UnexpectedRequests 50
EventLogging none
EnableTracing no
LogStamp datetime
BMIModules bmi_ib
FlowModules flowproto_multiqueue
PerfUpdateInterval 1000
ServerJobBMITimeoutSecs 30
ServerJobFlowTimeoutSecs 30
ClientJobBMITimeoutSecs 300
ClientJobFlowTimeoutSecs 300
ClientRetryLimit 5
ClientRetryDelayMilliSecs 2000
PrecreateBatchSize 512
PrecreateLowThreshold 256
StorageSpace /raid/pvfs2-storage
LogFile /tmp/pvfs2-server.log
</Defaults>
<Aliases>
Alias msh-new ib://msh-new:3335
Alias msh-new2 ib://msh-new2:3335
</Aliases>
<Filesystem>
Name pvfs2-fs
ID 1617899705
RootHandle 1048576
FileStuffing yes
<MetaHandleRanges>
Range msh-new 3-2305843009213693953
Range msh-new2 2305843009213693954-4611686018427387904
</MetaHandleRanges>
<DataHandleRanges>
Range msh-new 4611686018427387905-6917529027641081855
Range msh-new2 6917529027641081856-9223372036854775806
</DataHandleRanges>
<StorageHints>
TroveSyncMeta yes
TroveSyncData no
TroveMethod alt-aio
</StorageHints>
</Filesystem>
Please let me know if there is any more information I can provide to
debug this further.
Thanks,
Steven Truelove
Kyle Schochenmaier wrote:
Steven, All -
Attached is a patch which should fix compiling IB support with pvfs2 in 2.8.0.
The patch works against 2.8.0 release as well as CVSHEAD.
There is some more work that is necessary to clean up the build
process, I will try to clean that up today.
diff -Naupr /tmp/pvfs-2.8.0/src/io/bmi/bmi_ib/ib.h src/io/bmi/bmi_ib/ib.h
--- /tmp/pvfs-2.8.0/src/io/bmi/bmi_ib/ib.h 2009-02-11 15:38:06.000000000
-0600
+++ src/io/bmi/bmi_ib/ib.h 2009-02-17 14:00:11.000000000 -0600
@@ -11,6 +11,9 @@
#include <src/io/bmi/bmi-types.h>
#include <src/common/quicklist/quicklist.h>
#include <src/common/gossip/gossip.h>
+#include <pvfs2-debug.h>
+#include <pvfs2-encode-stubs.h>
+#include <pvfs2-types.h>
#ifdef __GNUC__
/* confuses debugger */
Kyle Schochenmaier
On Tue, Feb 17, 2009 at 1:32 PM, Steven Truelove <[email protected]> wrote:
Thanks very much!
Steven
Kyle Schochenmaier wrote:
I just verified it builds in 2.7.1 (and also that the pvfs2-ls error
is still there), and i get the same errors as you on my end with 2.8.0
I will try to figure it out.
Kyle Schochenmaier
On Tue, Feb 17, 2009 at 1:28 PM, Steven Truelove <[email protected]> wrote:
Hi,
Yes, I'm using openib. Here is my configure line:
./configure --prefix=/usr/local/pvfs2
--with-kernel=$HOME/pvfs2/linux2/linux-2.6.24 --with-openib=/usr
--without-bmi-tcp
I had problems with 2.7.1, but not with building it. I have looked at
the code the errors are complaining about, and I must admit I don't see any
changes between 2.7.1 and 2.8.0, so I am quite confused.
Thanks,
Steven Truelove
Kyle Schochenmaier wrote:
Steven -
Does 2.7.0/1 compile for you, I am running 8.10/gcc-4.2.4 here, and
can play with the ib stuff if I get around to pulling the IB libraries
this afternoon.
I can probably replicate your environment since I already have gcc
here and just need to pull the IB stuff, you're using openib?
~Kyle
Kyle Schochenmaier
On Tue, Feb 17, 2009 at 1:04 PM, Steven Truelove <[email protected]> wrote:
Hi,
I have tried to build pvfs2 2.8.0, and I have run into some build errors.
I am building on Ubuntu 8.04 with gcc 4.2.3. Any help that can be offered
would be very much appreciated. The first set of errors were:
CC src/io/bmi/reference-list-server.o
src/io/bmi/reference-list.c: In function 'ref_list_search_str':
src/io/bmi/reference-list.c:134: warning: passing argument 2 of
'qhash_search' d
iscards qualifiers from pointer target type
CC src/io/bmi/bmi_ib/ib-server.o
In file included from ./src/io/bmi/bmi-method-support.h:14,
from src/io/bmi/bmi_ib/ib.c:21:
./src/common/quicklist/quicklist.h: In function 'qlist_exists':
./src/common/quicklist/quicklist.h:195: warning: declaration of 'link'
shadows a
global declaration
/usr/include/unistd.h:757: warning: shadowed declaration is here
src/io/bmi/bmi_ib/ib.c: In function 'ib_check_cq':
src/io/bmi/bmi_ib/ib.c:124: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c:124: error: (Each undeclared identifier is reported
only
once
src/io/bmi/bmi_ib/ib.c:124: error: for each function it appears in.)
src/io/bmi/bmi_ib/ib.c: In function 'post_rr':
src/io/bmi/bmi_ib/ib.c:295: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'encourage_send_waiting_buffer':
src/io/bmi/bmi_ib/ib.c:312: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'encourage_send_incoming_cts':
src/io/bmi/bmi_ib/ib.c:398: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'encourage_recv_incoming':
src/io/bmi/bmi_ib/ib.c:485: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'encourage_rts_done_waiting_buffer':
src/io/bmi/bmi_ib/ib.c:661: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'send_bye':
src/io/bmi/bmi_ib/ib.c:687: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'send_cts':
src/io/bmi/bmi_ib/ib.c:720: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'post_send':
src/io/bmi/bmi_ib/ib.c:831: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'post_recv':
src/io/bmi/bmi_ib/ib.c:969: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use i
n this function)
src/io/bmi/bmi_ib/ib.c: In function 'test_sq':
src/io/bmi/bmi_ib/ib.c:1097: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'test_rq':
src/io/bmi/bmi_ib/ib.c:1162: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'BMI_ib_testcontext':
src/io/bmi/bmi_ib/ib.c:1350: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'BMI_ib_testunexpected':
src/io/bmi/bmi_ib/ib.c:1392: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'ib_close_connection':
src/io/bmi/bmi_ib/ib.c:1689: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'ib_tcp_server_check_new_connections':
src/io/bmi/bmi_ib/ib.c:1837: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'ib_block_for_activity':
src/io/bmi/bmi_ib/ib.c:1873: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: In function 'BMI_ib_initialize':
src/io/bmi/bmi_ib/ib.c:1978: error: 'GOSSIP_BMI_DEBUG_IB' undeclared (first
use
in this function)
src/io/bmi/bmi_ib/ib.c: At top level:
src/io/bmi/bmi_ib/ib.c:2112: warning: initialization from incompatible
pointer t
ype
src/io/bmi/bmi_ib/ib.c:2113: warning: initialization from incompatible
pointer t
ype
src/io/bmi/bmi_ib/ib.c:2114: warning: initialization from incompatible
pointer t
ype
src/io/bmi/bmi_ib/ib.c:2120: warning: initialization from incompatible
pointer t
ype
src/io/bmi/bmi_ib/ib.c:2121: warning: initialization from incompatible
pointer t
ype
src/io/bmi/bmi_ib/ib.c:2122: warning: initialization from incompatible
pointer t
ype
make: *** [src/io/bmi/bmi_ib/ib-server.o] Error 1
I solved this by changing the #if 1 on line 456 of src/io/bmi/bmi_ib/ib.h to
an #if 0. Now I get these errors:
CC src/io/bmi/bmi_ib/util-server.o
In file included from src/io/bmi/bmi_ib/ib.h:12,
from src/io/bmi/bmi_ib/util.c:15:
./src/common/quicklist/quicklist.h: In function 'qlist_exists':
./src/common/quicklist/quicklist.h:195: warning: declaration of 'link'
shadows a global declaration
/usr/include/unistd.h:757: warning: shadowed declaration is here
In file included from src/io/bmi/bmi_ib/util.c:15:
src/io/bmi/bmi_ib/ib.h: At top level:
src/io/bmi/bmi_ib/ib.h:254: error: expected '{' before ',' token
src/io/bmi/bmi_ib/ib.h:254: warning: type defaults to 'int' in declaration
of 'type name'
src/io/bmi/bmi_ib/ib.h:255: error: expected declaration specifiers or '...'
before 'credit'
src/io/bmi/bmi_ib/ib.h:255: warning: data definition has no type or storage
class
src/io/bmi/bmi_ib/ib.h:255: warning: type defaults to 'int' in declaration
of 'endecode_fields_2'
src/io/bmi/bmi_ib/ib.h:266: error: expected '{' before ',' token
src/io/bmi/bmi_ib/ib.h:266: warning: type defaults to 'int' in declaration
of 'type name'
src/io/bmi/bmi_ib/ib.h:267: error: expected declaration specifiers or '...'
before 'c'
src/io/bmi/bmi_ib/ib.h:268: error: expected declaration specifiers or '...'
before 'bmi_tag'
src/io/bmi/bmi_ib/ib.h:269: error: expected declaration specifiers or '...'
before '__pad'
src/io/bmi/bmi_ib/ib.h:269: warning: data definition has no type or storage
class
src/io/bmi/bmi_ib/ib.h:269: warning: type defaults to 'int' in declaration
of 'endecode_fields_4'
src/io/bmi/bmi_ib/ib.h:282: error: expected '{' before ',' token
src/io/bmi/bmi_ib/ib.h:282: warning: type defaults to 'int' in declaration
of 'type name'
src/io/bmi/bmi_ib/ib.h:283: error: expected declaration specifiers or '...'
before 'c'
src/io/bmi/bmi_ib/ib.h:284: error: expected declaration specifiers or '...'
before 'bmi_tag'
src/io/bmi/bmi_ib/ib.h:285: error: expected declaration specifiers or '...'
before '__pad'
src/io/bmi/bmi_ib/ib.h:286: error: expected declaration specifiers or '...'
before 'mop_id'
src/io/bmi/bmi_ib/ib.h:287: error: expected declaration specifiers or '...'
before 'tot_len'
src/io/bmi/bmi_ib/ib.h:287: warning: data definition has no type or storage
class
src/io/bmi/bmi_ib/ib.h:287: warning: type defaults to 'int' in declaration
of 'endecode_fields_6'
src/io/bmi/bmi_ib/ib.h:306: error: expected '{' before ',' token
src/io/bmi/bmi_ib/ib.h:306: warning: type defaults to 'int' in declaration
of 'type name'
src/io/bmi/bmi_ib/ib.h:307: error: expected declaration specifiers or '...'
before 'c'
src/io/bmi/bmi_ib/ib.h:308: error: expected declaration specifiers or '...'
before 'rts_mop_id'
src/io/bmi/bmi_ib/ib.h:309: error: expected declaration specifiers or '...'
before 'buflist_tot_len'
src/io/bmi/bmi_ib/ib.h:310: error: expected declaration specifiers or '...'
before 'buflist_num'
src/io/bmi/bmi_ib/ib.h:310: warning: data definition has no type or storage
class
src/io/bmi/bmi_ib/ib.h:310: warning: type defaults to 'int' in declaration
of 'endecode_fields_5'
src/io/bmi/bmi_ib/ib.h:321: error: expected '{' before ',' token
src/io/bmi/bmi_ib/ib.h:321: warning: type defaults to 'int' in declaration
of 'type name'
src/io/bmi/bmi_ib/ib.h:322: error: expected declaration specifiers or '...'
before 'c'
src/io/bmi/bmi_ib/ib.h:323: error: expected declaration specifiers or '...'
before 'mop_id'
src/io/bmi/bmi_ib/ib.h:323: warning: data definition has no type or storage
class
src/io/bmi/bmi_ib/ib.h:323: warning: type defaults to 'int' in declaration
of 'endecode_fields_3'
src/io/bmi/bmi_ib/ib.h:419: warning: 'enum PVFS_io_type' declared inside
parameter list
src/io/bmi/bmi_ib/ib.h:419: warning: its scope is only this definition or
declaration, which is probably not what you want
make: *** [src/io/bmi/bmi_ib/util-server.o] Error 1
--
Steven Truelove
Array Systems Computing, Inc.
1120 Finch Avenue West, 7th Floor
Toronto, Ontario
M3J 3H7
CANADA
http://www.array.ca
[email protected]
Phone: (416) 736-0900 x307
Fax: (416) 736-4715
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
--
Steven Truelove
Array Systems Computing, Inc.
1120 Finch Avenue West, 7th Floor
Toronto, Ontario
M3J 3H7
CANADA
http://www.array.ca
[email protected]
Phone: (416) 736-0900 x307
Fax: (416) 736-4715
--
Steven Truelove
Array Systems Computing, Inc.
1120 Finch Avenue West, 7th Floor
Toronto, Ontario
M3J 3H7
CANADA
http://www.array.ca
[email protected]
Phone: (416) 736-0900 x307
Fax: (416) 736-4715
--
Steven Truelove
Array Systems Computing, Inc.
1120 Finch Avenue West, 7th Floor
Toronto, Ontario
M3J 3H7
CANADA
http://www.array.ca
[email protected]
Phone: (416) 736-0900 x307
Fax: (416) 736-4715
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
