Hello,
I tried connecting 2 endpoints using the new uDAPL library.
The connection fails with an invalid route option.
Attached is the log with DAT_DBG_TYPE=0xffff and DAPL_DBG_TYPE=0xffff.
I traced the call to ib_at_route_by_ip(),
It seems like all the arguments (dst_ip, src_ip, r_qual..) are 0x00.
I have also attached my dat.conf, lsmod, /etc/ibhosts, and /etc/hosts.
Could you please direct me where to look?
Thanks
Aniruddha
DAT Registry: Started (dat_init)
DAT Registry: static registry file </etc/dat.conf>
DAT Registry: token
type eor
value <>
DAT Registry: token
type string
value <IB1>
DAT Registry: token
type string
value <u1.2>
DAT Registry: token
type string
value <nonthreadsafe>
DAT Registry: token
type string
value <default>
DAT Registry: token
type string
value </usr/local/lib/libdapl.so>
DAT Registry: token
type string
value <mv_dapl.1.2>
DAT Registry: token
type string
value <mthca0 1>
DAT Registry: token
type string
value < >
DAT Registry: token
type eor
value <>
DAT Registry: entry
ia_name IB1
api_version
type 0x0
major.minor 1.2
is_thread_safe 0
is_default 1
lib_path /usr/local/lib/libdapl.so
provider_version
id mv_dapl
major.minor 1.2
ia_params mthca0 1
DAT Registry: loading provider for IB1
DAT Registry: token
type eof
value <>
DAPL: NOT Setting Loopback
dapl_ib_init:
cm_thread_init(1348)
at_thread_init(1348)
Initializing xprt library
RPC: setting up UDP transport...
RPC: created transport 0x804ad78 with 16 slots
TRACE: Created xpt...
DAT Registry: dat_ia_openv (IB1,1:2,0) called
DAT Registry: IA IB1, trying to load library /usr/local/lib/libdapl.so
at_thread(1348,0xb7751bb0): ENTER: at_fd 3
DAT Registry: dat_registry_add_provider (IB1,1:2,0)
cm_thread(1348,0xb7f52bb0): ENTER: cm_fd 4
dapl_ia_open (IB1, 32, 0x804bdac, 0x804bda4)
open_hca: mthca0 - 0x804c868
open_hca: Found dev mthca0 0002c901081e7470
open_hca: GID subnet fe80000000000000 id 0002c901081e7471
get_hca_addr: ips_by_gid ret 0 at_rec 0xbfad58c0 -> id 72
ip_comp_handler: at_rec 0xbfad58c0 ->id 72 id 72 rec_num 1 c0a0a0a
at_thread: callback woke
cq_thread_init(0x804c868)
open_hca: mthca0, port 1, AF_INET 10.10.10.12 INLINE_MAX=128
cq_thread: ENTER hca 0x804c868
query_hca: mthca0 AF_INET 10.10.10.12
query_hca: (0.180000) ep 65520 ep_q 65535 evd 65408 evd_q 65535
query_hca: msg 2147483648 rdma 2147483648 iov 60 lmr 131056 rmr 0
setup_async_cb: ia 0x8051618 type 0 handle (nil) cb 0xb7f87348 ctx 0x80516a0
setup_async_cb: ia 0x8051618 type 1 handle (nil) cb 0xb7f87188 ctx 0x80516a0
setup_async_cb: ia 0x8051618 type 3 handle (nil) cb 0xb7f8727c ctx 0x8051618
dapl_ia_open () returns 0x0
dat_set_handle 0x8051618 to 1
dat_get_ia_handle from 1 to 0x8051618
dapl_ia_query (0x8051618, (nil), 0x7ffffffff, 0x804bdb8, 0x3ffffff, (nil))
query_hca: mthca0 AF_INET 10.10.10.12
query_hca: (0.180000) ep 65520 ep_q 65535 evd 65408 evd_q 65535
query_hca: msg 2147483648 rdma 2147483648 iov 60 lmr 131056 rmr 0
dapl_ia_query () returns 0x0
dat_get_ia_handle from 1 to 0x8051618
dapl_pz_create (0x8051618, 0x804bda8)
pd_alloc: pd_handle=0x8050cc0
dat_get_ia_handle from 1 to 0x8051618
dat_get_ia_handle from 1 to 0x8051618
dat_get_ia_handle from 1 to 0x8051618
dapl_evd_create (0x8051618, 32, 0x8053308, 0x20, 0x805329c)
dapl_ia_query (0x8051618, (nil), 0x0, (nil), 0x3ffffff, (nil))
dapl_ia_query () returns 0x0
dapls_ib_cq_alloc: evd 0x80533b0 cqlen=32
dapls_ib_cq_alloc: new_cq 0x80534a8 cqlen=63
setup_async_cb: ia 0x8051618 type 2 handle 0x80533b0 cb 0xb7f87624 ctx 0x80533b0
dapl_evd_create () returns 0x0
dat_get_ia_handle from 1 to 0x8051618
dapl_evd_create (0x8051618, 32, (nil), 0xa0, 0x80532a0)
dapl_ia_query (0x8051618, (nil), 0x0, (nil), 0x3ffffff, (nil))
dapl_ia_query () returns 0x0
dapls_ib_cq_alloc: evd 0x8055008 cqlen=32
dapls_ib_cq_alloc: new_cq 0x8053f40 cqlen=63
setup_async_cb: ia 0x8051618 type 2 handle 0x8055008 cb 0xb7f87624 ctx 0x8055008
dapl_evd_create () returns 0x0
dat_get_ia_handle from 1 to 0x8051618
dapl_evd_create (0x8051618, 32, (nil), 0x40, 0x8053298)
dapl_ia_query (0x8051618, (nil), 0x0, (nil), 0x3ffffff, (nil))
dapl_ia_query () returns 0x0
dapl_evd_create () returns 0x0
dat_get_ia_handle from 1 to 0x8051618
dapl_ep_create (0x8051618, 0x8052fb8, 0x80533b0, 0x8055008, 0x8055af8, (nil), 0x80532a4)
query_hca: MAX msg 2147483648 dto 65535 iov 60 rdma i4,o4
qp_alloc: ia_ptr 0x8051618 ep_ptr 0x8057218 ep_ctx_ptr 0x8057218
qp_alloc: qpn 0x2a0016 sq 16,9 rq 16,4
modify_qp_INIT: pi 0 port 1 acc f
dat_get_ia_handle from 1 to 0x8051618
mr_register: ia=0x8051618, lmr=0x8057f20 va=0x805a008 ln=1024 pv=0x0
mr_register: mr=0x8057fd0 h 3 pd 0x8050cc0 ctx 0x805154c ,lkey=0x804003b, rkey=0x804003b priv=400
dapl_ep_connect (0x8057218, {4294967280.4294967295.4294967295.4294967295}, 4A275800, 0, -1, (nil), 0, 0)
connect: r_SID 1244092416, pdata (nil), plen 0
connect: at_route ret=-1,Invalid argument req_id 0 GID 0000000000000000 0000000000000000
ib_at_route_by_ip Invalid argument
destroy_cm_id: conn 0x80832c8 id 134755424
dapl_ep_connect () returns 0x50000
dapl_evd_dequeue (0x80533b0, 0xbfad5ab0)
dapl_evd_dequeue () returns 0xd0000
dapl_ep_free (0x8057218)
dapl_ep_disconnect (0x8057218, 0)
dapl_ep_disconnect () returns 0x8007002e
dapl_ep_free: Free EP: 0, ep 0x8057218 qp_state 1 qp_handle 80578b0
qp_free: ep_ptr 0x8057218 qp 0x80578b0
dapl_evd_free (0x8055af8)
dapl_evd_free () returns 0x0
dapl_evd_free (0x8055008)
dapl_evd_free () returns 0x0
dapl_evd_free (0x80533b0)
dapl_evd_free () returns 0x0
TRACE: >>> °²ú·hÆú·
TRACE: Destroyed xpt...
DAPL: Stopped (dapl_fini)
dapl_ib_release:
at_thread_destroy(1348)
at_thread_destroy(1348) SIGUSR1 sent
at_thread_destroy: waiting for at_thread
at_thread: SIG? ret=Interrupted system call, destroy=1
at_thread: callback woke
at_thread(1348) EXIT
at_thread_destroy(1348) exit
cm_thread_destroy(1348)
cm_thread_destroy(1348) SIGUSR1 sent
cm_thread_destroy: waiting for cm_thread
cm_thread(1348): ERR Interrupted system call poll
cm_thread(1348) EXIT, cm_list=EMPTY
cm_thread_destroy(1348) exit
DAT Registry: Stopped (dat_fini)
#
# DAT 1.1 and 1.2 configuration file
#
# Each entry should have the following fields:
#
# <ia_name> <api_version> <threadsafety> <default> <lib_path> \
# <provider_version> <ia_params> <platform_params>
#
# Example for openib using the first Mellanox adapter, port 1 and port 2
IB1 u1.2 nonthreadsafe default /usr/local/lib/libdapl.so mv_dapl.1.2 "mthca0 1"
" "
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
10.10.10.11 filer-ib6a.discolab.rutgers.edu filer-ib6a
10.10.10.12 hora-3-ib0.discolab.rutgers.edu hora-3-ib0
Module Size Used by
ib_uat 15244 0
ib_ucm 19596 1
ib_cm 37196 1 ib_ucm
ib_at 28096 1 ib_uat
kdapl 11268 0
ib_uverbs 31632 0
ib_umad 18200 4
ib_ipoib 43396 0
ib_sa 16532 2 ib_at,ib_ipoib
ipv6 267264 18
i2c_dev 13952 0
i2c_core 25216 1 i2c_dev
sunrpc 145476 1
dm_mod 60316 0
video 20228 0
button 10896 0
battery 13700 0
ac 9092 0
uhci_hcd 36624 0
hw_random 9748 0
ib_mthca 111260 0
ib_mad 41248 4 ib_cm,ib_umad,ib_sa,ib_mthca
ib_core 46336 7
ib_cm,ib_uverbs,ib_umad,ib_ipoib,ib_sa,ib_mthca,ib_mad
e1000 105524 0
floppy 63188 0
filer-ib6a 0xfe80000000000000 0x0001730000008461
hora-3-ib0 0xfe80000000000000 0x0002c901081e7471
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
