OK. So this works for me with master commit bdaac2d619d603f53a16c07f9d7bd47751137c4c on Centos 7.5.1804.
I cloned the repo and ran './install-deps.sh' and './do_cmake.sh -DWITH_FIO=ON' then 'make all'. # find ./lib -iname '*.so*' | xargs nm -AD 2>&1 | grep _ZTIN13PriorityCache8PriCacheE ./lib/libfio_ceph_objectstore.so:00000000018f72d0 V _ZTIN13PriorityCache8PriCacheE # LD_LIBRARY_PATH=./lib ./bin/fio --enghelp=libfio_ceph_objectstore.so conf : Path to a ceph configuration file oi_attr_len : Set OI(aka '_') attribute to specified length snapset_attr_len : Set 'snapset' attribute to specified length _fastinfo_omap_len : Set '_fastinfo' OMAP attribute to specified length pglog_simulation : Enables PG Log simulation behavior pglog_omap_len : Set pglog omap entry to specified length pglog_dup_omap_len : Set duplicate pglog omap entry to specified length single_pool_mode : Enables the mode when all jobs run against the same pool preallocate_files : Enables/disables file preallocation (touch and resize) on init So my result above matches your result on ubuntu but not on centos. It looks to me like we used to define in libceph-common but currently it's defined in libfio_ceph_objectstore.so. For reasons that are unclear you are seeing the old behaviour. Why this is and why it isn't working as designed is not clear to me but I suspect if you clone the repo again and build from scratch (maybe in a different directory if you wish to keep debugging, see below) you should get a working result. Could you try that as a test? If, on the other hand, you wish to keep debugging your current environment I'd suggest looking at the output of the following command as it may shed further light on the issue. # LD_DEBUG=all LD_LIBRARY_PATH=./lib ./bin/fio --enghelp=libfio_ceph_objectstore.so 'LD_DEBUG=lib' may suffice but that's difficult to judge without knowing what the problem is. I still suspect somehow you have mis-matched libraries and, if that's the case, it's probably not worth pursuing. If you can give me specific steps so I can reproduce this from a freshly cloned tree I'd be happy to look further into it. Good luck. On Thu, Apr 18, 2019 at 7:00 PM Brad Hubbard <bhubb...@redhat.com> wrote: > > Let me try to reproduce this on centos 7.5 with master and I'll let > you know how I go. > > On Thu, Apr 18, 2019 at 3:59 PM Can Zhang <c...@canx.me> wrote: > > > > Using the commands you provided, I actually find some differences: > > > > On my CentOS VM: > > ``` > > # sudo find ./lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > _ZTIN13PriorityCache8PriCacheE > > ./libceph-common.so:000000000221cc08 V _ZTIN13PriorityCache8PriCacheE > > ./libceph-common.so.0:000000000221cc08 V _ZTIN13PriorityCache8PriCacheE > > ./libfio_ceph_objectstore.so: U > > _ZTIN13PriorityCache8PriCacheE > > ``` > > ``` > > # ldd libfio_ceph_objectstore.so |grep common > > libceph-common.so.0 => /root/ceph/build/lib/libceph-common.so.0 > > (0x00007fd13f3e7000) > > ``` > > On my Ubuntu VM: > > ``` > > $ sudo find ./lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > _ZTIN13PriorityCache8PriCacheE > > ./libfio_ceph_objectstore.so:00000000019d13e0 V > > _ZTIN13PriorityCache8PriCacheE > > ``` > > ``` > > $ ldd libfio_ceph_objectstore.so |grep common > > libceph-common.so.0 => > > /home/can/work/ceph/build/lib/libceph-common.so.0 (0x00007f024a89e000) > > ``` > > > > Notice the "U" and "V" from nm results. > > > > > > > > > > Best, > > Can Zhang > > > > On Thu, Apr 18, 2019 at 9:36 AM Brad Hubbard <bhubb...@redhat.com> wrote: > > > > > > Does it define _ZTIN13PriorityCache8PriCacheE ? If it does, and all is > > > as you say, then it should not say that _ZTIN13PriorityCache8PriCacheE > > > is undefined. Does ldd show that it is finding the libraries you think > > > it is? Either it is finding a different version of that library > > > somewhere else or the version you have may not define that symbol. > > > > > > On Thu, Apr 18, 2019 at 11:12 AM Can Zhang <c...@canx.me> wrote: > > > > > > > > It's already in LD_LIBRARY_PATH, under the same directory of > > > > libfio_ceph_objectstore.so > > > > > > > > > > > > $ ll lib/|grep libceph-common > > > > lrwxrwxrwx. 1 root root 19 Apr 17 11:15 libceph-common.so -> > > > > libceph-common.so.0 > > > > -rwxr-xr-x. 1 root root 211853400 Apr 17 11:15 libceph-common.so.0 > > > > > > > > > > > > > > > > > > > > Best, > > > > Can Zhang > > > > > > > > On Thu, Apr 18, 2019 at 7:00 AM Brad Hubbard <bhubb...@redhat.com> > > > > wrote: > > > > > > > > > > On Wed, Apr 17, 2019 at 1:37 PM Can Zhang <c...@canx.me> wrote: > > > > > > > > > > > > Thanks for your suggestions. > > > > > > > > > > > > I tried to build libfio_ceph_objectstore.so, but it fails to load: > > > > > > > > > > > > ``` > > > > > > $ LD_LIBRARY_PATH=./lib ./bin/fio > > > > > > --enghelp=libfio_ceph_objectstore.so > > > > > > > > > > > > fio: engine libfio_ceph_objectstore.so not loadable > > > > > > IO engine libfio_ceph_objectstore.so not found > > > > > > ``` > > > > > > > > > > > > I managed to print the dlopen error, it said: > > > > > > > > > > > > ``` > > > > > > dlopen error: ./lib/libfio_ceph_objectstore.so: undefined symbol: > > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > > > > > $ c++filt _ZTIN13PriorityCache8PriCacheE > > > > > typeinfo for PriorityCache::PriCache > > > > > > > > > > $ sudo find /lib* /usr/lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > /usr/lib64/ceph/libceph-common.so:00000000008edab0 V > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > /usr/lib64/ceph/libceph-common.so.0:00000000008edab0 V > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > > > > > It needs to be able to find libceph-common, put it in your path or > > > > > preload it. > > > > > > > > > > > ``` > > > > > > > > > > > > I found a not-so-relevant > > > > > > issue(https://tracker.ceph.com/issues/38360), the error seems to be > > > > > > caused by mixed versions. My build environment is CentOS 7.5.1804 > > > > > > with > > > > > > SCL devtoolset-7, and ceph is latest master branch. Does someone > > > > > > know > > > > > > about the symbol? > > > > > > > > > > > > > > > > > > Best, > > > > > > Can Zhang > > > > > > > > > > > > Best, > > > > > > Can Zhang > > > > > > > > > > > > > > > > > > On Tue, Apr 16, 2019 at 8:37 PM Igor Fedotov <ifedo...@suse.de> > > > > > > wrote: > > > > > > > > > > > > > > Besides already mentioned store_test.cc one can also use ceph > > > > > > > objectstore fio plugin > > > > > > > (https://github.com/ceph/ceph/tree/master/src/test/fio) to access > > > > > > > standalone BlueStore instance from FIO benchmarking tool. > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Igor > > > > > > > > > > > > > > On 4/16/2019 7:58 AM, Can ZHANG wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > I'd like to run a standalone Bluestore instance so as to test > > > > > > > > and tune > > > > > > > > its performance. Are there any tools about it, or any > > > > > > > > suggestions? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > Can Zhang > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > ceph-users mailing list > > > > > > > > ceph-users@lists.ceph.com > > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > _______________________________________________ > > > > > > ceph-users mailing list > > > > > > ceph-users@lists.ceph.com > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > > > > > -- > > > > > Cheers, > > > > > Brad > > > > > > > > > > > > -- > > > Cheers, > > > Brad > > > > -- > Cheers, > Brad -- Cheers, Brad _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
