Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-22 Thread Simon McVittie
On Sun, 22 Jan 2023 at 17:07:05 +, Simon McVittie wrote:
> On Sat, 07 Jan 2023 at 22:07:31 +, Matthew Vernon wrote:
> > I'm struggling a bit here; I wanted to try and bisect pcre2 upstream commits
> > to see where this bug might have been introduced (or get to the bottom of
> > what link-grammar's test is doing wrong, I see they've been troublesome in
> > the past cf #975696).
> 
> # apt install git
> # git clone https://github.com/PCRE2Project/pcre2
> # cd /root/pcre2
> # git checkout pcre2-10.42
> # ./autogen.sh
> # ./configure CFLAGS="-fsanitize=address -fno-omit-frame-pointer 
> -fsanitize=undefined" --prefix=/usr
> # make
> # ( cd /root/link-grammar-5.11.0~dfsg && debuild 
> -eASAN_OPTIONS=detect_leaks=0 -eDEB_BUILD_OPTIONS="noopt 
> sanitize=+address,+undefined" -eLD_PRELOAD=libasan.so.8 
> -eLD_LIBRARY_PATH=/root/pcre2/.libs -us -uc -b )
>   ... tests fail ...

Bisection suggests that the first bad commit is d90fb238 "Refactor
match_data() to always use the heap instead of having an initial frames
vector on the stack; some consequential adjustmentsneeded.", which seems
like a plausible place to have introduced a heap overflow.

smcv



Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-22 Thread Simon McVittie
On Sat, 07 Jan 2023 at 22:07:31 +, Matthew Vernon wrote:
> I'm struggling a bit here; I wanted to try and bisect pcre2 upstream commits
> to see where this bug might have been introduced (or get to the bottom of
> what link-grammar's test is doing wrong, I see they've been troublesome in
> the past cf #975696).

I tried using AddressSanitizer to get more information, which might be
helpful. I used a bookworm podman container for this, and for simplicity
I'm using uid 0 in the container, but you could probably do the same in
a Docker container, a VM or a chroot and as an unprivileged user.

$ podman run -it --rm debian:bookworm-slim
# apt install --no-install-recommends devscripts build-essential
# echo "deb-src http://deb.debian.org/debian bookworm main" >> 
/etc/apt/sources.list
# cd /root
# apt update
# apt source pcre2 link-grammar
# echo "deb-src http://deb.debian.org/debian sid main" >> /etc/apt/sources.list
# apt source pcre2
# apt build-dep pcre2 link-grammar
# cd /root/pcre2-10.40
# debuild -eASAN_OPTIONS=detect_leaks=0 -eDEB_BUILD_OPTIONS="noopt 
sanitize=+address,+undefined" -us -uc -b
# cd /root/pcre2-10.42
# debuild -eASAN_OPTIONS=detect_leaks=0 -eDEB_BUILD_OPTIONS="noopt 
sanitize=+address,+undefined" -us -uc -b
# cd /root/link-grammar-5.11.0~dfsg
# debuild -eASAN_OPTIONS=detect_leaks=0 -eDEB_BUILD_OPTIONS="noopt 
sanitize=+address,+undefined" -eLD_PRELOAD=libasan.so.8 
-eLD_LIBRARY_PATH=/root/pcre2-10.40/debian/tmp/usr/lib/x86_64-linux-gnu -us -uc 
-b
  ... tests pass ...
# debuild -eASAN_OPTIONS=detect_leaks=0 -eDEB_BUILD_OPTIONS="noopt 
sanitize=+address,+undefined" -eLD_PRELOAD=libasan.so.8 
-eLD_LIBRARY_PATH=/root/pcre2-10.42/debian/tmp/usr/lib/x86_64-linux-gnu -us -uc 
-b
  ... tests fail ...

(Note that the order of options to debuild is significant, debuild option -e
must come before dpkg-buildpackage options like -us, -uc, -b.)

With those steps, I get AddressSanitizer reporting a heap buffer overflow
when the test tries to match a regular expression, and a similar error
in the multi-java test.

A roughly equivalent setup with the upstream libpcre also "works" (by which
I mean, fails). AddressSanitizer output below.

# apt install git
# git clone https://github.com/PCRE2Project/pcre2
# cd /root/pcre2
# git checkout pcre2-10.42
# ./autogen.sh
# ./configure CFLAGS="-fsanitize=address -fno-omit-frame-pointer 
-fsanitize=undefined" --prefix=/usr
# make
# ( cd /root/link-grammar-5.11.0~dfsg && debuild -eASAN_OPTIONS=detect_leaks=0 
-eDEB_BUILD_OPTIONS="noopt sanitize=+address,+undefined" 
-eLD_PRELOAD=libasan.so.8 -eLD_LIBRARY_PATH=/root/pcre2/.libs -us -uc -b )
  ... tests fail ...

The same setup with pcre2-10.40 passes tests. The link-grammar build
later fails when using the sanitizers, because it uses d-shlibs which
is basically an anti-pattern (#902605), but that's enough to be able to
bisect (in progress).

smcv

===
   link-grammar 5.11.0: tests/test-suite.log
===

# TOTAL: 5
# PASS:  3
# SKIP:  0
# XFAIL: 0
# FAIL:  2
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: multi-thread
==

=
==107299==ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x60e00708 at pc 0x7f8c46092bb4 bp 0x7f8c3cb87800 sp 0x7f8c3cb877f8
READ of size 1 at 0x60e00708 thread T10
#0 0x7f8c46092bb3 in match (/root/pcre2/.libs/libpcre2-8.so.0+0x492bb3)
#1 0x7f8c460b4eac in pcre2_match_8 
(/root/pcre2/.libs/libpcre2-8.so.0+0x4b4eac)
#2 0x7f8c474930e3 in reg_match dict-common/regex-morph.c:219
#3 0x7f8c47494753 in match_regex dict-common/regex-morph.c:405
#4 0x7f8c475b2997 in regex_guess tokenize/tokenize.c:413
#5 0x7f8c475ca258 in separate_word tokenize/tokenize.c:2709
#6 0x7f8c475cdf73 in separate_sentence tokenize/tokenize.c:3116
#7 0x7f8c4745fc07 in sentence_split link-grammar/api.c:494
#8 0x55b9169165b8 in parse_one_sent tests/multi-thread.cc:34
#9 0x55b9169177e9 in parse_sents tests/multi-thread.cc:119
#10 0x55b9169209c2 in void std::__invoke_impl(std::__invoke_other, void (*&&)(Dictionary_s*, Parse_Options_s*, int, 
int), Dictionary_s*&&, Parse_Options_s*&&, int&&, int&&) 
/usr/include/c++/12/bits/invoke.h:61
#11 0x55b91692056f in std::__invoke_result::type 
std::__invoke(void (*&&)(Dictionary_s*, 
Parse_Options_s*, int, int), Dictionary_s*&&, Parse_Options_s*&&, int&&, int&&) 
/usr/include/c++/12/bits/invoke.h:96
#12 0x55b91691fffb in void std::thread::_Invoker >::_M_invoke<0ul, 1ul, 2ul, 3ul, 
4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) 
/usr/include/c++/12/bits/std_thread.h:252
#13 0x55b91691fc59 in std::thread::_Invoker >::operator()() 
/usr/include/c++/12/bits/std_thread.h:259
#14 0x55b91691fc11 in 
std::thread::_State_impl > >::_M_run() 
/usr/include/c++/12/bits/std_thread.h:210
#15 0x7f8c46ed44a2  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd44a

Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-20 Thread Bernhard Übelacker

Dear Maintainer,
I tried to reproduce this segfault.
Unfortunately this did not work as expected.
The segfault in multi-thread did not show up.
But I received one in multi-dict (in e.g. 3 of 6 runs).

This was done in a up-to-date unstable amd64 VM,
running at a AMD cpu, 16 threads given to the VM.

The segfault did manifest in each case in libhunspell.
This seems to be caused by the static variable "utf_tbl"
possibly being shared between threads, and being freed and
initialized while maybe used in another thread.

Kind regards,
Bernhard


(gdb) bt
#0  0x7f56b5b3159b in unicodetolower(unsigned short, int) () from 
/lib/x86_64-linux-gnu/libhunspell-1.7.so.0
#1  0x7f56b5b317ae in get_captype_utf8(std::vector 
> const&, int) () from /lib/x86_64-linux-gnu/libhunspell-1.7.so.0
#2  0x7f56b5b385bc in ?? () from /lib/x86_64-linux-gnu/libhunspell-1.7.so.0
...
(gdb) bt
#0  0x7f56b5b3159b in unicodetolower (c=c@entry=116, 
langnum=langnum@entry=0) at ./src/hunspell/csutil.cxx:2482
#1  0x7f56b5b317ae in get_captype_utf8 (word=std::vector of length 6, 
capacity 64 = {...}, langnum=0) at ./src/hunspell/csutil.cxx:2538
#2  0x7f56b5b385bc in HashMgr::get_clen_and_captype (workbuf=std::vector of length 6, 
capacity 64 = {...}, captype=, word="tenant", 
this=0x7f568eb64d90) at ./src/hunspell/hashmgr.cxx:468
#3  HashMgr::get_clen_and_captype (workbuf=..., captype=, 
word=..., this=0x7f568eb64d90) at ./src/hunspell/hashmgr.cxx:464
#4  HashMgr::load_tables (this=0x7f568eb64d90, tpath=, 
key=) at ./src/hunspell/hashmgr.cxx:690
#5  0x7f56b5b38dfc in HashMgr::HashMgr (this=this@entry=0x7f568eb64d90, 
tpath=tpath@entry=0x7f56adffa800 "/usr/share/hunspell/en_US.dic", 
apath=apath@entry=0x7f56adffa400 "/usr/share/hunspell/en_US.aff", key=key@entry=0x0) at 
./src/hunspell/hashmgr.cxx:101
#6  0x7f56b5b43616 in HunspellImpl::HunspellImpl (this=0x7f568fb8c850, affpath=0x7f56adffa400 
"/usr/share/hunspell/en_US.aff", dpath=0x7f56adffa800 
"/usr/share/hunspell/en_US.dic", key=0x0) at ./src/hunspell/hunspell.cxx:179
#7  0x7f56b5b43a27 in Hunspell_create (affpath=affpath@entry=0x7f56adffa400 
"/usr/share/hunspell/en_US.aff", dpath=dpath@entry=0x7f56adffa800 
"/usr/share/hunspell/en_US.dic") at ./src/hunspell/hunspell.cxx:2164
#8  0x7f56b60decc1 in spellcheck_create (lang=0x7f568d0e22c0 "en") at 
tokenize/spellcheck-hun.c:101
#9  0x7f56b60af6e5 in dictionary_six_str (lang=lang@entry=0x560b8f8ac37a "en", input=input@entry=0x7f568f9bc460 "\n %", '*' , 
"%\n %", ' ' , "%\n %   Copyright (C) 1991-1998  Daniel "..., dict_name=dict_name@entry=0x7f568fa3e630 "en/4.0.dict", 
pp_name=pp_name@entry=0x7f568fa3e6f0 "en/4.0.knowledge", cons_name=cons_name@entry=0x7f568fa3e8e0 "en/4.0.constituent-knowledge", 
affix_name=affix_name@entry=0x7f568fa3e930 "en/4.0.affix", regex_name=0x7f568fa3e070 "en/4.0.regex") at dict-file/dictionary.c:159
#10 0x7f56b60afc28 in dictionary_six (regex_name=0x7f568fa3e070 "en/4.0.regex", affix_name=0x7f568fa3e930 
"en/4.0.affix", cons_name=0x7f568fa3e8e0 "en/4.0.constituent-knowledge", pp_name=0x7f568fa3e6f0 
"en/4.0.knowledge", dict_name=0x7f568fa3e630 "en/4.0.dict", lang=0x560b8f8ac37a "en") at 
dict-file/dictionary.c:281
#11 dictionary_create_from_file (lang=0x560b8f8ac37a "en") at 
dict-file/dictionary.c:307
#12 0x560b8f8ab598 in parse_one_sent (sent_str=0x560b8f8ac248 "The line extends 
10 miles offshore.") at ./tests/multi-dict.cc:28
#13 parse_sents (thread_id=, niter=30) at 
./tests/multi-dict.cc:82
#14 0x7f56b5ed44a3 in std::execute_native_thread_routine 
(__p=0x560b90a39a70) at ../../../../../src/libstdc++-v3/src/c++11/thread.cc:82
#15 0x7f56b5ca7fd4 in start_thread (arg=) at 
./nptl/pthread_create.c:442
#16 0x7f56b5d2866c in clone3 () at 
../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

2482  return (utf_tbl) ? utf_tbl[c].clower : c;

=> 0x7f56b5b3159b <_Z14unicodetolowerti+27>:movzwl 0x4(%rdx,%rax,2),%eax
(gdb) print/x $rdx
$49 = 0x7f56ac614010
(gdb) print/x $rax
$50 = 0x15c
(gdb) x/1xg ($rdx + $rax * 2) + 0x4
0x7f56ac6142cc: Cannot access memory at address 0x7f56ac6142cc
(gdb) print utf_tbl
$51 = (unicode_info2 *) 0x0
# Unstable amd64 qemu VM 2023-01-20

apt update
apt dist-upgrade

apt install systemd-coredump mc gdb valgrind rr autopkgtest dpkg-dev 
libhunspell-1.7-0-dbgsym libstdc++6-dbgsym
apt build-dep link-grammar

autopkgtest --shell-fail link-grammar -- null



PASS: mem-leak
PASS: multi-java
PASS: dict-reopen
../test-driver: Zeile 112: 12435 Speicherzugriffsfehler  (Speicherabzug 
geschrieben) "$@" >> "$log_file" 2>&1
FAIL: multi-dict
PASS: multi-thread
===
   link-grammar 5.12.0: tests/test-suite.log
===

# TOTAL: 5
# PASS:  4
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: multi-dict







[  348.182545] multi-dict[12511]: segfault at 7f56ac6142cc ip 7f56b5b3159b 
sp 7f56adffa188 error 4 in libhun

Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-07 Thread Matthew Vernon

Hi,

I'm struggling a bit here; I wanted to try and bisect pcre2 upstream 
commits to see where this bug might have been introduced (or get to the 
bottom of what link-grammar's test is doing wrong, I see they've been 
troublesome in the past cf #975696).


I took unstable's link-grammar source, and built it thus:

./autogen.sh
#the python test fails for some reason
./configure --disable-python-bindings --disable-java-bindings
make
make check

I then cloned pcre2 upstream, checked out pcre2-10.42
./autogen.sh
./configure --prefix=/home/matthew/junk/pcre2bug/install
make
make install

Then I go back to my link-grammar tree and do
cd tests
LD_LIBRARY_PATH=/home/matthew/junk/pcre2bug/install/lib make check

which works

similarly
LD_LIBRARY_PATH=/home/matthew/junk/pcre2bug/install/lib ./multi-thread

works

With a slightly horrible hack:
lsof -p $(ps waux | grep multi-thread | grep pcre | cut -f 4 -d ' ')

I can confirm the running multi-thread test is definitely running my 
locally-built (and thus 10.42) pcre2 library.


...so I'm not currently able to reproduce this in any way that might 
help me track down what if anything in pcre2 changed. And I'm not sure 
upstream would be very happy with "this test suite from another package 
now fails in Debian CI" as a bug report...


Regards,

Matthew



Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-05 Thread Adrian Bunk
On Thu, Jan 05, 2023 at 03:07:58AM +0200, Adrian Bunk wrote:
>...
> #0  0x7f46a39011b1 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
> #1  0x7f46a38ebb85 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
> #2  0x7f46a38fa10f in pcre2_match_8 ()
>from /lib/x86_64-linux-gnu/libpcre2-8.so.0
>...

I should have double-checked that all relevant -dbgsym were installed,
here is a better backtrace:

#0  0x7f392ac891b1 in _pcre2_xclass_8 (c=1350565888, data=, 
utf=utf@entry=1) at src/pcre2_xclass.c:136
#1  0x7f392ac73b85 in match (
start_eptr=start_eptr@entry=0x7f38d8431aa0 "his", 
start_ecode=, top_bracket=, 
frame_size=frame_size@entry=128, 
match_data=match_data@entry=0x55fe226640d0, mb=mb@entry=0x7f39278df4e0)
at src/pcre2_match.c:2171
#2  0x7f392ac8210f in pcre2_match_8 (code=, 
subject=subject@entry=0x7f38d8431aa0 "his", length=3, 
length@entry=18446744073709551615, start_offset=start_offset@entry=0, 
options=options@entry=1073741824, match_data=, 
mcontext=) at src/pcre2_match.c:7289
#3  0x7f392b134a86 in reg_match (rn=0x55fe22663e00, s=0x7f38d8431aa0 "his")
at dict-common/regex-morph.c:219
#4  match_regex (rn=0x55fe22663e00, s=s@entry=0x7f38d8431aa0 "his")
at dict-common/regex-morph.c:405
#5  0x7f392b16af63 in regex_guess (gword=0x7f38d8450160, 
word=, dict=0x55fe2265fa00) at tokenize/tokenize.c:413
#6  separate_word (sent=sent@entry=0x7f38d8430070, 
unsplit_word=unsplit_word@entry=0x7f38d8450160, 
opts=opts@entry=0x55fe252c60a0) at tokenize/tokenize.c:2709
#7  0x7f392b16c55a in separate_sentence (sent=sent@entry=0x7f38d8430070, 
opts=opts@entry=0x55fe252c60a0) at tokenize/tokenize.c:3116
#8  0x7f392b1261ec in sentence_split (sent=0x7f38d8430070, 
opts=0x55fe252c60a0) at ./link-grammar/api.c:494
#9  0x55fe20ead670 in parse_one_sent (dict=, 
opts=0x55fe252c60a0, sent_str=)
at ./tests/multi-thread.cc:34
#10 0x55fe20ead9aa in parse_sents (dict=0x55fe2265fa00, 
opts=0x55fe252c60a0, thread_id=, niter=500)
at ./tests/multi-thread.cc:119
#11 0x7f392af8c4a3 in std::execute_native_thread_routine (
__p=0x55fe252b7f30)
at ../../../../../src/libstdc++-v3/src/c++11/thread.cc:82
#12 0x7f392ad3ffd4 in start_thread (arg=)
at ./nptl/pthread_create.c:442
#13 0x7f392adbf8d0 in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100


cu
Adrian



Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-05 Thread Adrian Bunk
On Thu, Jan 05, 2023 at 09:14:48AM +, Matthew Vernon wrote:
> Hi,
> 
> On 05/01/2023 01:07, Adrian Bunk wrote:
> 
> > https://tracker.debian.org/pkg/pcre2
> > Issues preventing migration:
> > ∙ ∙ autopkgtest for link-grammar/5.11.0~dfsg-1: amd64: Pass, arm64: 
> > Regression ♻ (reference ♻), armel: Regression ♻ (reference ♻), armhf: 
> > Regression ♻ (reference ♻), i386: Pass, ppc64el: Regression ♻ (reference 
> > ♻), s390x: Pass
> 
> Am I reading that right that it works on amd64, but fails on some other
> architectures?

No, it only says that it once succeeded:

https://buildd.debian.org/status/logs.php?pkg=link-grammar&arch=amd64
https://ci.debian.net/packages/l/link-grammar/testing/amd64/

For the autopkgtest that's success on 4th attempt, after 3 failures.

> > #0  0x7f46a39011b1 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
> 
> ...but it then fails in your local test on amd64?

The name of the failuing test is "multi-thread",
and wherever the problem is my guess would be a race condition.

> [but link-grammar compiles OK with the previous pcre2 version?]

That's what my local build tests say.

And also what the link-grammar autopkgtest says.  

> Regards,
> 
> Matthew

cu
Adrian



Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-05 Thread Matthew Vernon

Hi,

On 05/01/2023 01:07, Adrian Bunk wrote:


https://tracker.debian.org/pkg/pcre2
Issues preventing migration:
∙ ∙ autopkgtest for link-grammar/5.11.0~dfsg-1: amd64: Pass, arm64: Regression 
♻ (reference ♻), armel: Regression ♻ (reference ♻), armhf: Regression ♻ 
(reference ♻), i386: Pass, ppc64el: Regression ♻ (reference ♻), s390x: Pass


Am I reading that right that it works on amd64, but fails on some other 
architectures?



#0  0x7f46a39011b1 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0


...but it then fails in your local test on amd64?

[but link-grammar compiles OK with the previous pcre2 version?]

Regards,

Matthew



Bug#1027952: pcre2 causes crashes in link-grammar

2023-01-04 Thread Adrian Bunk
Package: libpcre2-8-0
Version: 10.42-1
Severity: serious
Tags: ftbfs
Control: affects -1 src:link-grammar

https://tracker.debian.org/pkg/pcre2
Issues preventing migration:
∙ ∙ autopkgtest for link-grammar/5.11.0~dfsg-1: amd64: Pass, arm64: Regression 
♻ (reference ♻), armel: Regression ♻ (reference ♻), armhf: Regression ♻ 
(reference ♻), i386: Pass, ppc64el: Regression ♻ (reference ♻), s390x: Pass

https://buildd.debian.org/status/logs.php?pkg=link-grammar&ver=5.11.0~dfsg-1%2Bb1

...
make  check-TESTS
make[4]: Entering directory '/<>/tests'
make[5]: Entering directory '/<>/tests'
PASS: mem-leak
PASS: multi-java
../test-driver: line 112: 3264343 Segmentation fault  "$@" >> "$log_file" 
2>&1
FAIL: multi-thread
PASS: multi-dict
PASS: dict-reopen
===
   link-grammar 5.11.0: tests/test-suite.log
===

# TOTAL: 5
# PASS:  4
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: multi-thread
==

FAIL multi-thread (exit status: 139)


Testsuite summary for link-grammar 5.11.0

# TOTAL: 5
# PASS:  4
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

See tests/test-suite.log
Please report to https://github.com/opencog/link-grammar

make[5]: *** [Makefile:806: test-suite.log] Error 1



A backtrace from a coredump of a local build of link-grammar
also suggests that pcre might be the problem:

Core was generated by `/tmp/link-grammar-5.11.0~dfsg/tests/.libs/multi-thread'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x7f46a39011b1 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
[Current thread is 1 (Thread 0x7f469e5546c0 (LWP 17344))]
(gdb) bt
#0  0x7f46a39011b1 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
#1  0x7f46a38ebb85 in ?? () from /lib/x86_64-linux-gnu/libpcre2-8.so.0
#2  0x7f46a38fa10f in pcre2_match_8 ()
   from /lib/x86_64-linux-gnu/libpcre2-8.so.0
#3  0x7f46a3da9a86 in reg_match (rn=0x562bf27ace00, 
s=0x7f467b5b2100 "Bill") at dict-common/regex-morph.c:219
#4  match_regex (rn=0x562bf27ace00, s=s@entry=0x7f467b5b2100 "Bill")
at dict-common/regex-morph.c:405
#5  0x7f46a3ddff63 in regex_guess (gword=0x7f467b59b3b0, 
word=, dict=0x562bf27a8a00) at tokenize/tokenize.c:413
#6  separate_word (sent=sent@entry=0x7f467b5b59e0, 
unsplit_word=unsplit_word@entry=0x7f467b59b3b0, 
opts=opts@entry=0x562bf547ae60) at tokenize/tokenize.c:2709
#7  0x7f46a3de155a in separate_sentence (sent=sent@entry=0x7f467b5b59e0, 
opts=opts@entry=0x562bf547ae60) at tokenize/tokenize.c:3116
#8  0x7f46a3d9b1ec in sentence_split (sent=0x7f467b5b59e0, 
opts=0x562bf547ae60) at ./link-grammar/api.c:494
#9  0x562bf0419670 in parse_one_sent (dict=, 
opts=0x562bf547ae60, sent_str=)
at ./tests/multi-thread.cc:34
#10 0x562bf04199aa in parse_sents (dict=0x562bf27a8a00, 
opts=0x562bf547ae60, thread_id=, niter=500)
at ./tests/multi-thread.cc:119
#11 0x7f46a3c044a3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#12 0x7f46a39b7fd4 in start_thread (arg=)
at ./nptl/pthread_create.c:442
#13 0x7f46a3a378d0 in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
(gdb)