subject:"\[PATCH\] textproc\/ispell segfaults immediately on being run"

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Bryan Linton

On 2017-06-03 14:02:55, Stuart Henderson  wrote:
> Ah good, it's the same with older libc so qsort isn't implicated.
> 
> On 2017/06/03 19:54, Bryan Linton wrote:
> > Hopefully this will allow you to duplicate this behavior.  If not,
> > please let me know what else I can do to help.
> 
> It does, thanks. I see the same if I update it to 3.4.00 (I've updated
> the port anyway as that was overdue).
> 
> I don't see where keywordbuf is intentionally getting set to anything
> other than a fresh malloc though, and don't know where to start looking
> to find something else that might be scribbling over it.
> 
> Simple replication of the problem: just run
> 
> DICTIONARY=/usr/local/lib/ispell/american.hash ispell
> 

Ah well, at least now I know what was causing it and a simple way
to avoid the crash, so I thank you for prodding me to dig a little
deeper :)

-- 
Bryan

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Stuart Henderson

Ah good, it's the same with older libc so qsort isn't implicated.

On 2017/06/03 19:54, Bryan Linton wrote:
> On 2017-06-03 10:59:17, Stuart Henderson  wrote:
> > On 2017/06/03 17:59, Bryan Linton wrote:
> > > On 2017-06-03 09:23:18, Stuart Henderson  wrote:
> > > > On 2017/06/03 15:50, Bryan Linton wrote:
> > > > > 
> > > > > Ping?  No users of ispell here?
> > > > > 
> > > > 
> > > > It works here, can you build with symbols (make clean && make repackage
> > > > DEBUG=-g) and get a backtrace?
> > > > 
> > > 
> > > Sure, here it is, along with some more information that may be relevant.
> > > 
> > > % ispell
> > > ispell(84800) in free(): bogus pointer (double free?) 0x687361
> > > zsh: abort (core dumped)  ispell
> > 
> > > #2  0x19945ba50566 in wrterror (d=0x7f7ea720, 
> > > msg=0x19945bb82168 "bogus pointer (double free?) %p")
> > > at /usr/src/lib/libc/stdlib/malloc.c:306
> > > #3  0x19945ba51c8d in ofree (argpool=0x19946bab8c60, p=0x687361, 
> > > clear=0,
> > > check=0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:1411
> > > #4  0x19945ba51f03 in free (ptr=0x687361)
> > > at /usr/src/lib/libc/stdlib/malloc.c:1444
> > > #5  0x1991e3c07711 in init_keyword_table (rawtags=Variable "rawtags" 
> > > is not available.
> > > ) at defmt.c:1316
> > > #6  0x1991e3c01a66 in main (argc=0, argv=0x7f7ebb30) at 
> > > ispell.c:889
> > 
> > 0x687361 (keywordbuf) seems unlikely to be a correct address and the
> > fact that it's a representation of ascii chars "ash" seems like it could
> > be more than a coincidence.
> > 
> > It would be really nice to be able to replicate this, let's try to
> > figure out what's different about your setup.
> > 
> > Do you have any .ispell* files?
> >
> 
> Yes, however mv'ing them away does not change the behavior.  But
> see below.
> 
> > Do you have any of the other dictionary packages (ispell-dutch,
> > ispell-french, etc) installed?
> >
> 
> % pkg_info -a | grep ispell
> ispell-3.2.06p9 interactive spelling checker
> %
> 
> > Does it happen with a clean environment ("env -i ispell") as well?
> > If not, what's in your usual environment?
> > 
> 
> "env -i ispell" works just fine for me, so I did some digging.
> 
> % env | grep ispell
> DICTIONARY=/usr/local/lib/ispell/american.hash
> % ispell 
> ispell(4541) in free(): bogus pointer (double free?) 0x687361
> zsh: abort (core dumped)  ispell
> % DICTIONARY=/usr/local/lib/ispell/british.hash 
> % ispell
> ispell(22859) in free(): bogus pointer (double free?) 0x6873
> zsh: abort (core dumped)  ispell
> % DICTIONARY=/usr/local/lib/ispell/default.hash 
> % ispell
> ispell(87738) in free(): bogus pointer (double free?) 0x6873
> zsh: abort (core dumped)  ispell
> % DICTIONARY=/usr/local/lib/ispell/americanmed+.hash
> % ispell
> ispell(57185) in free(): bogus pointer (double free?) 0x68736168002b64
> zsh: abort (core dumped)  ispell
> 
> 0x68736168002b64 would appear to be "hsah+d" when read
> forwards and "d+hash" when read backwards.

yep - if it's treated as a memory address on a little-endian machine
it would be backwards.

> It would appear that the hash files are somehow causing this
> behavior.  Also, to reiterate, when ispell is compiled with clang,
> this crash does not happen.  I'm not sure what clang might be
> doing differently, but I figure it might be pertinent information
> to have.

different optimization decisions, different memory layout in the
compiled code, various possibilities..

> Hopefully this will allow you to duplicate this behavior.  If not,
> please let me know what else I can do to help.

It does, thanks. I see the same if I update it to 3.4.00 (I've updated
the port anyway as that was overdue).

I don't see where keywordbuf is intentionally getting set to anything
other than a fresh malloc though, and don't know where to start looking
to find something else that might be scribbling over it.

Simple replication of the problem: just run

DICTIONARY=/usr/local/lib/ispell/american.hash ispell

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Todd C. Miller

On Sat, 03 Jun 2017 10:59:17 +0100, Stuart Henderson wrote:

> I don't know if it's related, but one thing which does stick out is that
> this is happening just after a qsort() and the implementation of this
> was changed recently in libc (17-30 May). CC'ing millert just in case.

Backing out qsort.c all the way to rev 1.13 didn't change the
behavior.  It still crashes using the hash file.

 - todd

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Bryan Linton

On 2017-06-03 10:59:17, Stuart Henderson  wrote:
> On 2017/06/03 17:59, Bryan Linton wrote:
> > On 2017-06-03 09:23:18, Stuart Henderson  wrote:
> > > On 2017/06/03 15:50, Bryan Linton wrote:
> > > > 
> > > > Ping?  No users of ispell here?
> > > > 
> > > 
> > > It works here, can you build with symbols (make clean && make repackage
> > > DEBUG=-g) and get a backtrace?
> > > 
> > 
> > Sure, here it is, along with some more information that may be relevant.
> > 
> > % ispell
> > ispell(84800) in free(): bogus pointer (double free?) 0x687361
> > zsh: abort (core dumped)  ispell
> 
> > #2  0x19945ba50566 in wrterror (d=0x7f7ea720, 
> > msg=0x19945bb82168 "bogus pointer (double free?) %p")
> > at /usr/src/lib/libc/stdlib/malloc.c:306
> > #3  0x19945ba51c8d in ofree (argpool=0x19946bab8c60, p=0x687361, 
> > clear=0,
> > check=0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:1411
> > #4  0x19945ba51f03 in free (ptr=0x687361)
> > at /usr/src/lib/libc/stdlib/malloc.c:1444
> > #5  0x1991e3c07711 in init_keyword_table (rawtags=Variable "rawtags" is 
> > not available.
> > ) at defmt.c:1316
> > #6  0x1991e3c01a66 in main (argc=0, argv=0x7f7ebb30) at ispell.c:889
> 
> 0x687361 (keywordbuf) seems unlikely to be a correct address and the
> fact that it's a representation of ascii chars "ash" seems like it could
> be more than a coincidence.
> 
> It would be really nice to be able to replicate this, let's try to
> figure out what's different about your setup.
> 
> Do you have any .ispell* files?
>

Yes, however mv'ing them away does not change the behavior.  But
see below.

> Do you have any of the other dictionary packages (ispell-dutch,
> ispell-french, etc) installed?
>

% pkg_info -a | grep ispell
ispell-3.2.06p9 interactive spelling checker
%

> Does it happen with a clean environment ("env -i ispell") as well?
> If not, what's in your usual environment?
> 

"env -i ispell" works just fine for me, so I did some digging.

% env | grep ispell
DICTIONARY=/usr/local/lib/ispell/american.hash
% ispell 
ispell(4541) in free(): bogus pointer (double free?) 0x687361
zsh: abort (core dumped)  ispell
% DICTIONARY=/usr/local/lib/ispell/british.hash 
% ispell
ispell(22859) in free(): bogus pointer (double free?) 0x6873
zsh: abort (core dumped)  ispell
% DICTIONARY=/usr/local/lib/ispell/default.hash 
% ispell
ispell(87738) in free(): bogus pointer (double free?) 0x6873
zsh: abort (core dumped)  ispell
% DICTIONARY=/usr/local/lib/ispell/americanmed+.hash
% ispell
ispell(57185) in free(): bogus pointer (double free?) 0x68736168002b64
zsh: abort (core dumped)  ispell

0x68736168002b64 would appear to be "hsah+d" when read
forwards and "d+hash" when read backwards.

It would appear that the hash files are somehow causing this
behavior.  Also, to reiterate, when ispell is compiled with clang,
this crash does not happen.  I'm not sure what clang might be
doing differently, but I figure it might be pertinent information
to have.

Hopefully this will allow you to duplicate this behavior.  If not,
please let me know what else I can do to help.

-- 
Bryan

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Stuart Henderson

On 2017/06/03 17:59, Bryan Linton wrote:
> On 2017-06-03 09:23:18, Stuart Henderson  wrote:
> > On 2017/06/03 15:50, Bryan Linton wrote:
> > > 
> > > Ping?  No users of ispell here?
> > > 
> > 
> > It works here, can you build with symbols (make clean && make repackage
> > DEBUG=-g) and get a backtrace?
> > 
> 
> Sure, here it is, along with some more information that may be relevant.
> 
> % ispell
> ispell(84800) in free(): bogus pointer (double free?) 0x687361
> zsh: abort (core dumped)  ispell

> #2  0x19945ba50566 in wrterror (d=0x7f7ea720, 
> msg=0x19945bb82168 "bogus pointer (double free?) %p")
> at /usr/src/lib/libc/stdlib/malloc.c:306
> #3  0x19945ba51c8d in ofree (argpool=0x19946bab8c60, p=0x687361, clear=0,
> check=0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:1411
> #4  0x19945ba51f03 in free (ptr=0x687361)
> at /usr/src/lib/libc/stdlib/malloc.c:1444
> #5  0x1991e3c07711 in init_keyword_table (rawtags=Variable "rawtags" is 
> not available.
> ) at defmt.c:1316
> #6  0x1991e3c01a66 in main (argc=0, argv=0x7f7ebb30) at ispell.c:889

0x687361 (keywordbuf) seems unlikely to be a correct address and the
fact that it's a representation of ascii chars "ash" seems like it could
be more than a coincidence.

I don't see the code setting keywordbuf to anything odd.

I don't know if it's related, but one thing which does stick out is that
this is happening just after a qsort() and the implementation of this
was changed recently in libc (17-30 May). CC'ing millert just in case.

 1308 qsort ((char *) keywords->kwlist, keywords->numkw,
 1309   sizeof keywords->kwlist[0],
 1310   (int (*) P ((const void *, const void *))) tagcmp);
 1311 
 1312 if (keywords->maxlen > maxkeywordlen)
 1313 {
 1314 maxkeywordlen = keywords->maxlen;
 1315 if (keywordbuf != NULL)
>1316 free (keywordbuf);
 1317 keywordbuf = (unsigned char *)
 1318   malloc ((maxkeywordlen + 1) * sizeof keywordbuf[0]);
 1319 if (keywordbuf == NULL)
 1320 {
 1321 fprintf (stderr, DEFMT_C_NO_SPACE);
 1322 exit(1);
 1323 }
 1324 }
 1325 return 0;
 1326 }

It would be really nice to be able to replicate this, let's try to
figure out what's different about your setup.

Do you have any .ispell* files?

Do you have any of the other dictionary packages (ispell-dutch,
ispell-french, etc) installed?

Does it happen with a clean environment ("env -i ispell") as well?
If not, what's in your usual environment?

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Bryan Linton

On 2017-06-03 09:23:18, Stuart Henderson  wrote:
> On 2017/06/03 15:50, Bryan Linton wrote:
> > 
> > Ping?  No users of ispell here?
> > 
> 
> It works here, can you build with symbols (make clean && make repackage
> DEBUG=-g) and get a backtrace?
> 

Sure, here it is, along with some more information that may be relevant.

% ispell
ispell(84800) in free(): bogus pointer (double free?) 0x687361
zsh: abort (core dumped)  ispell

% gdb `which ispell` ispell.core 
Core was generated by `ispell'.
Program terminated with signal 6, Aborted.
Loaded symbols for /usr/local/bin/ispell
Reading symbols from /usr/lib/libtermcap.so.14.0...done.
Loaded symbols for /usr/lib/libtermcap.so.14.0
Reading symbols from /usr/lib/libc.so.89.5...done.
Loaded symbols for /usr/lib/libc.so.89.5
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
#0  0x19945ba0a2da in thrkill () at {standard input}:5
5   {standard input}: No such file or directory.
in {standard input}
(gdb) bt
#0  0x19945ba0a2da in thrkill () at {standard input}:5
#1  0x19945ba24429 in *_libc_abort ()
at /usr/src/lib/libc/stdlib/abort.c:52
#2  0x19945ba50566 in wrterror (d=0x7f7ea720, 
msg=0x19945bb82168 "bogus pointer (double free?) %p")
at /usr/src/lib/libc/stdlib/malloc.c:306
#3  0x19945ba51c8d in ofree (argpool=0x19946bab8c60, p=0x687361, clear=0,
check=0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:1411
#4  0x19945ba51f03 in free (ptr=0x687361)
at /usr/src/lib/libc/stdlib/malloc.c:1444
#5  0x1991e3c07711 in init_keyword_table (rawtags=Variable "rawtags" is not 
available.
) at defmt.c:1316
#6  0x1991e3c01a66 in main (argc=0, argv=0x7f7ebb30) at ispell.c:889
Current language:  auto; currently asm
(gdb) quit

% ls -la /etc/malloc.conf
ls: /etc/malloc.conf: No such file or directory

% env | grep MALLOC
% dmesg
OpenBSD 6.1-current (GENERIC.MP-PPPOE_TERM_UNKNOWN_SESSIONS) #11: Tue May 23 
19:10:56 JST 2017

shoshon...@shoshoni-m.shoshoni.info:/usr/src/sys/arch/amd64/compile/GENERIC.MP-PPPOE_TERM_UNKNOWN_SESSIONS
real mem = 12539871232 (11958MB)
avail mem = 12154036224 (11590MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xbcc0d000 (67 entries)
bios0: vendor LENOVO version "GLET85WW (2.39 )" date 09/29/2016
bios0: LENOVO 20AWS27D00
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC DBGP ECDT HPET APIC MCFG SSDT SSDT SSDT SSDT SSDT 
SSDT SSDT PCCT SSDT TCPA UEFI MSDM ASF! BATB FPDT UEFI DMAR
acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP2(S4) EXP3(S4) XHCI(S3) 
EHC1(S3) EHC2(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-4300M CPU @ 2.60GHz, 2594.33 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2594330880 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5-4300M CPU @ 2.60GHz, 2594.00 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Core(TM) i5-4300M CPU @ 2.60GHz, 2594.00 MHz
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i5-4300M CPU @ 2.60GHz, 2594.00 MHz
cpu3:

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Stuart Henderson

On 2017/06/03 15:50, Bryan Linton wrote:
> On 2017-05-27 18:02:54, Bryan Linton  wrote:
> > Hello ports@
> > 
> > I noticed that somehow, ispell now segfaults immediately upon
> > being run with an error such as:
> > 
> > ispell(67135) in free(): bogus pointer (double free?) 0x687361
> > 
> > [...]
> > 
> > ispell Just Works(TM) when compiled with clang.
> > 
> > [...]
> >
> 
> Ping?  No users of ispell here?
> 
> -- 
> Bryan
> 

It works here, can you build with symbols (make clean && make repackage
DEBUG=-g) and get a backtrace?

Re: [PATCH] textproc/ispell segfaults immediately on being run

2017-06-03 Thread Bryan Linton

On 2017-05-27 18:02:54, Bryan Linton  wrote:
> Hello ports@
> 
> I noticed that somehow, ispell now segfaults immediately upon
> being run with an error such as:
> 
>   ispell(67135) in free(): bogus pointer (double free?) 0x687361
> 
> [...]
> 
> ispell Just Works(TM) when compiled with clang.
> 
> [...]
>

Ping?  No users of ispell here?

-- 
Bryan

[PATCH] textproc/ispell segfaults immediately on being run

2017-05-27 Thread Bryan Linton

Hello ports@

I noticed that somehow, ispell now segfaults immediately upon
being run with an error such as:

ispell(67135) in free(): bogus pointer (double free?) 0x687361

I compiled it with debug symbols, and attemped to debug it with gdb.
However, I'm still in the process of learning how to debug things,
and couldn't fix it even after poking at a few things here and there.

I attemped to compile ispell with clang, since IMO clang sometimes
gives better error messages than GCC and surprisingly enough,
ispell Just Works(TM) when compiled with clang.

Yes, I know this is probably just painting over a real problem in
ispell that should be fixed, but it gets ispell working for me.

I also realized after making the attached patch, that ispell
3.4.00 has been released, whereas the current version in ports is
3.2.06.  It may be easier to just update the port.  I'll attempt
to do so myself if I can find some extra time soon.

-- 
Bryan

Index: Makefile
===
RCS file: /cvs/ports/textproc/ispell/Makefile,v
retrieving revision 1.64
diff -u -r1.64 Makefile
--- Makefile3 Dec 2015 21:24:32 -   1.64
+++ Makefile27 May 2017 09:01:10 -
@@ -14,6 +14,10 @@
 DISTNAME=  ispell-${VERSION}
 CATEGORIES=textproc
 
+MODULES=   lang/clang
+MODCLANG_ARCHS=*
+MODCLANG_LANGS= c c++
+
 HOMEPAGE=  http://fmg-www.cs.ucla.edu/geoff/ispell.html
 
 MASTER_SITES=  http://fmg-www.cs.ucla.edu/geoff/tars/ \
@@ -57,7 +61,7 @@
 SUBST_VARS+=   VERSION
 
 PKGNAME-main=  ${DISTNAME}
-REVISION-main= 9
+REVISION-main= 10
 MULTI_PACKAGES=-main -dutch -french -german -swedish -russian 
-portuguese \
-slovak -spanish
 .for i in ${MULTI_PACKAGES}

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

Re: [PATCH] textproc/ispell segfaults immediately on being run

[PATCH] textproc/ispell segfaults immediately on being run

9 matches

Site Navigation

Mail list logo

Footer information