Re: std::collate delete [] non heap memory
Mark Wright wrote: Hello Martin, The top 4 bytes of the stack allocated pointer pbuf is overwritten with 4 zero bytes when I do next over this line: const _RWSTD_SIZE_T dst_size = strxfrm (just_in_case_buf, psrc, 0); I wondered if it could be a bug in strxfrm(). I recall a problem with the function (STDCXX-68) but that was on Windows. [...] I can reproduce it easilly, and I have stdcxx compiled with debug symbols, so its very easy for me to try stuff in the debuggger, just let me know if you want me to try something. Or if you have some diffs for an idea to try, I can rebuild it and let you know the results. It looks like this might be a bug in the Solaris 10u5 strxfrm(). Looks that way. Unfortunately I don't have Solaris support contract, so I can't access SunSolve, or log a support issue with Sun. FWIW, I've been using the Sun Developer Network bug database to report bugs to Sun: http://bugreport.sun.com/bugreport/index.jsp Doing a google search I did find this hit on an old report of a Solaris 8 strxfrm() memory overwrite bug: http://archives.postgresql.org/pgsql-ports/2002-05/msg0.php Anyway I was wondering if it might help to make the just_in_case_buf buffer large to try to work around Solaris 10's strxfrm() insanity? I can try it if you like. That might work but I agree that the patch you proposed in your other post is safer. There's no need to use the MSVC workaround on Solaris. Let me open an issue for this and apply your patch. It will be included in 4.2.2, whenever it comes out. Thanks for debugging it! Martin PS You can keep track of the issue here: http://issues.apache.org/jira/browse/STDCXX-1010
Re: std::collate delete [] non heap memory
> On Fri, 5 Sep 2008 21:27:09 +1000 > Mark Wright <[EMAIL PROTECTED]> wrote: > > Anyway I was wondering if it might help to make the > just_in_case_buf buffer large to try to work around Solaris 10's > strxfrm() insanity? The trouble with that idea though is the concern that strxfrm() on Solaris 10u5 may still overwrite past the end of the buffer no matter how large I made just_in_case_buf. So I instead propose this fix: goanna% diff -wc stdcxx-4.2.1/src/orig/collate.cpp stdcxx-4.2.1/src/collate.cpp *** stdcxx-4.2.1/src/orig/collate.cpp 2008-04-25 10:25:19.0 +1000 --- stdcxx-4.2.1/src/collate.cpp2008-09-05 22:00:05.074726012 +1000 *** *** 522,531 --- 522,537 src+= (last - src) + 1; } + #if defined(__sun) && defined(__SVR4) + // Solaris 10u5 overwrites memory past the end of + // just_in_case_buf[8], to avoid this, pass a 0 pointer + char *just_in_case_buf = (char *)0; + #else // provide a destination buffer to strxfrm() in case // it's buggy (such as MSVC's) and tries to write to // the buffer even if it's 0 char just_in_case_buf [8]; + #endif const _RWSTD_SIZE_T dst_size = strxfrm (just_in_case_buf, psrc, 0); // check for strxfrm() errors goanna% I tested this with my test program on Solaris 10u5 with Sun Studio 12 C++, it works fine. Thanks very much, Mark --
Re: std::collate delete [] non heap memory
Hello Martin,
The top 4 bytes of the stack allocated pointer pbuf is
overwritten with 4 zero bytes when I do next over this line:
const _RWSTD_SIZE_T dst_size = strxfrm (just_in_case_buf, psrc, 0);
(dbx) where
=>[1] __rw::__rw_strnxfrm(src = 0x416fb5 "", nchars = 0), line 529 in
"collate.cpp"
[2] std::collate_byname::do_transform(this = 0x4170e0, low = 0x416f98
"Et la marine va venir à Malte", high = 0x416fb5 ""), line 925 in "collate.cpp"
[3] std::collate_byname::do_compare(this = 0x4170e0, low1 = 0x416f98
"Et la marine va venir à Malte", high1 = 0x416fb5 "", low2 = 0x417048 "Et la
marine va venir à Malte", high2 = 0x417065 ""), line 895 in "collate.cpp"
[4] std::collate::compare(this = 0x4170e0, __low1 = 0x416f98 "Et la
marine va venir à Malte", __high1 = 0x416fb5 "", __low2 = 0x417048 "Et la
marine va venir à Malte", __high2 = 0x417065 ""), line 119 in "_collate.h"
[5] main(), line 10 in "collate.cpp"
(dbx) print &just_in_case_buf[0]
&just_in_case_buf[0] = 0xfd7fffdfb360 "n(·ï^?ýÿÿ"
(dbx) examine 0xfd7fffdfb360 /145c
0xfd7fffdfb360: 'n' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '\0' '\0' '\0'
'\0' '\0' '\0' '\0' '\0'
0xfd7fffdfb370: '¼' ''' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '»' ''' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb380: 'õ' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '§' '³' 'ß' 'ÿ'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb390: '\0' '\001' '\0' '\0' '\0' '\0' '\0' '\0' '§' '³' 'ß'
'ÿ' '\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3a0: '¾' ''' '·' 'ï' '\0177' 'ý' 'ÿ' 'E' 't' ' ' 'l' 'a' '
' 'm' 'a' 'r'
0xfd7fffdfb3b0: 'i' 'n' 'e' ' ' 'v' 'a' ' ' 'v' 'e' 'n' 'i' 'r' ' '
'à' ' ' 'M'
0xfd7fffdfb3c0: 'a' 'l' 't' 'e' '\0' 'ý' 'ÿ' 'ÿ' 'å' ''' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3d0: '\0' '\0' '\0' '\0' '\0' '\0' '\0' '\0' 'Î' '(' '·'
'ï' '\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3e0: 'Ñ' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' 'Ô' '(' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3f0: '»'
(dbx) examine 0xfd7fffdfb360 /145n
0xfd7fffdfb360: 110 40 183 239 127 253 255 255 0 0 0 0 0 0 0 0
0xfd7fffdfb370: 188 39 183 239 127 253 255 255 187 39 183 239 127 253
255 255
0xfd7fffdfb380: 245 40 183 239 127 253 255 255 167 179 223 255 127 253
255 255
0xfd7fffdfb390: 0 1 0 0 0 0 0 0 167 179 223 255 127 253 255 255
0xfd7fffdfb3a0: 190 39 183 239 127 253 255 69 116 32 108 97 32 109 97
114
0xfd7fffdfb3b0: 105 110 101 32 118 97 32 118 101 110 105 114 32 224 32
77
0xfd7fffdfb3c0: 97 108 116 101 0 253 255 255 229 39 183 239 127 253
255 255
0xfd7fffdfb3d0: 0 0 0 0 0 0 0 0 206 40 183 239 127 253 255 255
0xfd7fffdfb3e0: 209 40 183 239 127 253 255 255 212 40 183 239 127 253
255 255
0xfd7fffdfb3f0: 187
(dbx) print pbuf
pbuf = 0xfd7fffdfb3a7 "Et la marine va venir à Malte"
(dbx) print &pbuf
&pbuf = 0xfd7fffdfb398
(dbx)
Its the top 4 bytes of the pointer pbuf here that are overwritten:
0xfd7fffdfb390: 0 1 0 0 0 0 0 0 167 179 223 255 127 253 255 255
I do next and strxfrm() zeros the top 4 bytes of the pbuf pointer:
(dbx) next
(dbx) print pbuf
pbuf = 0xffdfb3a7 ""
(dbx) examine 0xfd7fffdfb360 /145c
0xfd7fffdfb360: 'n' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '\0' '\0' '\0'
'\0' '\0' '\0' '\0' '\0'
0xfd7fffdfb370: '¼' ''' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '»' ''' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb380: 'õ' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' '§' '³' 'ß' 'ÿ'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb390: '\0' '\001' '\0' '\0' '\0' '\0' '\0' '\0' '§' '³' 'ß'
'ÿ' '\0' '\0' '\0' '\0'
0xfd7fffdfb3a0: '\0' '\0' 'ÿ' 'ÿ' '\0177' 'ý' '·' 'ï' 'å' ''' 'ÿ' 'ÿ'
'\0' '\0' 't' 'e'
0xfd7fffdfb3b0: 'a' 'l' ' ' 'M' ' ' 'à' 'i' 'r' 'e' 'n' ' ' 'v' 'v'
'a' 'e' ' '
0xfd7fffdfb3c0: 'i' 'n' 'a' 'r' ' ' 'm' 'l' 'a' 't' ' ' 'ÿ' 'E'
'\0177' 'ý' '·' 'ï'
0xfd7fffdfb3d0: '¾' ''' 'ÿ' 'ÿ' '\0177' 'ý' '\0' '\0' 'Î' '(' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3e0: 'Ñ' '(' '·' 'ï' '\0177' 'ý' 'ÿ' 'ÿ' 'Ô' '(' '·' 'ï'
'\0177' 'ý' 'ÿ' 'ÿ'
0xfd7fffdfb3f0: '»'
(dbx) examine 0xfd7fffdfb360 /145n
0xfd7fffdfb360: 110 40 183 239 127 253 255 255 0 0 0 0 0 0 0 0
0xfd7fffdfb370: 188 39 183 239 127 253 255 255 187 39 183 239 127 253
255 255
0xfd7fffdfb380: 245 40 183 239 127 253 255 255 167 179 223 255 127 253
255 255
0xfd7fffdfb390: 0 1 0 0 0 0 0 0 167 179 223 255 0 0 0 0
0xfd7fffdfb3a0: 0 0 255 255 127 253 183 239 229 39 255 255 0 0 116 101
0xfd7fffdfb3b0: 97 108 32 77 32 224 105 114 101 110 32 118 118 97 101
32
0xfd7fffdfb3c0: 105 110 97 114 32 109 108 97 116 32 255 69 127 253 183
239
0xfd7fffdfb3d0: 190 39 255 255 127 253 0 0 206 40 183 239 127 253 255
255
0xfd7fffdfb3e0: 209 40 183 239 127 253 255 255 212 40 183 239 127 253
255 255
0xfd7fffdfb3f0: 187
(dbx) examine psrc /145c
0xfd7fffdfb3a7: 'ï' 'å' ''' 'ÿ' 'ÿ' '\0'
Re: std::collate delete [] non heap memory
Mark Wright wrote: Hi, I was just wondering if I'm doing something wrong in this little program compiled with stdcxx 4.2.1, Sun Studio C++ 12, Solaris 10u5 AMD64, compiled as 64 bit: [...] Running it encounters a sigsegv, when it calls delete [] pbuf on non heap memory: I can't reproduce this problem in my environment. The program runs fine, both with libumem and in dbx with check -memuse on. Looking at the source code for __rw::__rw_strnxfrm(), pbuf is assigned one of two values: the address of the local array buf, and the result of the new expression. Its value is never assigned to another pointer that is then deleted and each of its delete expressions is guarded by a test for (pbuf != buf), so I don't see how it can ever be invalid. Can you step through the code to help debug it? Martin goanna% export LD_FLAGS_64="preload=libumem.so.1" goanna% export UMEM_DEBUG=default goanna% ./collate zsh: segmentation fault (core dumped) ./collate goanna% unset LD_FLAGS_64 goanna% unset UMEM_DEBUG goanna% dbx collate core Reading collate core file header read successfully Reading ld.so.1 Reading libumem.so.1 Reading libstd15D.so.4.2.1 Reading libCrun.so.1 Reading libm.so.2 Reading libthread.so.1 Reading libc.so.1 Reading fr_FR.ISO8859-1.so.3 [EMAIL PROTECTED] ([EMAIL PROTECTED]) program terminated by signal SEGV (no mapping at the fault address) 0xfd7fefd6374f: process_free+0x002f:movl (%rsi),%r8d Current function is __rw::__rw_strnxfrm 577 delete[] pbuf; (dbx) where current thread: [EMAIL PROTECTED] [1] process_free(0xffdfb5b7, 0xffdfb5af, 0x0, 0xfd7fffdfb42f, 0xfd7fffdfb5b7, 0xffdfb5b7), at 0xfd7fefd6374f [2] free(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfd7fefd638f5 [3] operator delete(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfd7fefbc9621 [4] operator delete[](0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfd7fefbc9549 =>[5] __rw::__rw_strnxfrm(src = 0x450f45 "", nchars = 0), line 577 in "collate.cpp" [6] std::collate_byname::do_transform(this = 0x451f20, low = 0x450f28 "Et la marine va venir à Malte", high = 0x450f45 ""), line 925 in "collate.cpp" [7] std::collate_byname::do_compare(this = 0x451f20, low1 = 0x450f28 "Et la marine va venir à Malte", high1 = 0x450f45 "", low2 = 0x450e28 "Et la marine va venir à Malte", high2 = 0x450e45 ""), line 895 in "collate.cpp" [8] std::collate::compare(this = 0x451f20, __low1 = 0x450f28 "Et la marine va venir à Malte", __high1 = 0x450f45 "", __low2 = 0x450e28 "Et la marine va venir à Malte", __high2 = 0x450e45 ""), line 119 in "_collate.h" [9] main(), line 10 in "collate.cpp" (dbx) print pbuf pbuf = 0xffdfb5b7 "" (dbx) loadobject -list m /h/goanna/2/eng/dev/cxx/collate/collate (primary) m /lib/amd64/libumem.so.1 m /h/goanna/1/a_5.10_m64/c/lib/libstd15D.so.4.2.1 m /usr/lib/amd64/libCrun.so.1 m /lib/amd64/libm.so.2 m /lib/amd64/libthread.so.1 m /lib/amd64/libc.so.1 m /usr/lib/locale/fr_FR.ISO8859-1/amd64/fr_FR.ISO8859-1.so.3 (dbx) Thanks very much, Mark
