New submission from Nir Soffer:
GNU readline let the user select limit the history size by setting:
$ cat ~/.inputrc
set history-size 1000
So I cooked this test script:
$ cat history.py
from __future__ import print_function
import readline
readline.read_history_file(".history")
print("current_history_length", readline.get_current_history_length())
print("history_length", readline.get_history_length())
print("history_get_item(1)", readline.get_history_item(1))
print("history_get_item(1000)", readline.get_history_item(1000))
input()
readline.write_history_file(".history")
And this history file generator:
$ cat make-history
for i in range(2000):
print("%04d" % i)
Generating .history file with 2000 entries:
$ python3 make-history > .history
Finally running the test script:
$ python3 history.py
current_history_length 1000
history_length -1
history_get_item(1) None
history_get_item(1000) None
please crash
Segmentation fault (core dumped)
So we have few issues here:
- segfault
- history_get_item returns None for both 1 and 1000
although we have 1000 items in history
- history_length is always wrong (-1), instead of
the expected value (1000), set in .inputrc
Running with gdb we see:
$ gdb python3
GNU gdb (GDB) Fedora 7.12.1-46.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...Reading symbols from
/usr/lib/debug/usr/libexec/system-python.debug...done.
done.
(gdb) run history.py
Starting program: /usr/bin/python3 history.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
current_history_length 1000
history_length -1
history_get_item(1) None
history_get_item(1000) None
crash?
Program received signal SIGSEGV, Segmentation fault.
0x00007fffeff60fab in call_readline (sys_stdin=<optimized out>,
sys_stdout=<optimized out>, prompt=<optimized out>) at
/usr/src/debug/Python-3.5.2/Modules/readline.c:1281
1281 line = (const char *)history_get(length)->line;
(gdb) list
1276 if (using_libedit_emulation) {
1277 /* handle older 0-based or newer 1-based indexing */
1278 line = (const char *)history_get(length +
libedit_history_start - 1)->line;
1279 } else
1280 #endif /* __APPLE__ */
1281 line = (const char *)history_get(length)->line;
1282 else
1283 line = "";
1284 if (strcmp(p, line))
1285 add_history(p);
So we assume that history_get(length) returns non-null
when length > 0, but this assumption is not correct.
In 2 other usages in Modules/readline.c, we validate
that history_get() return value is not null before
using it.
If we change the .history contents to 1999 lines, we get:
$ python3 make-history | head -1999 > .history
$ python3 history.py
current_history_length 1000
history_length -1
history_get_item(1) None
history_get_item(1000) 0999
crash?
$ wc -l .history
1000 .history
$ head -1 .history
1000
$ tail -1 .history
crash?
So now it does not crash, but item 1 is still None.
Trying again with history file with 1000 entries:
$ python3 make-history | head -1000 > .history
$ python3 history.py
current_history_length 1000
history_length -1
history_get_item(1) 0000
history_get_item(1000) 0999
looks fine!
$ wc -l .history
1000 .history
$ head -1 history
head: cannot open 'history' for reading: No such file or directory
$ head -1 .history
0001
$ tail -1 .history
looks fine!
Finally trying with 1001 items:
$ python3 make-history | head -1001 > .history
$ python3 history.py
current_history_length 1000
history_length -1
history_get_item(1) None
history_get_item(1000) 0999
And item 1 is wrong.
I got same results with python 2.7, 3.5 and master
on fedora 25.
The root cause seems to be a readline bug when history
file is bigger than the history-size in .inputrc,
but I could not find yet readline library documentation,
so I don't know if the issues is incorrect usage of the
readline apis, or bug in readline.
----------
components: Extension Modules
messages: 289865
nosy: nirs
priority: normal
severity: normal
status: open
title: Segfault when readline history is more then 2 * history size
versions: Python 2.7, Python 3.5, Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29854>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
