New submission from Milt Epstein:
I'm trying to use csv.Sniffer().sniff(sample_data) to determine the delimiter
on a number of input files. Through some trial and error, many "Could not
determine delimiter" errors, and analyzing how this routine works/behaves, I
settled on sample_data being some number of lines of the input file,
particularly 30. This value seems to allow the routine to work more
frequently, although not always, particularly on short input files.
I realize the way this routine works is somewhat idiosyncratic, and it won't be
so easy to improve it generally, but there's one simple change that occurred to
me that would help in some cases. Currently the function _guess_delimiter() in
csv.py contains the following lines:
# build a list of possible delimiters
modeList = modes.items()
total = float(chunkLength * iteration)
So total is increased by chunkLength on each iteration. The problem occurs
when total becomes greater than the length of sample_data, that is, the
iteration would go beyond the end of sample_data. That reading is handled
fine, it's truncated at the end of sample_data, but total is needlessly set too
high. My suggested change is to add the following two lines after the above:
if total > len(data):
total = float(len(data))
----------
components: Library (Lib)
messages: 286570
nosy: mepstein
priority: normal
severity: normal
status: open
title: improve csv.Sniffer().sniff() behavior
type: behavior
versions: Python 3.5
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29405>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
