[issue11818] tempfile.TemporaryFile example in docs doesnt work
New submission from eduardo :
>From the example:
http://docs.python.org/py3k/library/tempfile.html#examples
The error message is weird... but I guess the problem is the default mode 'w+b'.
Python 3.3a0 (default:78a66c98288d, Apr 9 2011, 16:13:31)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> fp = tempfile.TemporaryFile()
>>> fp.write('hello')
Traceback (most recent call last):
File "", line 1, in
TypeError: 'str' does not support the buffer interface
>>> fp2 = tempfile.TemporaryFile('w+')
>>> fp2.write('hello')
5
>>>
--
assignee: docs@python
components: Documentation, Library (Lib)
messages: 133447
nosy: docs@python, schettino72
priority: normal
severity: normal
status: open
title: tempfile.TemporaryFile example in docs doesnt work
versions: Python 3.2
___
Python tracker
<http://bugs.python.org/issue11818>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1148] TypeError on join - httplib mixing str and bytes
New submission from Eduardo Padoan:
To reproduce:
>>> import httplib
>>> conn = httplib.HTTPConnection("www.python.org")
>>> conn.request("GET", "/index.html")
>>> r1 = conn.getresponse()
>>> r1.read()
Traceback (most recent call last):
File "", line 1, in
File "/home/eopadoan/pub/py3k/Lib/httplib.py", line 540, in read
s = self._safe_read(self.length)
File "/home/eopadoan/pub/py3k/Lib/httplib.py", line 627, in _safe_read
return "".join(s)
TypeError: sequence item 0: expected string or Unicode, bytes found
Also, shouldn't the message be like "...expected str, bytes found"?
--
components: Library (Lib)
messages: 55827
nosy: eopadoan
severity: normal
status: open
title: TypeError on join - httplib mixing str and bytes
type: crash
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1148>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1151] "TypeError: expected string, bytes found" instead of KeyboardInterrupt
New submission from Eduardo Padoan:
On revision 54803, interactive mode, on linux: if type ctrl+c you type
ctrl+c, it should raise KeyboardInterrupt, but "TypeError: expected
string, bytes found" printed. Also, I could *not* catch it doing:
>>> try:
... while True: pass
... except KeyboardInterrupt:
... print('Ok')
... except TypeError:
... print('Ops')
Ok
To reproduce:
>>> # press ctrl+c...
TypeError: expected string, bytes found
>>>
It seems that it is simply printing the wrong error...
--
components: Interpreter Core
messages: 55833
nosy: eopadoan
severity: normal
status: open
title: "TypeError: expected string, bytes found" instead of KeyboardInterrupt
type: behavior
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1151>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1161] Mangled chars in offending line of SyntaxError traceback
New submission from Eduardo Padoan: Python 3.0a1 (py3k:58103, Sep 11 2007, 13:52:21) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> asd asd File "", line 1 P�� ^ SyntaxError: invalid syntax -- components: Interpreter Core messages: 55888 nosy: eopadoan severity: normal status: open title: Mangled chars in offending line of SyntaxError traceback type: behavior versions: Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1161> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1161] Garbled chars in offending line of SyntaxError traceback
Changes by Eduardo Padoan: -- title: Mangled chars in offending line of SyntaxError traceback -> Garbled chars in offending line of SyntaxError traceback __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1161> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1165] Should itertools.count work for arbitrary integers?
New submission from Eduardo Padoan: Currently, itertools.count.__next__ checks wether the current value is > PY_SSIZE_T_MAX and raises OverflowError if so. Shouldn't the value be stored as "long", at least in Py3k? Not that I have any use case for it, so it is minor. -- components: Library (Lib) messages: 55927 nosy: eopadoan severity: minor status: open title: Should itertools.count work for arbitrary integers? type: behavior versions: Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1165> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1165] Should itertools.count work for arbitrary integers?
Changes by Eduardo Padoan: -- severity: critical -> minor __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1165> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1161] Garbled chars in offending line of SyntaxError traceback
Eduardo Padoan added the comment: Can't reproduce this error anymore with revision 58472. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1161> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1151] "TypeError: expected string, bytes found" instead of KeyboardInterrupt
Eduardo Padoan added the comment: Can't reproduce this error anymore with revision 58472. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1151> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2046] patch to fix_import: UserDict -> collections
New submission from Eduardo Padoan: UserDict moved from UserDict module (deleted) to collections on py3k. This patch adds this case to fix_import.py on 2to3. -- assignee: collinwinter components: 2to3 (2.x to 3.0 conversion tool) files: fix_import_udict.diff messages: 62186 nosy: collinwinter, eopadoan, rhettinger severity: normal status: open title: patch to fix_import: UserDict -> collections type: behavior versions: Python 2.6, Python 3.0 Added file: http://bugs.python.org/file9388/fix_import_udict.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2046> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2060] python2.6 -3 gives "warning: callable() not supported in 3.x" on startup
New submission from Eduardo Padoan: Running python2.6 with the -3 option, you get 6 warnings about callable(): [EMAIL PROTECTED]:~/dev/svn/python2.6$ python2.6 -3 warning: callable() not supported in 3.x warning: callable() not supported in 3.x warning: callable() not supported in 3.x warning: callable() not supported in 3.x warning: callable() not supported in 3.x warning: callable() not supported in 3.x Python 2.6a0 (trunk:60717, Feb 10 2008, 19:53:48) [GCC 4.2.3 (Ubuntu 4.2.3-1ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> -- components: Interpreter Core messages: 62260 nosy: eopadoan severity: minor status: open title: python2.6 -3 gives "warning: callable() not supported in 3.x" on startup versions: Python 2.6 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2060> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2060] python2.6 -3 gives "warning: callable() not supported in 3.x" on startup
Eduardo Padoan added the comment: > By the way, what are you supposed you use in py3k instead of callable? Either "try: foo(); except TypeError: ...", or "if hasattr(foo, '__call__'): foo()". __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2060> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
New submission from Eduardo Morales : Running on MacOS Monterrey throws following error: ```malloc: *** error for object 0x7ffb5ea1a120: pointer being freed was not allocatedPython(4899,0x1061a8600)``` This started happening right after upgrading to the new MacOS Beta. -- messages: 399247 nosy: edumorlom priority: normal severity: normal status: open title: MacOS Monterrey malloc issue versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Eduardo Morales added the comment: I am not sure how to reproduce it, I think it's being caused by an imported library. -- ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Eduardo Morales added the comment: These are the libraries that I'm using: ``` ibm-db~=3.0.4 Flask~=2.0.1 pandas~=1.2.5 requests~=2.25.1 flask_basicauth~=0.2.0 schedule~=1.1.0 ``` -- ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Eduardo Morales added the comment: I've been able to tackle the issue down and reported it to the library. Although I'm not sure if it's a Python or library issue. https://github.com/ibmdb/python-ibmdb/issues/648 -- ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Eduardo Morales added the comment: Not sure if this is helpful, but I am attaching the MacOS bug log that is auto-generated when Python fails. -- Added file: https://bugs.python.org/file50267/bug.log ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Eduardo Morales added the comment: Also: (base) ➜ backend git:(development) ✗ PYTHONFAULTHANDLER=1 python3 server.py CONFIG_FILE=../config/development.json Python(27657,0x10839f600) malloc: *** error for object 0x7ffb4ac81d10: pointer being freed was not allocated Python(27657,0x10839f600) malloc: *** set a breakpoint in malloc_error_break to debug Fatal Python error: Aborted Current thread 0x00010839f600 (most recent call first): File "/Users/edumoralesibm/fantasy-football-health-dashboard/backend/server.py", line 26 in [1]27657 abort PYTHONFAULTHANDLER=1 python3 server.py -- ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44869] MacOS Monterrey malloc issue
Change by Eduardo Morales : -- versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue44869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35626] Python dictreader KeyError issue
New submission from Eduardo Orochena :
def load_file(filename):
with open(filename, 'r', encoding='utf-8') as fin:
header = fin.readline()
print('Found ' + header)
reader = csv.DictReader(fin)
for row in reader:
print(type(row), row)
print('Beds {} '.format(row['beds']))
This results in a KeyError exception
whilst
open_f = open(filename, 'r', encoding='utf-8')
read_it = csv.DictReader(open_f)
for i in read_it:
print('Beds {}'.format(i['beds']))
behaves as expected
--
components: Build
messages: 332810
nosy: eorochena
priority: normal
severity: normal
status: open
title: Python dictreader KeyError issue
type: crash
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue35626>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21697] shutil.copytree() handles symbolic directory incorrectly
Eduardo Seabra added the comment: Berker Peksag, I don't think your patch is okay. When symlinks is set to true, it should copy the symbolic link of the directory. Your code is calling copytree instead. I think the following patch is working, no errors on regression tests. -- nosy: +Eduardo.Seabra Added file: http://bugs.python.org/file35660/issue21697.patch ___ Python tracker <http://bugs.python.org/issue21697> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Eduardo Seabra added the comment: I've attached a patch with @mmarkk proposal. -- keywords: +patch nosy: +Eduardo.Seabra Added file: http://bugs.python.org/file35662/issue21579.patch ___ Python tracker <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21261] Teach IDLE to Autocomplete dictionary keys
Eduardo Seabra added the comment: >From the example, I couldn't know if the patch should also autocomplete int >and other types. So here's a patch that autocompletes string dictionary keys. I'm new contributing so let me know if I made anything wrong and I'll fix as soon as possible. -- keywords: +patch nosy: +Eduardo.Seabra Added file: http://bugs.python.org/file35700/issue21261.patch ___ Python tracker <http://bugs.python.org/issue21261> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18728] Increased test coverage for filecmp.py
Eduardo Seabra added the comment: I've also increased the test coverage of filecmp.py. Don't know if I should merge my patch with Alex.Volkov's patch. I'm uploading it as a separate patch. -- nosy: +Eduardo.Seabra Added file: http://bugs.python.org/file32801/test_filecmp.patch ___ Python tracker <http://bugs.python.org/issue18728> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21261] Teach IDLE to Autocomplete dictionary keys
Eduardo Seabra added the comment: I've added three lines to cls.code to test_hyperparser. So I can test for subscripts with double quotes, single quotes and with no strings at all. Should I implement try_open_completions_event for COMPLETE_DICTIONARY? Calling this event everytime someone types a string seemed a bit expensive in my opinion. I'm attaching the new patch. As fas as the signed contributor, I've already signed last week but still waiting. -- Added file: http://bugs.python.org/file35709/issue21261.patch ___ Python tracker <http://bugs.python.org/issue21261> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5395] array.fromfile not checking I/O errors
Eduardo Aguiar added the comment: Maybe you could create a file without read permission (000) and try to read from it. -- ___ Python tracker <http://bugs.python.org/issue5395> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5395] array.fromfile not checking I/O errors
Eduardo Aguiar added the comment:
Another try. I have opened a file for writing, and have tried to read
from it:
>>> fp = open ('xxx', 'w')
>>> fp.read ()
Traceback (most recent call last):
File "", line 1, in
IOError: [Errno 9] Bad file descriptor
--
___
Python tracker
<http://bugs.python.org/issue5395>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5395] array.fromfile not checking I/O errors
New submission from Eduardo Aguiar :
At arraymodule.c (line 1258):
nread = fread(item + (Py_SIZE(self) - n) * itemsize,
itemsize, n, fp);
if (nread < (size_t)n) {
Py_SIZE(self) -= (n - nread);
PyMem_RESIZE(item, char, Py_SIZE(self)*itemsize);
self->ob_item = item;
self->allocated = Py_SIZE(self);
PyErr_SetString(PyExc_EOFError,
"not enough items in file");
return NULL;
}
When fread returns 0, ferror should be called to check if it was an EOF
or an error condition. It is not handling OSErrors, such as I/O errors,
raising always "not enough items in file".
--
components: Library (Lib)
messages: 82936
nosy: aguiar
severity: normal
status: open
title: array.fromfile not checking I/O errors
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue5395>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5396] os.read not handling O_DIRECT flag
New submission from Eduardo Aguiar :
At posixmodule.c (line 6306)
static PyObject *
posix_read(PyObject *self, PyObject *args)
{
int fd, size, n;
PyObject *buffer;
if (!PyArg_ParseTuple(args, "ii:read", &fd, &size))
return NULL;
if (size < 0) {
errno = EINVAL;
return posix_error();
}
buffer = PyString_FromStringAndSize((char *)NULL, size);
if (buffer == NULL)
return NULL;
Py_BEGIN_ALLOW_THREADS
n = read(fd, PyString_AsString(buffer), size);
Py_END_ALLOW_THREADS
if (n < 0) {
Py_DECREF(buffer);
return posix_error();
}
if (n != size)
_PyString_Resize(&buffer, n);
return buffer;
}
os.read does not work with O_DIRECT flag. It fails with errno = EINVAL.
>From read(2) man page:
EINVAL fd is attached to an object which is unsuitable for
reading; or
the file was opened with the O_DIRECT flag, and
either the
address specified in buf, the value specified in count,
or the
current file offset is not suitably aligned.
if os.open is called with O_DIRECT flag enabled, the buffer used in
"read" must be page aligned and "size" must be multiple of pagesize also.
--
components: Library (Lib)
messages: 82938
nosy: aguiar
severity: normal
status: open
title: os.read not handling O_DIRECT flag
type: behavior
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue5396>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5396] os.read not handling O_DIRECT flag
Eduardo Aguiar added the comment: Hi, I think I have a few more issues you can consider: 1) to allocated an aligned buffer it is as simple as allocate 4096 + len (buffer) and truncate address to 4k boundary. 2) I wrote a floppy imager, and without O_DIRECT, it gives me 8 sectors (4k = kernel page) errors whenever one sector is bad. 3) There is os.O_DIRECT and os.open help page references it Best regards, Eduardo Aguiar -- ___ Python tracker <http://bugs.python.org/issue5396> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2811] doctest doesn't treat unicode literals as specified by the file declared encoding
New submission from Paulo Eduardo Neves <[EMAIL PROTECTED]>: Doctest doesn't obey the specified file encoding for unicode literals. I've put the minimum test case that demonstrate the error in the attached file. The program has the # -*- coding: utf-8 -*- as the first line and is saved in this encoding. My computer environment is configured as iso8859-1. Doctest ignores the file encoding specification and interprets the u'á' as u'á' (the utf-8 text decoded as iso8859-1 ) I've reproduced this error in python 2.5 in linux and windows. This is the output of the program below that runs the function normalize from inside doctest and directly from python. They show different results. ** File "doctesteerror.py", line 7, in __main__.normalize Failed example: normalize(u'á') Expected: u'b' Got: u'\xc3\xa1' ** 1 items had failures: 1 of 1 in __main__.normalize ***Test Failed*** 1 failures. without doctest ===>>> b -- components: Library (Lib) files: doctesteerror.py messages: 66559 nosy: neves severity: normal status: open title: doctest doesn't treat unicode literals as specified by the file declared encoding type: behavior versions: Python 2.5 Added file: http://bugs.python.org/file10265/doctesteerror.py __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2811> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4219] Problem with regular expression
New submission from Carlos Eduardo Klock <[EMAIL PROTECTED]>:
Hello,
I am having a weird problem with regex. I am trying to get the tokens
that match the pattern below, but it is not working only for a specific
case. I do this for many lines of text, and it works, except for the
string '1214578800'.
Any idea of what is happening? Is it a problem of my code or a bug in
regular expressions?
Thanks for any help,
Carlos.
import re
r =
re.compile(",'([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9])'[,|)]")
text =
"('25','2','3','2','0','1','0','0/350','30','21','5','','1211641200','1214578800','0','2','1214662622');"
timestamps = r.findall(text)
print timestamps
OUTPUT:
Python 2.5.2 (r252:60911, Jul 31 2008, 17:31:22)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on Trabalho15, Standard
>>> ['1211641200', '1214662622']
--
components: Regular Expressions
messages: 75286
nosy: carlosklock
severity: normal
status: open
title: Problem with regular expression
type: behavior
versions: Python 2.5
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4219>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4219] Problem with regular expression
Carlos Eduardo Klock <[EMAIL PROTECTED]> added the comment: Sorry, it is really a problem with the comma. Thanks for helping! :) ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4219> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32223] distutils doesn't correctly read UTF-8 content from config files
Eduardo Suarez-Santana added the comment: In my opinion, 'setup.cfg' is setuptools stuff and not part of python project. Anyway I think it is a good idea to read it as UTF-8. Setuptools seems to be defaulting to UTF-8: https://github.com/pypa/setuptools/pull/1735 -- nosy: +esuarezsantana ___ Python tracker <https://bugs.python.org/issue32223> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32398] GDAL compilation error
New submission from Eduardo Suarez-Santana : When compiling GDAL with python support, under certain build chain environment variables, next error may appear when building and linking python extensions (https://www.mail-archive.com/freebsd-ports@freebsd.org/msg41030.html): /bin/sh: -d: invalid option See the Github PR for proposed solution. -- components: Distutils messages: 308887 nosy: dstufft, eric.araujo, esuarezsantana priority: normal pull_requests: 4857 severity: normal status: open title: GDAL compilation error type: behavior versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue32398> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32398] OSX C++ linking workaround in distutils breaks other packages
Change by Eduardo Suarez-Santana : -- title: GDAL compilation error -> OSX C++ linking workaround in distutils breaks other packages ___ Python tracker <https://bugs.python.org/issue32398> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32398] OSX C++ linking workaround in distutils breaks other packages
Eduardo Suarez-Santana added the comment: When compiling GDAL with python support, before Lib/distutils/unixccompiler.py:180, self.compiler_cxx = ['/bin/sh', '/usr/x86_64-pc-linux-gnu/bin/libtool', '--mode=compile', '--tag=CXX', 'x86_64-pc-linux-gnu-c++'] linker = ['x86_64-pc-linux-gnu-cc', '-pthread', '-shared', '-march=native', '-pipe', '-O2', '-Wall', '-Wdeclaration-after-statement', '-Wextra', '-Winit-self', '-Wunused-parameter', '-Wmissing-prototypes', '-Wmissing-declarations', '-Wformat', '-Werror=format-security', '-Wno-format-nonliteral', '-Wlogical-op', '-Wshadow', '-Werror=vla', '-Wdeclaration-after-statement', '-Wnull-dereference', '-Wduplicated-cond', '-DOGR_ENABLED', '-march=native', '-pipe', '-O2', '-I/var/tmp/paludis/build/sci-libs-gdal-2.1.1/work/gdal-2.1.1/port', '-DGDAL_COMPILATION'] and after the workaround code: linker = ['/bin/sh', '-pthread', '-shared', '-march=native', '-pipe', '-O2', '-Wall', '-Wdeclaration-after-statement', '-Wextra', '-Winit-self', '-Wunused-parameter', '-Wmissing-prototypes', '-Wmissing-declarations', '-Wformat', '-Werror=format-security', '-Wno-format-nonliteral', '-Wlogical-op', '-Wshadow', '-Werror=vla', '-Wdeclaration-after-statement', '-Wnull-dereference', '-Wduplicated-cond', '-DOGR_ENABLED', '-march=native', '-pipe', '-O2', '-I/var/tmp/paludis/build/sci-libs-gdal-2.1.1/work/gdal-2.1.1/port', '-DGDAL_COMPILATION'] which leads to next error: /bin/sh: -d: invalid option Some fixes to compilation (https://www.michael-joost.de/gdal_install.html) have been found, but anyway the existing workaround: 1. ...is a hack about OSX but there is no platform checking, 2. ...assumes linker and compiler commands have similar structure and environment settings (no documentation reference found about that), and 3. ...assumes `env`, if used, does not come with any modifier. My proposal is to remove the workaround and delegate right environmental variables to packagers. -- ___ Python tracker <https://bugs.python.org/issue32398> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
New submission from Eduardo Robles Elvira: The patch attached provides implementation for multivolume support for tarfile module. It contains both the changes in the module and a battery of unit tests. It contains support for multivolume for both GNU and PAX formats. The main idea behind this proposal is that for dealing with new volumes, the user will provide a callback function. In this callback function the user typically calls to TarFile.open_volume(filename) with the appropiate filename. This is quite flexible in the sense that the callback function could even ask the user for the filename of the next volume (as tar command does), or do anything they need before returning or before calling to open_volume. Please feel free to comment on how to improve the implementation or the API. Documentation for the new feature will be provided at a later stage of the review process if the patch gets a good review. -- components: Library (Lib) files: cpython-tarfile-multivolume.patch keywords: patch messages: 191979 nosy: edulix priority: normal severity: normal status: open title: Multivolume support in tarfile module versions: Python 3.4 Added file: http://bugs.python.org/file30720/cpython-tarfile-multivolume.patch ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment: Sure, I will fill it out. But is it required? -- ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18575] Fixing tarfile._mode when using gzip via ":gz"
New submission from Eduardo Robles Elvira: A TarFile object constructor accepts a fileobj argument. When this argument is set and it has a "mode" property, tarfile._mode is blindly copied from there. Otherwise, mode is set using the mode argument in the constructor. This usually works, but in the case where fileobj is a gzip.GzipFile, the "mode" property is not a string like "rb" or "wb" but an integer, which is not expected in TarFile. This has not been noticed before probably because the TarFile._mode property is usually unused internally, but in my case it's a problem when using it together with tarfile multivolume mode provided in issue #18321. -- components: Library (Lib) files: gzip.patch keywords: patch messages: 193812 nosy: edulix priority: normal severity: normal status: open title: Fixing tarfile._mode when using gzip via ":gz" versions: Python 3.4 Added file: http://bugs.python.org/file31061/gzip.patch ___ Python tracker <http://bugs.python.org/issue18575> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18575] Fixing tarfile._mode when using gzip via ":gz"
Changes by Eduardo Robles Elvira : Removed file: http://bugs.python.org/file31061/gzip.patch ___ Python tracker <http://bugs.python.org/issue18575> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18575] Fixing tarfile._mode when using gzip via ":gz"
Eduardo Robles Elvira added the comment: Fixing gzip.patch, it was using basestring (python2) instead of str -- Added file: http://bugs.python.org/file31062/gzip.patch ___ Python tracker <http://bugs.python.org/issue18575> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11445] python.exe on OS X shared-llbrary build erroneously linked to MacPorts python library
Changes by Eduardo Cereto Carvalho : -- nosy: +Eduardo.Cereto.Carvalho ___ Python tracker <http://bugs.python.org/issue11445> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment: > The example I gave is based on the idea that there is a TarVolumeSet class in > the tarfile module that implements all the required file-object methods (e.g. > read(), write(), seek(), etc.) and acts as if the sequence of volumes is > actually one big file. It is passed to tarfile.open() as the fileobj > argument. This TarVolumeSet class is supposed to be subclassable to let the > user implement her/his own mode of operation. This way the open_volume() > method can do whatever the user wants it to do. The TarVolumeSet class might > as well have a new_volume() method for writing multivol archives, the example > only covered the case of reading a multivol archive. This is not so easy, because for example if you want to move the logic in addfile() that manages volumes to the write() function, you have some issues: * write() will need to take into account blocks (BLOCKSIZE), just to be able to split the volumes correctly. This is something that usually shouldn't belong in a write() function as it has to do to tarfile.. and it can be messy that both layers deal with it (write() splitting volumes, and tarfile adding NUL at the end of a BLOCK..) This can be done I guess, but remember, we split a volume only in the middle of a big file, not in any other case (AFAIK). Hopefully you don't get huge pax headers or anything strange. Usually all other records are either in the begining or just occupy one block, but can we rule this problem out for good? * multivolume logic in write() needs read/write access to the current tarinfo being written (look for "tarfile" in addfile() funcion in my patch to see why). How do you propose this object should be accessed from write()? > BTW, my version of GNU tar refuses to create compressed multiple-volume > archives which is why I doubt the usefulness of this feature overall. But it has multivolume support right? Which is what I am proposing here. Also, you can gzip (or encrypt or anything) the volumes after creating the volumes.. >> [...] because a multivol tarfile is not exactly the same as a normal tarfile >> chopped up. > No, I think it is exactly that. The only purpose of the GNUTYPE_MULTIVOL > header that is at the start of each subsequent volume is to give GNU tar the > ability to detect if it is reading the correct volume. It is not essential > and could as well be left out. I'm not going to discuss this, because I think that we can implement it in the way you propose to implement it anyway. But my patch supports it and I think it's an *useful* feature, so I want it in :-P -- ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment:
>> [...] but remember, we split a volume only in the middle of a big file, not
>> in any other case (AFAIK). Hopefully you don't get huge pax headers or
>> anything strange. [...]
> Hopefully? Sorry, but have you tested this? I did. I let GNU tar create a two
> volume archive that is split exactly between the two blocks of an XHDTYPE pax
> header.
>
> The result is terrifying. At the beginning of the second volume GNU tar
> creates an XGLTYPE header as the pax replacement for a GNUTYPE_MULTIVOL
> header, followed by an XHDTYPE header ("GNUFileParts") that somehow decorates
> the following REGTYPE(!) tar header that contains the continuation of the
> split XHDTYPE header data from the previous volume. After that comes the
> REGTYPE file that the split XHDTYPE header was actually meant for as
> decoration.
>
> I attached the archive to this issue.
>
> What happens if a GNUTYPE_LONGNAME header is split in two? I don't wanna
> know...
>
>> write() will need to take into account blocks (BLOCKSIZE), just to be able
>> to split the volumes correctly.
>
> It is mandatory to do the split on a block boundary (a multiple of 512).
>>> BTW, my version of GNU tar refuses to create compressed multiple-volume
>>> archives which is why I doubt the usefulness of this feature overall.
>> But it has multivolume support right? Which is what I am proposing here.
>> Also, you can gzip (or encrypt or anything) the volumes after creating the
>> volumes..
>
> Yeah, it has multivolume support, but a very limited one that is not only
> weird but isn't even usable together with compression. And sure, I can
> compress and encrypt the volumes afterward, but I can also create a
> compressed archive and pipe it through split(1) to split it into parts. Both
> ways create tar archives that are not readable by GNU tar because they're
> non-standard. So what?
>
> Please tell me, what is your actual personal use-case for this feature?
I'm willing modify the patch to remove the "weirdness" you refer to. I differ
on that it's not usable: it might not be useful to you, but it's certainly a
feature that covers part of the functionality of GNU tar. Actually, some of the
unit tests are like this: use GNU Tar to compress, then extract with tarfile -
and viceversa.
Of course you can use split. And I could use Ruby or Perl, but I'm using python
and tarfile, and this is a GNU tar feature that is just not supported in python
tarfile upstream, and I'm just trying to contribute this feature, if possible
:-).
BTW, If I create a multivol tar file and then compress the volumes, that does
not make it "non-standard", in the same way that if I create a PNG file and
then compress it and then store it in EXTFS, it doesn't make it non-standard.
I'm just using multiple layers of standards.
I'm a contractor, and I have been asked by a client to develop a python-based
backup tool. The client is technical and had already an idea of what he wanted
to do: use python-tarfile and add support to multivolume and some other
goodies, and the client also wanted to try to push the changes upstream as we
believe it is the correct thing to do.
BTW, when we designed the backup tool, we ruled out the possibility of using
split because split wouldn't allow to correctly list all the files in each
file-slice separately. We wanted to be able to recover all the files of each
"volume" so that if we lose other volumes, we can still recover all the data
from the volumes we have.
Anyway, if you are the maintainer of tarfile and you think it's not possible to
push tar-multivolume support upstream in python tarfile for whatever reason,
please tell me.
--
___
Python tracker
<http://bugs.python.org/issue18321>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21109] tarfile: Traversal attack vulnerability
Eduardo Robles Elvira added the comment: Do we have any final decision on what's the best approach to solve this? I see some possibilities: a) leave the issue to the library user. I think that's a not good solution security-wise as many will be unaware of the problem and this promotes code duplication for the fix. On the other hand, this does not change default behavior. b) fix the problem as proposed in the patch sent by Daniel. This makes the tarfile secure against this kind of attacks. It does change the behavior and doesn't allow to extract in arbitrary paths, though. c) fix the problem so that by default extracting in arbitrary paths is not allowed, but allow somehow to do that optionally. This way we change the default behavior but provide an easy fix for those that depend on that functionality. d) do not change the default, but provide a well documented and easy way to activate the safety checks that fix this kind of attacks. The advantage is that it doesn't change the default behavior, the disadvantage is that many people will have to modify their code to be secure, and that the default is not very secure. For what is worth, I believe either b or c should be chosen to fix this issue. -- nosy: +edulix ___ Python tracker <http://bugs.python.org/issue21109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21109] tarfile: Traversal attack vulnerability
Eduardo Robles Elvira added the comment: Also, I guess this patch solves and is closely related to #1044 which was, at the time (2007), considered "not a bug". -- ___ Python tracker <http://bugs.python.org/issue21109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment: could you please check if my contributor form is already processed? -- ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22163] max_wbits set incorrectly to -zlib.MAX_WBITS in tarfile, shouldn't be negative
New submission from Eduardo Robles Elvira: I think I have found a small typo-bug in tarfile.py, that seems to be present in cpython upstream, which makes tarfile compression slower. The issue can be seen here, in line 415 [1] of tarfile.py: self.cmp = self.zlib.compressobj(9, self.zlib.DEFLATED, -self.zlib.MAX_WBITS, self.zlib.DEF_MEM_LEVEL, 0) The minus sign doesn't make sense to me, because zlib.MAX_WBITS is 15, and according to the documentation [2] wbits is by default 15 and "This should be an integer from 8 to 15. Higher values give better compression, but use more memory". -15 is not even in range. So I guess the expected value should be the zlib default, 15. Or maybe another value, but it should at least be in range. I marked it as "performance" type bug as this only really affects performance. I might have gotten it all wrong and it's fine the way it is, but I thought it'd be best to report it, as it looked fishy to me. -- [1] http://hg.python.org/cpython/file/94d0e842b9ea/Lib/tarfile.py#l415 [2] https://docs.python.org/3.4/library/zlib.html#zlib.compressobj -- components: Library (Lib) files: max_wbits.patch keywords: patch messages: 225006 nosy: edulix priority: normal severity: normal status: open title: max_wbits set incorrectly to -zlib.MAX_WBITS in tarfile, shouldn't be negative type: performance versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file36299/max_wbits.patch ___ Python tracker <http://bugs.python.org/issue22163> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment: Do we have any news on this patch? -- ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment: > I cannot yet go into the details, because I have not tested the patch. > The comments, docstrings and quoting are not very consistent with the rest of > the module. There are a few spelling mistakes. I can try to take care of this, though it'd be best if you can point me out concrete examples. > The open_volume() method is more or less a copy of the open() method which is > not optimal. I know, but trying to do something else might be even worse.. > The patch adds a lot of complexity to the tarfile module for a use case that > only a few connoisseurs benefit from. It seems to alter some inherent TarFile > mechanics that people might rely on, e.g. the members attribute contains only > the members stored in the current volume not the overall entirety of members. Well, that doesn't make much sense to me. You say that people rely on something like "members attribute contains only the members stored in the current volume not the overall entirety of members", but as you know, python tarfile lib didn't support volumes before this patch, so I guess people couldn't be relying on that anyway. Also please bear in mind that tarfile volumes support is part of the tar standard, and altough not everyone uses this, it has its niche uses (sliced backups etc). > Does this patch reliably allow random-access? Yes and no. No: when you open a volume for reading, the list of members is cleared as you pointed so you cannot access to them. Yes: you can open any volume at the begining of a file, and it read the tarfile from there. I do that in my use-case. > Would it be possible/easier to add the same functionality using a separate > class MultiVolumeTarFile instead? If you find open_volume similar to open() and don't like, I'm quite sure you would like all the needed copy-paste having this a separate class would entail. Also as I said before, might not make much sense as this is part of the standard. -- ___ Python tracker <http://bugs.python.org/issue18321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18321] Multivolume support in tarfile module
Eduardo Robles Elvira added the comment:
I guess I got it wrong, it's not part of the POSIX standard, just part of the
GNU tar documentation. About the getmembers and getnames not reflecting the
entirety of the archive, it's an optimization I needed and I think ccan be
quite handy. It's also consistent with how the tar command works afaik, just
listing the contents of the current volume. But it's true that could be done.
Regarding the TarVolumeSet idea, it could work but it's not as easy as that.
You don't want to directly do a plain open in there, because you want to be
able to deal with read/write modes, with gzip/bzip/Stream class. You also want
to give the user maximum flexibility. That's why in my implementation
new_volume_handler and open_volume functions are separated. Similarly, we could
do something like this:
class MyTarVolumeSet(tarfile.TarVolumeSet):
def __init__(self, template, max_vol_size):
self.template = template
self.max_vol_size = max_vol_size
def new_volume_handler(self, tarfile, volume_number):
self.open_volume(self.template % volume_number, tarfile)
volumes = MyTarVolumesSet("test.tar.%03d")
with tarfile.open(fileobj=volumes, mode="w:") as tar:
for t in tar:
print(t.name)
Note that the new_volume_handler in this example receives more or less the same
params as in my patch (not the base name, because it's already stored in the
template), and that the open_volume really abstracts which way it will be
opened. It could also have, as in my patch, an optional fileobj parameter, for
a new indirection/abstraction.
In any case, this kind of abstraction would still really need some kind of
hooking with tarfile, because a multivol tarfile is not exactly the same as a
normal tarfile chopped up. This might be doable unilateraly by a smart
TarVolumeSet getting the information from the tarfile and modifying/patching
anything needed. This is one of the reasons why one would probably would still
need access to the tarfile inside the open_volume function.
--
___
Python tracker
<http://bugs.python.org/issue18321>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2441] Mac build_install.py script fetches unavailable SQLite version
New submission from Carlos Eduardo de Paula <[EMAIL PROTECTED]>: The build_installer.py script, used to create MacPython installers tries to fetch a SQLite version that is not available anymore. I provided a patch with an updated version and its corresponding hash. Maybe this should be backported to 2.5 and 2.6 branches. -- components: Installation files: build_installer.diff keywords: patch messages: 64218 nosy: carlosedp severity: normal status: open title: Mac build_install.py script fetches unavailable SQLite version type: behavior versions: Python 3.0 Added file: http://bugs.python.org/file9796/build_installer.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2441> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21253] unittest assertSequenceEqual can lead to Difflib.compare() crashing on mostly different sequences
Ernesto Eduardo Medina Núñez added the comment: While this gets fixed, can you provide a workaround? or recommend another library? -- nosy: +Ernesto Eduardo Medina Núñez ___ Python tracker <https://bugs.python.org/issue21253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
New submission from Eduardo A. Bustamante López: I found that http://en.wikipedia.org/robots.txt returns 403 if the provided user agent is in a specific blacklist. And since robotparser doesn't provide a mechanism to change the default user agent used by the opener, it becomes unusable for that site (and sites that have a similar policy). I think the user should have the possibility to set a specific user agent string, to better identify their bot. I attach a patch that allows the user to change the opener used by RobotFileParser, in case the need of some specific behavior arises. I also attach a simple example of how it solves the issue, at least with wikipedia. -- components: Library (Lib) files: robotparser.py.diff keywords: patch messages: 169718 nosy: Eduardo.A..Bustamante.López priority: normal severity: normal status: open title: Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default. type: enhancement versions: Python 2.7 Added file: http://bugs.python.org/file27100/robotparser.py.diff ___ Python tracker <http://bugs.python.org/issue15851> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
Changes by Eduardo A. Bustamante López : Added file: http://bugs.python.org/file27101/myrobotparser.py ___ Python tracker <http://bugs.python.org/issue15851> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
Eduardo A. Bustamante López added the comment: I guess a workaround is to do: robotparser.URLopener.version = 'MyVersion' -- ___ Python tracker <http://bugs.python.org/issue15851> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
Eduardo A. Bustamante López added the comment:
I'm not sure what's the best approach here.
1. Avoid changes in the Lib, and document a work-around, which involves
installing an opener with the specific User-agent. The draw-back is that it
modifies the behaviour of urlopen() globally, so that change affects any
other call to urllib.request.urlopen.
2. Revert to the old way, using an instance of a FancyURLopener (or URLopener),
in the RobotFileParser class. This requires a modification of the Lib, but
allows us to modify only the behaviour of that specific instance of
RobotFileParser. The user could sub-class FancyURLopener, set the appropiate
version string.
I attach a script, tested against the ``default`` branch of the mercurial
repository. It shows the work around for python3.3.
--
Added file: http://bugs.python.org/file27158/test.py
___
Python tracker
<http://bugs.python.org/issue15851>
___import urllib.robotparser
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'MyUa/0.1')]
urllib.request.install_opener(opener)
rp = urllib.robotparser.RobotFileParser('http://localhost:')
rp.read()
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
Eduardo A. Bustamante López added the comment: I forgot to mention that I ran a nc process in parallel, to see what data is being sent: ``nc -l -p ``. -- ___ Python tracker <http://bugs.python.org/issue15851> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
Eduardo A. Bustamante López added the comment: Hi Senthil, > I fail to see the bug in here. Robotparser module is for reading and > parsing the robot.txt file, the module responsible for fetching it > could urllib. You're right, but robotparser's read() does a call to urllib.request.urlopen to fetch the robots.txt file. If robotparser took a file object, or something like that instead of a Url, I wouldn't think of this as a bug, but it doesn't. The default behaviour is for it to fetch the file itself, using urlopen. Also, I'm aware that you shouldn't normally worry about setting a specific user-agent to fetch the file. But that's not the case of Wikipedia. In my case, Wikipedia returned 403 for the urllib user-agent. And since there's no documented way of specifying a particular user-agent in robotparser, or to feed a file object to robotparser, I decided to report this. Only after reading the source of 2.7.x and 3.x, one can find work-arounds for that problem, since it's not really clear how these make the requests for the robots.txt in the documentation. -- ___ Python tracker <http://bugs.python.org/issue15851> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
