[issue12800] 'tarfile.StreamError: seeking backwards is not allowed' when extract symlink
Change by Chris AtLee : -- pull_requests: +20149 pull_request: https://github.com/python/cpython/pull/20972 ___ Python tracker <https://bugs.python.org/issue12800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12800] 'tarfile.StreamError: seeking backwards is not allowed' when extract symlink
Chris AtLee added the comment:
It's caused by the combination of the symlink existing, and having the tarfile
opened in r| mode.
If I run the attached test file in a fresh directory, I get the following
exception:
raceback (most recent call last):
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2227, in makelink
os.symlink(tarinfo.linkname, targetpath)
FileExistsError: [Errno 17] File exists: 'message.txt' -> './symlink.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "../test.py", line 12, in
tf.extractall()
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2024, in extractall
self.extract(tarinfo, path, set_attrs=not tarinfo.isdir(),
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2065, in extract
self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2145, in _extract_member
self.makelink(tarinfo, targetpath)
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2237, in makelink
self._extract_member(self._find_link_target(tarinfo),
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2137, in _extract_member
self.makefile(tarinfo, targetpath)
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line
2176, in makefile
source.seek(tarinfo.offset_data)
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 513,
in seek
raise StreamError("seeking backwards is not allowed")
tarfile.StreamError: seeking backwards is not allowed
--
Added file: https://bugs.python.org/file49168/test.py
___
Python tracker
<https://bugs.python.org/issue12800>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12800] 'tarfile.StreamError: seeking backwards is not allowed' when extract symlink
Chris AtLee added the comment: Is there anything I can do to help get this landed? The PR in github works for me. -- nosy: +catlee ___ Python tracker <https://bugs.python.org/issue12800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19217] Calling assertEquals for moderately long list takes too long
Chris AtLee added the comment: Switching this to unified_diff allows the test case to finish nearly instantly. The downside is that the output is changed in the case of a difference found: FF == FAIL: test_compare (__main__.SCSE) -- Traceback (most recent call last): File "test_case.py", line 14, in test_compare self.assertEqual(arr1, arr2) AssertionError: Lists differ: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,[29953 chars]1, 1] != [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,[29953 chars]2, 2] First differing element 0: 1 2 Diff is 100034 characters long. Set self.maxDiff to None to see it. == FAIL: test_compare_2 (__main__.SCSE) -- Traceback (most recent call last): File "test_case.py", line 24, in test_compare_2 self.assertEqual(arr1, arr2) AssertionError: Lists differ: [1, 1[29932 chars] 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] != [1, 1[29932 chars] 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2] First differing element : 1 2 --- +++ @@ -9997,4 +9997,4 @@ 1, 1, 1, - 1] + 2] -- Ran 2 tests in 0.098s FAILED (failures=2) ------ nosy: +Chris AtLee ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19217> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19217] Calling assertEquals for moderately long list takes too long
Changes by Chris AtLee <chris.at...@gmail.com>: Added file: http://bugs.python.org/file41824/cpython-issue19217.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19217> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18167] cgi.FieldStorage fails to handle multipart/form-data when \r\n appears at end of 65535 bytes without other newlines
Chris AtLee added the comment: Thanks, your patch is definitely much simpler! I was worried about the case where you have interrupted \r\n that appears in the middle of the content. But that case is handled by the next readline(), which returns a single \n. One question about the tests you've attached - would it be better to be explicit about the line endings in check()? Do triple quoted strings in python always use \n for EOL regardless of the source code EOL format? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18167] cgi.FieldStorage fails to handle multipart/form-data when \r\n appears at end of 65535 bytes without other newlines
Chris AtLee added the comment: To demonstrate how to hit this in a real use case, run the attached script which implements a simple http server that saves POSTed files to a local file got_data. It returns the sha1sum of the POSTed file as the http response. Then, create a test file consisting of 65,535 null bytes, and submit with curl. - % dd if=/dev/zero of=data bs=1 count=65535 65535+0 records in 65535+0 records out 65535 bytes (66 kB) copied, 0.0890487 s, 736 kB/s - % sha1sum data 391edab7225a1de662ebc3a1a670a20d8e6a226b data - % curl -Fdata=@data http://localhost:8080/ 8dd623ef130a8cd3e97086101a6e1255a91fb916% -- Added file: http://bugs.python.org/file30529/test_cgi_server.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18167] cgi.FieldStorage fails to handle multipart/form-data when \r\n appears at end of 65535 bytes without other newlines
New submission from Chris AtLee: cgi.FieldStorage uses fp.readline(1 16) to read in POSTed file data if no content length has been specified. All HTTP clients I've looked at terminate the file body with CRLF and then the final MIME boundary. If the file body is 65,535 bytes, and doesn't contain \n or \r\n, then fp.readline(1 16) will return the original 65,535 bytes of the file plus the \r from the final \r\n sequence before the final boundary string. Since \r isn't considered a line ending, it gets considered as part of the POSTed file data, and you end up with an extra \r at the end of the file data. -- components: Library (Lib) files: cgi-test-cpython.patch keywords: patch messages: 190784 nosy: catlee priority: normal severity: normal status: open title: cgi.FieldStorage fails to handle multipart/form-data when \r\n appears at end of 65535 bytes without other newlines type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file30502/cgi-test-cpython.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18167] cgi.FieldStorage fails to handle multipart/form-data when \r\n appears at end of 65535 bytes without other newlines
Chris AtLee added the comment: This is a possible fix to this issue. It's not as clean as I'd like, but the simpler versions I tried could end up with the entire file contents in memory for degenerate (or malicious) inputs. The trick is handling the case where the current line ends with \r. We can't know if this is just a normal character in the file, or represents the end of a line until we see the start of the next line. -- Added file: http://bugs.python.org/file30503/cgi-cpython.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3243] Support iterable bodies in httplib
Chris AtLee [EMAIL PROTECTED] added the comment: The attached patch implements this for python 2.7. It also adds support for iterable bodies in urllib2, where it is more generally useful. urllib2 enforces the presence of a Content-Length header in the request if the body is an iterable, whereas httplib does not. The patch also includes updates to docs and tests (which all pass as of r67584 on my macbook) -- keywords: +patch Added file: http://bugs.python.org/file12241/python-3243.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3243 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3244] multipart/form-data encoding
Chris AtLee [EMAIL PROTECTED] added the comment: I also wrote some software to handle this: http://atlee.ca/software/poster/poster.encode.html The reason I wrote this is to avoid having the load the entire file into memory before posting the request. This, along with Issue #3243, would allow streaming uploads of files via HTTP POST. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3244 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3730] Update BaseHTTPServer docs
New submission from Chris AtLee [EMAIL PROTECTED]: The BaseHTTPServer docs don't mention 'server' as an instance variable in the instance variable section for BaseHTTPRequestHandler. It is mentioned in passing a few paragraphs above in the BaseHTTPServer class description, but it's too easy to miss there. Index: basehttpserver.rst === --- basehttpserver.rst (revision 66056) +++ basehttpserver.rst (working copy) @@ -68,6 +68,11 @@ address. + .. attribute:: server + + Contains the server instance. + + .. attribute:: command Contains the command (request type). For example, ``'GET'``. -- assignee: georg.brandl components: Documentation messages: 72144 nosy: catlee, georg.brandl severity: normal status: open title: Update BaseHTTPServer docs type: feature request versions: Python 2.5, Python 2.6, Python 3.0 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3730 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3243] Support iterable bodies in httplib
New submission from Chris AtLee [EMAIL PROTECTED]: httplib should support requests whose bodies are iterable objects. This would facilitate doing large file uploads via HTTP since you wouldn't have to load the entire file into memory to create the request string. Index: Lib/httplib.py === --- Lib/httplib.py (revision 64600) +++ Lib/httplib.py (working copy) @@ -688,7 +688,12 @@ self.__state = _CS_IDLE def send(self, str): -Send `str' to the server. +Send `str` to the server. + +``str`` can be a string object, a file-like object that supports +a .read() method, or an iterable object that supports a .next() +method. + if self.sock is None: if self.auto_open: self.connect() @@ -710,6 +715,10 @@ while data: self.sock.sendall(data) data=str.read(blocksize) +elif hasattr(str,'next'): +if self.debuglevel 0: print sendIng an iterable +for data in str: +self.sock.sendall(data) else: self.sock.sendall(str) except socket.error, v: -- components: Library (Lib) messages: 69014 nosy: catlee severity: normal status: open title: Support iterable bodies in httplib type: feature request versions: Python 2.7 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3243 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3244] multipart/form-data encoding
New submission from Chris AtLee [EMAIL PROTECTED]: The standard library should provide a way to encode data using the standard multipart/form-data encoding. This encoding is required to support file uploads via HTTP POST (or PUT) requests. Ideally file data could be streamed to the remote server if httplib supported iterable request bodies (see issue #3243). Mailing list thread: http://mail.python.org/pipermail/python-dev/2008-June/080783.html -- components: Library (Lib) messages: 69015 nosy: catlee severity: normal status: open title: multipart/form-data encoding type: feature request versions: Python 2.7 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3244 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3224] Small typo in 2.6 what's new
New submission from Chris AtLee [EMAIL PROTECTED]: Index: Doc/whatsnew/2.6.rst === --- Doc/whatsnew/2.6.rst(revision 64571) +++ Doc/whatsnew/2.6.rst(working copy) @@ -1284,7 +1284,7 @@ Here are all of the changes that Python 2.6 makes to the core Python language. * The :func:`hasattr` function was catching and ignoring all errors, - under the assumption that they meant a :meth:`__getattr__` method has + under the assumption that they meant a :meth:`__getattr__` method was failing somewhere and the return value of :func:`hasattr` would therefore be ``False``. This logic shouldn't be applied to :exc:`KeyboardInterrupt` and :exc:`SystemExit`, however; Python 2.6 will -- assignee: georg.brandl components: Documentation messages: 68868 nosy: catlee, georg.brandl severity: normal status: open title: Small typo in 2.6 what's new versions: Python 2.6 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3224 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1673409] datetime module missing some important methods
Chris AtLee added the comment: timedelta.todays() could be useful in general I suppose. I think timedelta.toweeks() could be omitted since it's simple division by an easy to recognize constant...also the timedelta docs say that it stores data in terms of microseconds, seconds, and days. OTOH tohours(), tominutes(), etc. (all the units that the constructor uses) could be useful in some cases. My feeling is that adding a method for each time unit bloats the API too much. Personally I've only ever wanted to know the number of seconds a timedelta represents. It seems like seconds are a good base unit to support since it allows easier interoperability with other python types and external data. _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1673409 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1673409] datetime module missing some important methods
Chris AtLee added the comment:
I keep needing to know the number of seconds that a timedelta represents,
so I implemented the following patch.
This returns only the integer portion, but could be modified to return a
floating point value.
--
nosy: +catlee
Added file: http://bugs.python.org/file8837/timedelta.diff
_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1673409
_Index: Doc/library/datetime.rst
===
--- Doc/library/datetime.rst (revision 59236)
+++ Doc/library/datetime.rst (working copy)
@@ -266,7 +266,13 @@
efficient pickling, and in Boolean contexts, a :class:`timedelta` object is
considered to be true if and only if it isn't equal to ``timedelta(0)``.
+Instance methods:
+.. method:: timedelta.tosecs()
+
+ Returns the total number of seconds this timedelta object represents.
+ This is equivalent to (timedelta.days * 3600 * 24) + timedelta.seconds
+
.. _datetime-date:
:class:`date` Objects
Index: Lib/test/test_datetime.py
===
--- Lib/test/test_datetime.py (revision 59236)
+++ Lib/test/test_datetime.py (working copy)
@@ -475,6 +475,11 @@
self.assertEqual(str(t3), str(t4))
self.assertEqual(t4.as_hours(), -1)
+def test_tosecs(self):
+for s in (100, 1000, 1000, 1000):
+td = timedelta(seconds = s)
+self.assertEqual(td.tosecs(), s)
+
#
# date tests
Index: Modules/datetimemodule.c
===
--- Modules/datetimemodule.c (revision 59236)
+++ Modules/datetimemodule.c (working copy)
@@ -2058,6 +2058,28 @@
return Py_BuildValue(ON, Py_Type(self), delta_getstate(self));
}
+static PyObject *
+delta_tosecs(PyDateTime_Delta* self)
+{
+PyObject*t1, *t2;
+PyObject*days, *secs;
+PyObject*retval;
+
+t1 = PyLong_FromLong(GET_TD_DAYS(self));
+t2 = PyLong_FromLong(3600*24);
+days = PyNumber_Multiply(t1, t2);
+Py_DECREF(t1);
+Py_DECREF(t2);
+
+secs = PyLong_FromLong(GET_TD_SECONDS(self));
+
+retval = PyNumber_Add(days, secs);
+Py_DECREF(days);
+Py_DECREF(secs);
+
+return retval;
+}
+
#define OFFSET(field) offsetof(PyDateTime_Delta, field)
static PyMemberDef delta_members[] = {
@@ -2077,6 +2099,8 @@
{__reduce__, (PyCFunction)delta_reduce, METH_NOARGS,
PyDoc_STR(__reduce__() - (cls, state))},
+{tosecs, (PyCFunction)delta_tosecs, METH_NOARGS,
+ PyDoc_STR(Total number of seconds represented by this timedelta object.)},
{NULL, NULL},
};
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
