[issue26877] tarfile use wrong code when read from fileobj
Марк Коренберг added the comment: The same in tarfile.copyfileobj() -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26877] tarfile use wrong code when read from fileobj
New submission from Марк Коренберг:
tarfile.py: _FileInFile():
(near line 687)
b = self.fileobj.read(length)
if len(b) != length:
raise ReadError("unexpected end of data")
every read() API does not guarantee that it will read `length` bytes. So, if
fileobj reads less than requestedm that is not an error (!)
In my case it was a pipe...
--
components: Library (Lib)
messages: 264450
nosy: mmarkk
priority: normal
severity: normal
status: open
title: tarfile use wrong code when read from fileobj
type: behavior
versions: Python 3.5
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26877>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Марк Коренберг added the comment: https://github.com/SpiderOak/ZipStream tries to implement streaming in one more kind. Also libarchive have experimental support of streaming write to zip archives in newest version. So problem is actual. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Марк Коренберг added the comment: Also, Python have problems with streaming READ of zip archive. I mean ability to read (in some form iterate over) archive when seeking is not available. I mean iteration like one in TAR archives. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Марк Коренберг added the comment: I have the same problem, and make monkey-patch by myself BEFORE seeing this issue (!) Example how I can do that is attached under name "socketpair.py". It will be nice if you take my idea. And after that streaming of zip files would be possible. -- nosy: +mmarkk Added file: http://bugs.python.org/file42311/main.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15948] Unchecked return value of I/O functions
Марк Коренберг added the comment: In a common case, if (write(thread.fd, thread.header, thread.header_len) == -1) should be replaced with if (write(thread.fd, thread.header, thread.header_len) != thread.header_len) -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue15948> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26501] bytes splitlines() method returns strings without decoding
Марк Коренберг added the comment:
$ python3.5
Python 3.5.0+ (default, Oct 11 2015, 09:05:38)
[GCC 5.2.1 20151010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> u'a\nb£'.encode('utf-8').splitlines()
[b'a', b'b\xc2\xa3']
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26501>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26501] bytes splitlines() method returns strings without decoding
Марк Коренберг added the comment: No, I have checked, it returns list of `bytes` objects. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19251] bitwise ops for bytes of equal length
Марк Коренберг added the comment: in order to increase perofrmance even more, use block operation on bytes. I.e. Xor by 8 bytes first (on 64-bit system) while size remainig is bigger or equal to 8, then by 4 bytes using same loop, and then xor remaining bytes by one byte. This will increase performance roughly to 8 times on 64bit systems and by 4 times on 32bit systems. See my PR https://github.com/KeepSafe/aiohttp/pull/687/files for details -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19251> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
Марк Коренберг added the comment: The main idea: if wile does not have a name, it should not have it! -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
Марк Коренберг added the comment:
1. nasty bugs not in my code!
2. It is not documented that name may be an int, so if applications rely on
undocumented stuff are definitely broken, so if IMHO we CAN change that.
3. It will be much stricter to delattr('name') instead of setting as None. This
will automatically cure issue21996 and issue22208.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26318>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
Марк Коренберг added the comment: oops, issue22208 is not related issue -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26342] Faster bit ops for single-digit positive longs
Марк Коренберг added the comment: You should add a tests. especially for edge cases, for negative values for example. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26342> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
Марк Коренберг added the comment: In particular, `io.open(fd, ...)` can not be used with tarfile, since it use it's "name" property, and think that it is real file name, and not expecte it to be int. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
Марк Коренберг added the comment: in any case, passing INTEGER as name is wrong thing, since (due to duck typing), name should be a string. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26318] `io.open(fd, ...).name` returns numeric fd instead of None
New submission from Марк Коренберг: `io.open(fd, ...).name` returns numeric fd instead of None. This lead to some nasty bugs. In order to bring consistency and make that predictable, please make `.name` for that case to return None. (and document it) -- components: IO, Library (Lib) messages: 259915 nosy: mmarkk priority: normal severity: normal status: open title: `io.open(fd, ...).name` returns numeric fd instead of None type: behavior versions: Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26237] UnboundLocalError error while handling exception
New submission from Марк Коренберг:
This works right in Python 2.7, but fails in python3:
UnboundLocalError: local variable 'e' referenced before assignment
def test():
try:
raise Exception('a')
except Exception as e:
pass
else:
return
print(e)
test()
--
messages: 259201
nosy: mmarkk
priority: normal
severity: normal
status: open
title: UnboundLocalError error while handling exception
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26237>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26158] File truncate() not defaulting to current position as documented
Марк Коренберг added the comment: text files and seek() offset: issue25849 -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26158> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25752] asyncio.readline - add customizable line separator
Марк Коренберг added the comment: During development, we decide not to change readline() function. So, yes, this issue is closed. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25752> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26050] Add new StreamReader.readuntil() method
Changes by Марк Коренберг <socketp...@gmail.com>: -- keywords: +patch Added file: http://bugs.python.org/file41595/asyncio-stream-doc-preliminary.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26050] Add new StreamReader.readuntil() method
Марк Коренберг added the comment: Attached patch, but did not compile documentation until text is validated. These descriptions are just copies of docstrings of corresponding functions. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26050] Add new StreamReader.readuntil() method
New submission from Марк Коренберг: See code discussion here: https://github.com/python/asyncio/pull/297 -- components: asyncio messages: 257776 nosy: gvanrossum, haypo, mmarkk, yselivanov priority: normal severity: normal status: open title: Add new StreamReader.readuntil() method type: enhancement versions: Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26037] Crash when reading sys.stdin.buffer in a daemon thread
Марк Коренберг added the comment: 16.2.4.3. Multi-threading FileIO objects are thread-safe to the extent that the operating system calls (such as read(2) under Unix) they wrap are thread-safe too. Binary buffered objects (instances of BufferedReader, BufferedWriter, BufferedRandom and BufferedRWPair) protect their internal structures using a lock; it is therefore safe to call them from multiple threads at once. >>>> TextIOWrapper objects are not thread-safe. <<<< Maybe problem here? -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26037> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26037] Crash when reading sys.stdin.buffer in a daemon thread
Марк Коренберг added the comment: sys.stdin sys.stdout sys.stderr ... These streams are regular text files ... -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26037> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19251] bitwise ops for bytes of equal length
Марк Коренберг added the comment: Just mention it here :) https://github.com/KeepSafe/aiohttp/issues/686 -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19251> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Changes by Марк Коренберг <socketp...@gmail.com>: -- resolution: -> wont fix status: open -> closed ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25927] add dir_fd for mkstemp, and also maybe to all tempfile.*
Марк Коренберг added the comment: Yes, can I do it on, say, github ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25927> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8604] Adding an atomic FS write API
Марк Коренберг added the comment: instead of realpath, os.lstat() may be used in order to detect if target file object is not plain file. Also, os.rename() cannot be used since it follows symlinks by default. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8604> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8604] Adding an atomic FS write API
Марк Коренберг added the comment: Also, modern kernels allows to replace entire directory! renameat2() + RENAME_EXCHANGE So, there should be more atomic operations -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8604> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: David Murray, in your code, if temporary file cannot be created, it attempts to unlink unknown name :). But, I already have almost the same solution in production. I mention it sources in issue8604. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25927] add dir_fd for mkstemp, and also maybe to all tempfile.*
New submission from Марк Коренберг: dir_fd support is good in `os` module. but tempfile module lack support of that great feature. Please add. -- messages: 256856 nosy: mmarkk priority: normal severity: normal status: open title: add dir_fd for mkstemp, and also maybe to all tempfile.* versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25927> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8604] Adding an atomic FS write API
Марк Коренберг added the comment: You also forgot about two things: 1. set temporary file permissions before rename 2. fsync(open(os.dirname(temporaryfile))) 3. if original file name is symlink, replace will works wrong. os.realpath should be used. So, here are real life function, that we use in production: # TODO: save_mtime, save_selinux, extended attr, chattrs and so on... # TODO: malicious user may replace directory with another while writing to file, # so we should use open directory first, and then use openat() and renameat() # with relative filename instead of specifying absolute path. # also NameTemporaryFile should allow to specify dir= @contextmanager def replace_file(path, save_perms=False, fsync=True, **kwargs): realpath = os.path.realpath(path) operating_dir = os.path.dirname(realpath) uid = None gid = None mode = None if save_perms: try: stinfo = os.lstat(realpath) uid = stinfo.st_uid gid = stinfo.st_gid mode = stinfo.st_mode except OSError as e: if e.errno != errno.ENOENT: raise with NamedTemporaryFile(dir=operating_dir, **kwargs) as fff: filedes = fff.fileno() if None not in (uid, gid, mode): os.fchown(filedes, uid, gid) os.fchmod(filedes, mode & 0o) yield fff # survive application crash fff.flush() if fsync: # survive power outage, is not required if that is temporary file os.fsync(filedes) os.rename(fff.name, realpath) # see http://bugs.python.org/issue21579 fff._closer.delete = False if fsync: # Sync directory: http://stackoverflow.com/questions/3764822/how-to-durably-rename-a-file-in-posix dirfd = os.open(operating_dir, os.O_RDONLY | os.O_CLOEXEC | os.O_DIRECTORY) try: os.fsync(dirfd) finally: os.close(dirfd) -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8604> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12797] io.FileIO and io.open should support openat
Марк Коренберг added the comment: But... os.openat() is still missing... why status is closed() ?! -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12797> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25927] add dir_fd for mkstemp, and also maybe to all tempfile.*
Changes by Марк Коренберг <socketp...@gmail.com>: -- components: +Library (Lib) ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25927> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment:
So, I'm required to re-implement NamedTemporaryFile by hand like that:
fd, name = tempfile.mkstemp(...)
try:
with io.open(fd, ...) as fff:
fff.write('hello')
fff.flush()
os.fdatasync(fff)
os.rename(name, ...)
except:
os.unlink(name)
raise
This is sad to see...
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue21579>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25752] asyncio.readline - add customizable line separator
Марк Коренберг added the comment: ReadStream.read_until() has been implemented instead enhancing readline(). -- resolution: -> wont fix ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25752> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: ping. THis is essential in our software, so we use private field in order to work-around this bug. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25898] Check for subsequence inside a sequence
Марк Коренберг added the comment: Really optimal search algorithm should be something like that: https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25898> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
Марк Коренберг added the comment: Well, 03e61104f7a2 adds good description, why not to enforce checks instead of saying that some values are unsupported ? Also, idea in returning special object instance from tell(), this object should incapsulate byte offset. And allow for the seek() either such objects or zero. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
Марк Коренберг added the comment: First, it seems that there are no real "reconstruction algorithm" at all. Seek is allowed to point to any byte position, even to place "inside" characters for multibyte encodings, such as UTF-8. Second, about performance: I talk about implementation mentioned in first message. If it is not used (and will not be used), we may forget about that sentence. Next, once again: I consider it is a bug in allowing to seek to invalid byte offsets for text files. Since we cannot easily calculate what offset will be valid (for example, seek past the end of file, or places inside character), just disallow seek. In real applications, no one will seek/peek to places other than * beginning of the file * current byte offset * seeking to the end of file. so this seeks/peeks must be allowed. This is applicable only to variable multibyte encodings (such as UTF-8). -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
Марк Коренберг added the comment: s/peek/tell/ -- status: closed -> open ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
Марк Коренберг added the comment: Also, can you provide the case, where such random seeks can be used on text files? It would be programmer error to seek to places other I mention. Does not it ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
Марк Коренберг added the comment:
https://docs.python.org/3.5/library/io.html?highlight=stringio#id3 :
Also, TextIOWrapper.tell() and TextIOWrapper.seek() are both quite slow due to
the reconstruction algorithm used.
What is reconstruction algorightm ? Experiments show, that seek() and tell()
returns values of count of bytes (not letters).
#!/usr/bin/python3.5
import tempfile
with tempfile.TemporaryFile(mode='r+t') as f:
l = f.write('привет')
print(l, f.tell()) # "6 12"
f.seek(3)
f.write('прекол42')
f.seek(0)
print(f.read()) # raise UnicodeDecodeError
So, please reopen. Issue is still here.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25849>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13033] Add shutil.chowntree
Марк Коренберг added the comment: note, that there are many other usecases for that function, like chattr, chmod g+w, touch and so on. But I'm personally consider this will bloat python library, since everyone can write it's own cycle over os.walk() in his program. Also, chown itself is allowed only for superuser (although chgrp is allowed in some cases to generic user), so this is very rare usecase, as I think. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue13033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13033] Add shutil.chowntree
Марк Коренберг added the comment:
Instead, it may be desirable to implement wrapper over ow.walk() that apply
given function to every member.
i.e.:
recursive_apply('/some/dir', lambda x: os.chown(x, 42, 42))
--
nosy: +mmarkk
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue13033>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25849] files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes
New submission from Марк Коренберг: It seems, that we should deprecate .seek() on files, opened in text mode. Since it is not possible to seek to position between symbols. Yes, it is possible to decode UTF-8 (or other charset) starting from beginning of the file and count symbols, but it is EXTREMELY SLOW, and is not what user expect. If so, seeking from end of file back to begin may be implemented in even more hard and error-prone way. Moreover, I consider that we should disallow seek in text files except seek() to begin of the file (position 0) or end of file (seek(0, SEEK_END)). Seel also issue25190 #25190 about something related for that. -- components: IO, Library (Lib), Unicode messages: 256291 nosy: ezio.melotti, haypo, mmarkk priority: normal severity: normal status: open title: files, opened in unicode (text): write() returns symbols count, but seek() expect offset in bytes versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25849> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25190] Define StringIO seek offset as code point offset
Марк Коренберг added the comment: #25849 ? -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25190> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25608] ascynio readexactly() should raise ValueError if passed length <= 0 in argument
Марк Коренберг added the comment: Okay, https://github.com/python/asyncio/pull/298 is ready -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25608> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24660] Heapq + functools.partial : TypeError: unorderable types
Марк Коренберг added the comment: Yes, this is not a bug. Python 3.5 works as expected. Orderable lambda is the bug in python2.7. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25752] asyncio.readline - add customizable line separator
Марк Коренберг added the comment: Well, https://github.com/python/asyncio/pull/297 is ready :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25752> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12460] SocketServer.shutdown() does not have "timeout=None" parameter
Марк Коренберг added the comment: Please close this bug since asyncio is much more suitable, and things like socketserver is obsolete as I think. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12460> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25752] asyncio.readline - add customizable line separator
Марк Коренберг added the comment: When I reported that bug, I want to suggest multibyte separators, but implementation is much complex than current one. For example, we should decide what to do if partial separator is read and EOF occur. So, I think we should merge ability to use custom one-byte separator, and next create another pull-request to make support of multibyte separator (like '\r\n'). Okay, I can make pull request on asyncio, but asyncio now is the part of Python. Does not it ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25752> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18161] call fchdir if subprocess.Popen(cwd=integer|fileobject)
Марк Коренберг added the comment: Do not understand what should be done in order to fix that... It will be nice if I can create pull request by myself -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18161> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24523] coroutine asyncio.wait() does not preserve order of elements
Марк Коренберг added the comment: Yes this is not a bug. Use asyncio.gather(), it preserves order of elements :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24523> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15500] Python should support naming threads
Changes by Марк Коренберг <socketp...@gmail.com>: -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue15500> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25608] ascynio readexactly() should raise ValueError if passed length <= 0 in argument
Марк Коренберг added the comment: result of readexactly(0) is indistinguishable from EOF, so no one should try to use that as I think. What is the useful case when zero bytes should be read ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25608> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25756] asyncio WriteTransport documentation typo
New submission from Марк Коренберг: Here is the match against master. Doc/library/asyncio-protocol.rst: @@ -156,9 +156,9 @@ WriteTransport high-water limit is given, the low-water limit defaults to an implementation-specific value less than or equal to the high-water limit. Setting *high* to zero forces *low* to zero as - well, and causes :meth:`pause_writing` to be called whenever the + well, and causes :meth:`resume_writing` to be called whenever the buffer becomes non-empty. Setting *low* to zero causes - :meth:`resume_writing` to be called only once the buffer is empty. + :meth:`pause_writing` to be called only once the buffer is empty. Use of zero for either limit is generally sub-optimal as it reduces opportunities for doing I/O and computation concurrently. -- assignee: docs@python components: Documentation messages: 255508 nosy: docs@python, mmarkk priority: normal severity: normal status: open title: asyncio WriteTransport documentation typo type: enhancement versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25756> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: No activity last week ? Why not to merge ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: Please merge patch, proposed by Eduardo Seabra (Eduardo.Seabra). And document that this is UNIX-only solution. -- versions: +Python 3.5, Python 3.6 Added file: http://bugs.python.org/file41055/test_tempfile.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25608] ascynio readexactly() should raise ValueError if passed length <= 0 in argument
New submission from Марк Коренберг: ascynio readexactly() should raise ValueError if passed length <= 0 in argument. Now, it return empty string, which is just error hiding. Why not to raise ValueError ? Returning empty string is error prone. This behaviour was not changed since initial commit of asyncio, so probably was not done intentionally. -- components: asyncio messages: 254545 nosy: gvanrossum, haypo, mmarkk, yselivanov priority: normal severity: normal status: open title: ascynio readexactly() should raise ValueError if passed length <= 0 in argument type: behavior versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25608> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21247] test_asyncio: test_subprocess_send_signal hangs on Fedora builders
Марк Коренберг added the comment: Bug still reproduced. Jenkins running from init.d use /usr/bin/daemon. This mean SIGHUP will be in SIG_IGN state. Since echo.py does not setup sighup handler, sighup will be equivalent of SIGKILL. So, why not to use, say, SIGTERM instead? After such change all tests passed. If not, signal handling tests should reset signal handling to SIG_DFL. Please reopen -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21247> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25420] "import random" blocks on entropy collection on Linux with low entropy
Марк Коренберг added the comment: Man getrandom() As of Linux 3.19, the following bug exists: * Depending on CPU load, getrandom() does not react to interrupts before reading all bytes requested. So, is it goot to use this syscall now? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25420> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21515] Use Linux O_TMPFILE flag in tempfile.TemporaryFile?
Марк Коренберг added the comment:
Okay, seemes it is not documented that
os.open('.', os.O_RDWR |os.O_EXCL | os.O_DIRECTORY)
Should return EISDIR
I did not found that in Linux manpages. Using undocumented features is bad.
Maybe I should report this to Michael Kerrisk to update manpage ?
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue21515>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21515] Use Linux O_TMPFILE flag in tempfile.TemporaryFile?
Марк Коренберг added the comment: Well, it's not said explicit, that O_DIRECTORY cannot be combined with O_RDWR. So, everything is valid now, very hacky, but works without bugs. It will be nice, if someone comment that hacks in source code -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21515] Use Linux O_TMPFILE flag in tempfile.TemporaryFile?
Марк Коренберг added the comment: Huge thanks for that patch. Now things are much cleaner. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18673] Add O_TMPFILE to os module
Марк Коренберг added the comment: Just for link. Issue #21515 — tempfile use this functionality now. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18673> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21515] Use Linux O_TMPFILE flag in tempfile.TemporaryFile?
Марк Коренберг added the comment: Suppose conditions: - Old linux kernel ignoring flag - malicious hacker force use of PLAIN FILE instead of directory On new kernel it will fail On old kernel it will just open that file! So, we can make a HACK! Just add last slash to directory name. This will not hurt on new kernels, but protect on old kernels. tests should also test a case when directory is symlink really. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25420] "import random" blocks on entropy collection on Linux with low entropy
Марк Коренберг added the comment: Just install rngd and setup it to user /dev/urandom as entropy source. I think thread is closed :) -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25420> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12939] Add new io.FileIO using the native Windows API
Марк Коренберг added the comment: Please reopen. I still have interest for that. Fact that we survive 8 years is not enough for closing bugs as "Won't fix". This fact point only on that ther are no good specialists that can fix that bug, since it is Windows-only, actually. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12939] Add new io.FileIO using the native Windows API
Марк Коренберг added the comment: Sorry, for the "there are no good specialist". I mean "There are no well motivated good specialist" :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24660] Heapq + functools.partial : TypeError: unorderable types
New submission from Марк Коренберг: import heapq from functools import partial qwe = [(0, lambda x: 42), (0, lambda x: 56)] heapq.heapify(qwe) qwe = [(0, partial(lambda x: 42)), (0, partial(lambda x: 56))] heapq.heapify(qwe) Traceback (most recent call last): File /usr/lib/python3.4/code.py, line 90, in runcode exec(code, self.locals) File input, line 1, in module TypeError: unorderable types: functools.partial() functools.partial() So it is not realiable to use heapq if element of the array is (int, callable) -- components: Library (Lib) messages: 246894 nosy: mmarkk priority: normal severity: normal status: open title: Heapq + functools.partial : TypeError: unorderable types type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24660 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24660] Heapq + functools.partial : TypeError: unorderable types
Марк Коренберг added the comment: Exactly the same with bound class methods -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24660 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24532] asyncio.sock_recv() blocks normal ioloop actions.
New submission from Марк Коренберг:
Suppose that program:
import asyncio
import socket
def receiver(loop):
(a, b) = socket.socketpair()
loop.call_later(1, lambda: print('Should be called inside the loop'))
end = loop.time() + 3
print('Starting busy receiver')
while loop.time() end:
a.send(b'test')
yield from loop.sock_recv(b, 65536)
# yield from asyncio.sleep(0) # =
print('Busy receiver complete')
# just not to stop ioloop immediatelly
yield from asyncio.sleep(0.5)
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(receiver(loop))
loop.close()
if __name__ == '__main__':
main()
Without asyncio.sleep(0) it will not fire time-delayed calls! If I add
asyncio.sleep(0), everything work right. As I think, It is because recv()
syscall is always succeeded, and we never return to ioloop (to epoll() I mean).
In other words, it is classical `reader starvation` as mentioned in man epoll.
It is not documented, that this function may block event loop, in spite of it
returns coroutine! I thought that this function will setup EPOLLIN event for
socket's FD + call recv() after that. I spent many time to debug program.
--
components: asyncio
messages: 245952
nosy: gvanrossum, haypo, mmarkk, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.sock_recv() blocks normal ioloop actions.
type: behavior
versions: Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24532
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24532] asyncio.sock_recv() blocks normal ioloop actions.
Марк Коренберг added the comment: $ PYTHONASYNCIODEBUG=1 ./bug.py Starting busy receiver Busy receiver complete Executing Task pending coro=receiver() running at ./bug.py:16 wait_for=Future pending cb=[Task._wakeup()] created at /usr/lib/python3.4/asyncio/tasks.py:490 cb=[_run_until_complete_cb() at /usr/lib/python3.4/asyncio/base_events.py:123] created at /usr/lib/python3.4/asyncio/base_events.py:296 took 3.001 seconds Should be called inside the loop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24532 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24532] asyncio.sock_recv() blocks normal ioloop actions.
Марк Коренберг added the comment: Adding of b.setblocking(0) after socketpair() does not help. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24532 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24523] coroutine asyncio.wait() does not preserve order of elements
New submission from Марк Коренберг: If I pass an list of futures/coroutines as (done, _) = asyncio.wait([...]), order of elements in `done` does NOT match order of elements in `wait`'s argument. This is not documented, and I don't know if that is a bug or not. Also, documentation say: Wait for the Futures and coroutine objects given by the SEQUENCE futures to complete. Really, this should be SET, since order is not preserved. Or, sources should be patched to preserve the order. I think, it is not hard to preserve order [in asyncio code]. P.S. For example, in Tornado, it is guaranteed (and documented) that order is preserved, so I can write [res1, res2] = yield [fut1(), fut2()] and it is guaranteed, that res1 - is a result of fut1() -- components: asyncio messages: 245899 nosy: gvanrossum, haypo, mmarkk, yselivanov priority: normal severity: normal status: open title: coroutine asyncio.wait() does not preserve order of elements type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24523 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24509] Undocumented features of asyncio: call_at, call_later
Changes by Марк Коренберг socketp...@gmail.com: -- assignee: - docs@python components: +Documentation, asyncio nosy: +docs@python, gvanrossum, haypo, yselivanov type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24509 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24509] Undocumented features of asyncio: call_at, call_later
New submission from Марк Коренберг: These function returns handle, so, registered callback can be cancelled. -- messages: 245803 nosy: mmarkk priority: normal severity: normal status: open title: Undocumented features of asyncio: call_at, call_later versions: Python 3.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24509 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24509] Undocumented features of asyncio: call_at, call_later
Марк Коренберг added the comment: For delayed execution methods: Returned handle is actually timer object. That timer can be deactivated using asyncio.Handle.cancel() method, so registered callback won't be called. For other callback registration methods: Returned handle may be used to unregister callback using asyncio.Handle.cancel(), so registered callback won't be called. Also: 1. Documentation of .cancel does not say if it is allowed to cancel cancelled handle. 2. Documentation does not say if it is allowed to cancel timer, that was already fired. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24509 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: So, maybe API change? like delete=True|False|Maybe ? don't think that this is good decisions. My approach is based on ext4 behaviour about delayed allocation and atomic file replacements. In my case, either old file (with contents) or new file appear. In any case, corrupted data cannot appear. This is required behaviour for my application. https://www.kernel.org/doc/Documentation/filesystems/ext4.txt http://lwn.net/Articles/322823/ === auto_da_alloc(*)Many broken applications don't use fsync() when noauto_da_alloc replacing existing files via patterns such as fd = open(foo.new)/write(fd,..)/close(fd)/ rename(foo.new, foo), or worse yet, fd = open(foo, O_TRUNC)/write(fd,..)/close(fd). If auto_da_alloc is enabled, ext4 will detect the replace-via-rename and replace-via-truncate patterns and force that any delayed allocation blocks are allocated such that at the next journal commit, in the default data=ordered mode, the data blocks of the new file are forced to disk before the rename() operation is committed. This provides roughly the same level of guarantees as ext3, and avoids the zero-length problem that can happen when a system crashes before the delayed allocation blocks are forced to disk. === So, this is essential functionality. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21579 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: why not to make tmpfileobj.delete as property that really sets self._closer.delete ? this will fix my problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21579 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Марк Коренберг added the comment: Is issue 21579 fixed in that bug? too many letters..sorry... -- nosy: +mmarkk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
Марк Коренберг added the comment: Yes, but O_TMPFILE should be set ONLY when used with TemporaryFile, not with NamedTemporaryFile. My problem refer only NamedTemporaryFile. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21579 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21579] Python 3.4: tempfile.close attribute does not work
New submission from Марк Коренберг: Suppose code: = import os import tempfile want_to_replace = 'zxc.dat' tmpdir=os.path.dirname(os.path.realpath(want_to_replace)) with tempfile.NamedTemporaryFile(dir=tmpdir) as fff: # do somewhat with fff here... and then: fff.flush() os.fdatasync(fff) os.rename(fff.name, want_to_replace) fff.delete = False = In python 3.3 and lower that works FINE. In Python 3.4 the fff._closer attribute was introduced, so fff.close=False stopped to work. I think this is major loss of functionality. The close attribute was not marked as private, so may be used in past. -- components: Library (Lib) messages: 219132 nosy: mmarkk priority: normal severity: normal status: open title: Python 3.4: tempfile.close attribute does not work type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21579 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8052] subprocess close_fds behavior should only close open fds
Марк Коренберг added the comment: Calling getdents()/readdir64() repeatedly while closing descriptors provides unexpected behaviour. Reading directory while it modified is not safe by default. For example: http://en.it-usenet.org/thread/18514/15719/. So, we should re-open directory if we received full array of structures. I don't know if just lseek(dirfd, 0) sufficies. Please reopen bug, as Linux behaviour of stable reading /proc/pid/fd may be broken in future without any error at python side (!) (typically, second call returns empty list if dir was modified) -- nosy: +mmarkk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8052 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8052] subprocess close_fds behavior should only close open fds
Марк Коренберг added the comment: Also, it is not said in manual if getdents() may be interrupted by signal. Assuming current code, error is not checked, so some (or all) descriptors will be skipped in case of error. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8052 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20979] Calling getdents()/readdir64() repeatedly while closing descriptors provides unexpected behaviour.
New submission from Марк Коренберг: 1. Please see last comments/patches for issue8052 2. Not closing some descriptos is security breach (PEP-0446 describes that) = Calling getdents()/readdir64() repeatedly while closing descriptors provides unexpected behaviour. Reading directory while it modified is not safe by default. For example: http://en.it-usenet.org/thread/18514/15719/. So, we should re-open directory if we received full array of structures. I don't know if just lseek(dirfd, 0) sufficies. Please reopen bug, as Linux behaviour of stable reading /proc/pid/fd may be broken in future without any error at python side (!) (typically, second call returns empty list if dir was modified) = Also, please check exit code of getdents() instead of just ignoring error and NOT closing file descriptors. = P.S. Please set affected python versions... -- components: Library (Lib) messages: 214100 nosy: mmarkk priority: normal severity: normal status: open title: Calling getdents()/readdir64() repeatedly while closing descriptors provides unexpected behaviour. type: security ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20979 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
New submission from Марк Коренберг: Results or running attached program: $ python2.7 qwe.py TCP mode, makefile method. 198807.2 lines per second (189.6 MB/s). Delay is 5.03 seconds TCP mode, fdopen method. 1041666.7 lines per second (993.4 MB/s). Delay is 0.96 seconds UNIX mode, makefile method. 2040816.3 lines per second (1946.3 MB/s). Delay is 0.49 seconds UNIX mode, fdopen method. 1923076.9 lines per second (1834.0 MB/s). Delay is 0.52 seconds $ python3.2 qwe.py TCP mode, makefile method. 275482.1 lines per second (262.7 MB/s). Delay is 3.63 seconds TCP mode, fdopen method. 909090.9 lines per second (867.0 MB/s). Delay is 1.10 seconds UNIX mode, makefile method. 323624.6 lines per second (308.6 MB/s). Delay is 3.09 seconds UNIX mode, fdopen method. 1694915.3 lines per second (1616.4 MB/s). Delay is 0.59 seconds -- 1. in every case, socket.makefile() is MUCH slower than os.fdopen() when used as for line in fileobject 2. compare speeds between python 2.7 and python 3.2 in same operation. Especially, socketpair+makefile 3. Why not to return os.fdopen() for sockets when socket.makefile() called on unix systems? -- components: IO, Library (Lib) files: qwe.py messages: 192044 nosy: mmarkk priority: normal severity: normal status: open title: for line in socket.makefile() speed degradation type: performance versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file30731/qwe.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
Марк Коренберг added the comment: Yes, results are repeatable, and for python 2.7 I have roughly same timings for UNIX socket. Also, I have straced all variants and see that in all 4 cases (and for both python versions) IO is done using 8192 blocks in size, so buffering is not cause of problems. I have linux 3.5.0, Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
Марк Коренберг added the comment: Well, python 3.3 is slightly faster: $ python3.3 qwe.py TCP mode, makefile method. 380228.1 lines per second (362.6 MB/s). Delay is 2.63 seconds TCP mode, fdopen method. 877193.0 lines per second (836.6 MB/s). Delay is 1.14 seconds UNIX mode, makefile method. 469483.6 lines per second (447.7 MB/s). Delay is 2.13 seconds UNIX mode, fdopen method. 1562500.0 lines per second (1490.1 MB/s). Delay is 0.64 seconds But problem still exists -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
Changes by Марк Коренберг socketp...@gmail.com: Removed file: http://bugs.python.org/file30731/qwe.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
Марк Коренберг added the comment: Eliminate unicode conversion for python3, but results still the same $ python2.7 qwe.py TCP mode, makefile method. 211416.5 lines per second (201.6 MB/s). Delay is 4.73 seconds TCP mode, fdopen method. 1041666.7 lines per second (993.4 MB/s). Delay is 0.96 seconds UNIX mode, makefile method. 2040816.3 lines per second (1946.3 MB/s). Delay is 0.49 seconds UNIX mode, fdopen method. 1886792.5 lines per second (1799.4 MB/s). Delay is 0.53 seconds $ python3.2 qwe.py TCP mode, makefile method. 487804.9 lines per second (465.2 MB/s). Delay is 2.05 seconds TCP mode, fdopen method. 900900.9 lines per second (859.2 MB/s). Delay is 1.11 seconds UNIX mode, makefile method. 625000.0 lines per second (596.0 MB/s). Delay is 1.60 seconds UNIX mode, fdopen method. 1492537.3 lines per second (1423.4 MB/s). Delay is 0.67 seconds $ python3.3 qwe.py TCP mode, makefile method. 512820.5 lines per second (489.1 MB/s). Delay is 1.95 seconds TCP mode, fdopen method. 884955.8 lines per second (844.0 MB/s). Delay is 1.13 seconds UNIX mode, makefile method. 680272.1 lines per second (648.8 MB/s). Delay is 1.47 seconds UNIX mode, fdopen method. 1587301.6 lines per second (1513.8 MB/s). Delay is 0.63 seconds Also, io.BufferedReader() is not my case. I understand that intermediate buffer will increase performance. Problem in implementation of socket.makefile(). That is I want to be fixed. -- Added file: http://bugs.python.org/file30732/qwe.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18329] for line in socket.makefile() speed degradation
Марк Коренберг added the comment: Can anyone test in python 3.4 ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18329 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18161] call fchdir if subprocess.Popen(cwd=integer|fileobject)
New submission from Марк Коренберг: Today, subprocess allow to change directory only by using its name. What if I have only file descriptor referring to that dir? It will be nice if such feture will be implemented. Now, I use preexc_fn to call os.fchdir() by hand. We should handle close_fds *AFTER* calling fchdir() :) -- components: Library (Lib) messages: 190770 nosy: mmarkk priority: normal severity: normal status: open title: call fchdir if subprocess.Popen(cwd=integer|fileobject) type: enhancement versions: Python 3.3, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18161 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18006] Set thread nema in linux kernel
New submission from Марк Коренберг: In linux (Since 2.6.9) we can use syscall prctl(PR_SET_NAME, Some thread name) to set thread name to the kernel. This thread is seen under this name in some process tool (like top, ps, pstree (have bug reported connected with this) and others). It will be nice if this syscall will be called (from correspoding thread=TID) when changing (setting) thread name. Note, that in current implementation name will be truncated to 15 bytes in kernel. This work very well using ctypes or python-prctl module. Also there is error in manpage about prctl saying that name is set to process (already sent to maintainer). Really, name is set to each thread. -- components: Library (Lib) messages: 189507 nosy: mmarkk priority: normal severity: normal status: open title: Set thread nema in linux kernel type: enhancement versions: Python 3.3, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18006 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18006] Set thread name in linux kernel
Changes by Марк Коренберг socketp...@gmail.com: -- title: Set thread nema in linux kernel - Set thread name in linux kernel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18006 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16858] tarfile silently hides errors
Марк Коренберг added the comment: Ups. hiding EOFHeaderError is not an error. But handilng of other errors is not perfect. Please review TarFile.next() for cases where .tar file is corrupted. For example, TruncatedHeaderError is re-raised only if problem at the start of the file. Really, it can appear in the (original) middle of the file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16858 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12939] Add new io.FileIO using the native Windows API
Марк Коренберг added the comment: Yes, re-writing windows IO to direct API, without intemediate layer is still needed. Please don't close bug. Maybe someone will implement this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14810] tarfile does not support timestamp older 1970-01-01
Марк Коренберг added the comment: Yes, bug exists in python 2.7. The same problem - negative timestamp in mtime field. It prepresented in binary value as '\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xc7\xfc' in my case. -- nosy: +mmarkk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14810 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14810] tarfile does not support timestamp older 1970-01-01
Марк Коренберг added the comment: And yes, bug does not appear in python3.2 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14810 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
