Alexander Mohr added the comment:
btw my work-around was to set maxheaderlen=sys.maxsize, worked for AWS SES at
least
--
___
Python tracker
<https://bugs.python.org/issue44
Change by Alexander Mohr :
--
title: EmailMessage incorrectly splits reply-to header -> EmailMessage
incorrectly splits name and address header
___
Python tracker
<https://bugs.python.org/issu
New submission from Alexander Mohr :
If you have a reply-to list that contains a name with a comma it must be
quoted, however if the line length is just right python with split the name
incorrectly and not keep the quote. Note that the CC line keeps the quote in
the name but the reply
Alexander Mohr added the comment:
so I did some investigation into the difference between aiohttp + httpx and it
appears that for some reason httpx does an extra read after the connection is
closed.
--
___
Python tracker
<https://bugs.python.
Alexander Mohr added the comment:
I've updated https://repl.it/@thehesiod/PristineBonyBugs#main.py to contain
both httpx + aiohttp testcases, and now httpx reproduces almost immediately and
aiohttp is still ok
--
___
Python tracker
<ht
Alexander Mohr added the comment:
as an FYI I've reproduced this with the httpx library as well:
https://repl.it/repls/PristineBonyBugs#main.py. It reproduces on this site but
I've been unable to reproduce it locally. It does always reproduce on our
production systems.
what's interesting
Change by Alexander Mohr :
--
nosy: +thehesiod
___
Python tracker
<https://bugs.python.org/issue36098>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Alexander Mohr :
--
nosy: +thehesiod
___
Python tracker
<https://bugs.python.org/issue38529>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexander Mohr added the comment:
@yselivanov even better :)
--
___
Python tracker
<https://bugs.python.org/issue31033>
___
___
Python-bugs-list mailin
Alexander Mohr added the comment:
I hit this as well in debian:jessie, and if I enabled -std=c99 I hit this
issue: https://github.com/dvarrazzo/py-setproctitle/issues/62
--
nosy: +thehesiod
___
Python tracker
<https://bugs.python.org/issue36
Alexander Mohr added the comment:
going to close, I've verified that it fixes my original issue, ty!!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Alexander Mohr added the comment:
doing the __future__ doesn't help anything as it is an invalid type
--
___
Python tracker
<https://bugs.python.org/issue36
Alexander Mohr added the comment:
I'm not a typing expert, but from what I understand you could do:
class Bar: pass
foo: Callable[[], Bar] = weakref.ref(Bar())
but you lose the typing weakref.ref.
OTOH if you simply do:
foo: weakref.ref = weakref.ref(Bar())
you lose the info of Bar
New submission from Alexander Mohr :
For valid types which encapsulate other types, like typing.List or
typing.Tuple, you can declare what's contained in them like so:
foo: typing.List[int] = []
Unfortunately weakref.ReferenceType does not allow you to do this. You get the
error:
>>
Change by Alexander Mohr :
--
versions: +Python 3.6
___
Python tracker
<https://bugs.python.org/issue34745>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexander Mohr added the comment:
sorry, by "raw" I mean in the context of aiohttp, so just using the normal
python ssl context and asyncio sockets. I don't think it's an object not
getting GC'd because I didn't see any increase on object counts, nor leaks per
tracemalloc. I
New submission from Alexander Mohr :
I've been trying to track down a leak in aiohttp:
https://github.com/aio-libs/aiohttp/issues/3010
it seems like this leak now occurs with raw asyncio SSL sockets.
when the gist script is run like so: python3.7 `which mprof` run --interval=1
~/dev
Alexander Mohr added the comment:
we're compiling similar to debian and getting the same issues now only with 3.7
so interested in what the resolution is. I tried both w/ and w/o setting
SOURCE_DATE_EPOCH and still get failures.
--
nosy: +thehesiod
Alexander Mohr <thehes...@gmail.com> added the comment:
this is how my friends talk here, see:
https://english.stackexchange.com/questions/11816/is-guy-gender-neutral
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python
Alexander Mohr <thehes...@gmail.com> added the comment:
of course, sorry, I meant in a gender neutral way
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python
Alexander Mohr <thehes...@gmail.com> added the comment:
ok fair enough, however a caveat is that it's not necessarily end of function
as I was able to expand the function by a few more lines and the callstack
stayed the same, however I'm guessing the interpreter was smart enough to
r
Alexander Mohr <thehes...@gmail.com> added the comment:
actually, thinking about this more on my way to work, this should NOT be
closed. The callstack I initially mentioned still has no explanation and we
now know is not correct. It should either have listed something related to
wa
Alexander Mohr <thehes...@gmail.com> added the comment:
Ok I've verified that the patch does indeed fix the leak detected. Thank you
very much INADA for knowing that there was a leak in the warnings module, I
would have never guessed, especially given the tracemalloc stack given
Alexander Mohr <thehes...@gmail.com> added the comment:
not fixing this means that 3.6 slowly leaks for many people in prod. It's not
often possible to fix all the warnings on large dynamic applications, I highly
suggest finding a way to get this into 3.6. I bet there are
Alexander Mohr <thehes...@gmail.com> added the comment:
I'll try with that fix and see if the leak is fixed, is the reasoning that if
the warning happens after the try/except scope and that's why the callstack has
it?
--
___
Python tracke
Alexander Mohr <thehes...@gmail.com> added the comment:
INADA Naoki: Unfortunately you'll need to use credentials from a free AWS
account: https://aws.amazon.com/free/. Then create a credentials file in
~/.aws/credentials:
https://docs.aws.amazon.com/cli/latest/userguide/cli-
Alexander Mohr <thehes...@gmail.com> added the comment:
yes, memory does go up. If you click the botocore bug link you'll see a graph
of memory usage over time.
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python
Alexander Mohr <thehes...@gmail.com> added the comment:
that's not going to affect
http://pytracemalloc.readthedocs.io/api.html#get_traced_memory. There is no
filter for that :)
as to your sum that's exactly what my original callstack lists:
21 memory blocks: 4.7 KiB
this means 21
Alexander Mohr <thehes...@gmail.com> added the comment:
I believe your method is flawed, when enabling tracemalloc it's going to be
using memory as well to store the traces. I still believe you need to use the
method I mentioned and further even if we don't take into account the total
Alexander Mohr <thehes...@gmail.com> added the comment:
here's another problem, if I change that function signature to:
def test(s3_client):
try:
method = getattr(s3_client, sys.argv[1])
method(Bucket='archpi.dabase.com', Key='doesnotexist')
Alexander Mohr <thehes...@gmail.com> added the comment:
actually it does show in 3.5.2, it doesn't show when using get_object, but it
does when using head_object, and they both throw the same exception, so weird.
--
___
Python tracke
Alexander Mohr <thehes...@gmail.com> added the comment:
so based on my last comment I just realized we could easily see if something
was amiss by comparing results from python 3.5.2 to 3.6.5 and low and behold
the callstack in question does not appear in the tracemalloc results from
Alexander Mohr <thehes...@gmail.com> added the comment:
I realize it doesn't track leaks, it's a tool to help find leaks when used
correctly :) This example should be similar to using the compare snapshots
mechanism as I start tracking from a stable point (several iterations in after
Alexander Mohr <thehes...@gmail.com> added the comment:
here's a version that tries to do something similar but does not reproduce the
issue
--
Added file: https://bugs.python.org/file47602/tracemalloc_test2.py
___
Python tracke
Change by Alexander Mohr <thehes...@gmail.com>:
--
type: -> behavior
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33565>
___
__
New submission from Alexander Mohr <thehes...@gmail.com>:
while investigating https://github.com/boto/botocore/issues/1464 I used
tracemalloc (like I've done before in 3.5.2) to try to figure out where the
leak was. To my surprise tracemalloc listed stacks that didn't make any
Alexander Mohr <thehes...@gmail.com> added the comment:
closing as I'm not quite sure this is right
--
stage: -> resolved
status: open -> closed
___
Python tracker <rep...@bugs.python.org>
<https://bugs.
New submission from Alexander Mohr <thehes...@gmail.com>:
I'm seeing a lot of leaks via valgrind against the hashlib module. It appears
that it's calling OpenSSL_add_all_digests(); on init, and never calling the
corresponding EVP_Cleanup on free:
https://www.openssl.org/docs/man1.1.0/
Alexander Mohr <thehes...@gmail.com> added the comment:
I was about to open a new bug, but I think my idea overlaps with this one.
From what I understand there are two ways to cancel tasks:
1) calling task.cancel()
2) explicitly raising a CancelledError
with #2 you can get a tra
Alexander Mohr <thehes...@gmail.com> added the comment:
another idea would be neat to somehow plug into the stats so people can trace
(ex: via datadog tracing) "slow" tasks, similarly to things that are outputted
via PYTHONASYNCIODEBUG, however in this case be able to
Alexander Mohr <thehes...@gmail.com> added the comment:
here's how to repro, download fresh debian:stretch image, then install reqs for
python:
apt-get update && apt-get install curl build-essential libssl-dev libffi-dev
libmemcached-dev zlib1g-dev
install pyenv-installer:
Alexander Mohr <thehes...@gmail.com> added the comment:
sorry if my report is confusing, the issue is when run in a debian:stretch
docker container on an OSX host, so running this: docker run --rm -ti
docker:stretch on osx. So if you have access to an OSX machine and have docker
r
Alexander Mohr <thehes...@gmail.com> added the comment:
btw there are some other tests that fail too after removing that test like:
test test_tokenize failed -- Traceback (most recent call last):
File "/build/Python-3.6.3/Lib/test/test_tokenize.py", line 1557, in
t
New submission from Alexander Mohr <thehes...@gmail.com>:
This test fails when run in a debian docker container from a OSX host with the
following error:
test test_os failed -- Traceback (most recent call last):
File "/build/Python-3.6.3/Lib/test/test_os.py", line 3273, in
New submission from Alexander Mohr <thehes...@gmail.com>:
currently gather/wait allow you to return on the first exception and leave the
children executing.
A very common use case that I have is of launching multiple tasks, and if any
of them fail, then all should fail..otherwise the
Change by Alexander Mohr <thehes...@gmail.com>:
--
nosy: +thehesiod
versions: +Python 3.5
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python
Alexander Mohr <thehes...@gmail.com> added the comment:
my understanding is that the PR in https://bugs.python.org/issue30698 fixes
this issue no? If so can we get it merged?
--
___
Python tracker <rep...@bugs.python.org>
<https://
Alexander Mohr <thehes...@gmail.com> added the comment:
@grzgrzgrz3, does this resolve the issue in https://bugs.python.org/issue29406
? I'm guessing you based this PR on that issue. If so I'd like it merged ASAP
as otherwise our prod services will be incompatible with all future
Alexander Mohr <thehes...@gmail.com> added the comment:
oh for second example I meant something like this:
>>> class Foo: pass
>>> Foo.organizer = None
>>> Foo.blah = Foo
>>> Foo.blah.organizer = None
>>> Foo.blah.org
New submission from Alexander Mohr <thehes...@gmail.com>:
Currently subscriting a attribute that's None reports the following:
>>> class Foo: pass
>>> Foo.organizer = None
>>> Foo.organizer['start']
Traceback (most recent call last):
File ""
Alexander Mohr added the comment:
looks like these were meant to be internal methods, retracting new issues
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/iss
Alexander Mohr added the comment:
from 3.6:
>>> AddrlistClass('John Smith <john.smith(comment)@example.org>').getcomment()
''
>>> AddrlistClass('John Smith <john.smith(comment)@example.org>').getdomain()
Changes by Alexander Mohr <thehes...@gmail.com>:
--
nosy: +thehesiod
versions: +Python 3.6, Python 3.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.o
Alexander Mohr added the comment:
let me know if I need to do anything
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29302>
___
__
Alexander Mohr added the comment:
my vote is yes due to the defaultdict issue. We were hitting this in our prod
env
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
hmm, may be my fault due to docker image tagging issue. Will redeploy and
update if the issue persists. If I don't reply again sorry for the noise.
--
___
Python tracker <rep...@bugs.python.org>
Alexander Mohr added the comment:
bad news, I just got a crash in the same place (updating defaultdict) after
running for a week with the fixes from this and inada naoki's patches. I think
the threadpool may be leaking threads too as I had > 40 threads after running
for a week when I
Alexander Mohr added the comment:
I've verified that this along with the changes in 31095 resolve the crashes
I've been seeing in our production environment
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
omg I just realized I need the default dict one too, great investigation work!
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
actually another idea: could the PR for this also update
https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_dealloc to
mention about these macros and when they should be used? That, along with all
the other locations correctly calling
Alexander Mohr added the comment:
I suggest any places that don't need the calls should have comments so that
future reviewers know why.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
should the base method which calls tp_dealloc do this? Maybe can kill all
birds with one stone.
--
nosy: +thehesiod
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
ok, created: https://github.com/python/cpython/pull/2966
there are some other deallocs in there, mind verifying the rest?
--
pull_requests: +3014
___
Python tracker <rep...@bugs.python.org>
<http://bugs.p
Alexander Mohr added the comment:
I'm hoping this is the fix:
--- Modules/_asynciomodule.c.orig 2017-07-31 12:16:16.0 -0700
+++ Modules/_asynciomodule.c2017-07-31 13:08:52.0 -0700
@@ -953,15 +953,18 @@
FutureObj_dealloc(PyObject *self)
{
FutureObj *fut
Alexander Mohr added the comment:
another core had a different gc object:
$1 = {ob_base = {ob_base = {_ob_next = 0x7f801eac3158, _ob_prev =
0x7f801eab95a0, ob_refcnt = 41, ob_type = 0x7f80238e76e0 },
ob_size = 0}, tp_name = 0x7f801e8967af "_asyncio.Task", tp_basicsize = 128,
tp_it
Alexander Mohr added the comment:
so I just discovered that the object that has the zero refcount has the same
tp_dealloc:
(gdb) print *FROM_GC(gc)->ob_type
$8 = {ob_base = {ob_base = {_ob_next = 0x7f80c8aafc88, _ob_prev =
0x7f80c8aafd00, ob_refcnt = 7, ob_type = 0x7f80cd8c86e0 }, ob_s
Alexander Mohr added the comment:
oh, so this is looking like an asyncio issue, the "gc" that is causing the
crash is:
(gdb) print *FROM_GC(gc)->ob_type
$8 = {ob_base = {ob_base = {_ob_next = 0x7f80c8aafc88, _ob_prev =
0x7f80c8aafd00, ob_refcnt = 7, ob_type = 0x7f80cd8c86e0 },
Alexander Mohr added the comment:
I'm tracking something very similar issue to this in bug:
http://bugs.python.org/issue31061 Given its similarities, anyone have any
ideas? Based on the second callstack I'm starting to think this is an issue
with defaultdict
--
nosy: +thehesiod
Alexander Mohr added the comment:
btw got slightly difference stacktrace on second core file
--
Added file: http://bugs.python.org/file47051/python crash2.txt
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
hmm, how would I do that? btw I'm not 100% sure this is due to asyncio.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
this is the comment on the assert:
/* Python's cyclic gc should never see an incoming refcount
* of 0: if something decref'ed to 0, it should have been
* deallocated immediately at that time.
* Possible cause (if the assert triggers): a tp_dealloc
Alexander Mohr added the comment:
ok got a full debug core file, let me know what other information I can provide.
--
status: closed -> open
Added file: http://bugs.python.org/file47049/python crash.txt
___
Python tracker <rep...@bugs.pyth
Alexander Mohr added the comment:
the problem with this crash is that it only happens periodically in our prod
environment :( If I try running the exact same docker container with the same
inputs locally it doesn't reproduce, so frustrating. I've created a whole
workflow now for deploying
Alexander Mohr added the comment:
so looks like disabling the _asyncio module just caused the crash to happen
less often, closing and will continue investigating after a get a core file
--
stage: -> resolved
status: open -> closed
___
Alexander Mohr added the comment:
btw I've seen this issue in 3.5.2 + 3.6.2 on debian jessie + stretch
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
New submission from Alexander Mohr:
I have a project in a prod environment which heavily uses asyncio and a
threadpool. It uses the threadpool to run CPU heavy tasks (in this case
populating a defaultdict) to avoid blocking the main thread (no async code in
thread). For some time now my
Changes by Alexander Mohr <thehes...@gmail.com>:
--
nosy: +thehesiod
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30698>
___
_
Alexander Mohr added the comment:
awesome! Thanks for finding a proposing fix pitrou! btw I found an example of
freeing this structure here:
http://www.zedwood.com/article/c-openssl-parse-x509-certificate-pem
--
___
Python tracker <
Alexander Mohr added the comment:
see graphs here: https://github.com/kennethreitz/requests/issues/3933, x-axis
is number of requests not what it says (seconds).
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
ya, my sample script hits google.com <http://google.com/>, it's pretty fast.
It just does a "HEAD".
> On Apr 11, 2017, at 9:14 AM, Antoine Pitrou <rep...@bugs.python.org> wrote:
>
>
> Antoine Pitrou added the comment:
Alexander Mohr added the comment:
the interesting part is it doesn't leak with a local https server, it appears
to need to be a remove server.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
yes, in the gist I created you can switch between the various clients, by
default right now it uses raw sockets.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
@pitrou: sys.getallocatedblocks does not seem to increase
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
ok I've updated the gist with a base class and sync + async sub-classes. The
way it worked out I think is nice because we can have the same method names
across both sync+async. Let me know what you guys think! btw, it seems
Alexander Mohr added the comment:
interestingly the valgrind run doesn't show a leak in the profile
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
adding valgrind log of 3.5.3 on debian: jessie
--
Added file: http://bugs.python.org/file46750/valgrind.log.gz
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alexander Mohr added the comment:
validated 3.6 in fedora is affected as well, see github bug for charts. So it
seems all 3.5.3+ versions are affected. I'm guessing it was introduced in one
of the SSL changes in 3.5.3:
https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-3
New submission from Alexander Mohr:
When upgrading to 3.5.3 we noticed that the requests module was leaking memory
rather quickly. This led to me logging the issue:
https://github.com/kennethreitz/requests/issues/3933. After more investigation
I've found that the leak is caused by the raw
Alexander Mohr added the comment:
Thanks for the feedback Nick! If I get a chance I'll see about refactoring my
gist into a base class and two sub-classes with the async supporting non-async
but not vice-versa. I think it will be cleaner. Sorry I didn't spend too much
effort on the existing
Alexander Mohr added the comment:
I believe this is now worse due to https://github.com/python/asyncio/pull/452
before I was able to simply create a new run loop from sub-processes however
you will now get the error "Cannot run the event loop while another loop is
running".
Alexander Mohr added the comment:
Thanks so much for the patch!
may want to change spelling of what was supposed to be "shutdown" =) Also
think it's worth a comment stating why it's needed? Like certain Apache
servers were noticed to not complete the SSL shutdo
Alexander Mohr added the comment:
updating to make default the error case (madis)
--
Added file: http://bugs.python.org/file46470/scratch_1.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Changes by Alexander Mohr <thehes...@gmail.com>:
Removed file: http://bugs.python.org/file46469/scratch_1.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
New submission from Alexander Mohr:
with the attached code note how HttpClient.connection_lost callback is never
called for the madis server. The madis server is an apache server, I tried with
the OSX apache server and could not reproduce the issue so it seems something
particular about
Alexander Mohr added the comment:
created gist: https://gist.github.com/thehesiod/b8442ed50e27a23524435a22f10c04a0
I've now updated the imple to support both __aenter__/__aexit__ and
__enter__/__exit__ so I don't need two ExitStacks
--
___
Python
Changes by Alexander Mohr <thehes...@gmail.com>:
Removed file: http://bugs.python.org/file46322/exit_stack.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
New submission from Alexander Mohr:
ExitStack is a really useful class and would be a great to have an async
version. I've gone ahead and created an implementation based on the existing
Python 3.5.2 implementation. Let me know what you guys think. I think it would
be possible to combine
Alexander Mohr added the comment:
I'm sure it would work, I just wanted a solution that didn't changes to our
build infrastructure. btw if we're marking this as a duplicate of the other
bug, can we update the other bug to say it affects python3.x as well? Thanks!
--
status: pending
Alexander Mohr added the comment:
ya I did a monkey patch which resolved it.
if sys.platform == 'darwin':
import botocore.vendored.requests.utils, urllib.request
botocore.vendored.requests.utils.proxy_bypass =
urllib.request.proxy_bypass_environment
Alexander Mohr added the comment:
interestingly I haven't been able to get this to crash in a separate test app.
There must be either timing related to some interaction with another module.
let me know how you guys would like to proceed. I can definitely reproduce it
consistently in our
1 - 100 of 121 matches
Mail list logo
