Dan Gindikin gmail.com> writes:
>
> Antoine Pitrou pitrou.net> writes:
> > Does cPickle bytecode have some kind of NOP instruction?
> > You could keep track of which PUTs weren't necessary and zero them out at
> > the
> > end. It would be much cheaper than writing a whole other "optimized" stre
Antoine Pitrou pitrou.net> writes:
> Does cPickle bytecode have some kind of NOP instruction?
> You could keep track of which PUTs weren't necessary and zero them out at the
> end. It would be much cheaper than writing a whole other "optimized" stream.
For a large file, I'm not sure it is much fa
Collin Winter google.com> writes:
> I don't think it's possible in general to remove any PUTs if the
> pickle is being written to a file-like object. It is possible to reuse
> a single Pickler to pickle multiple objects: this causes the Pickler's
> memo dict to be shared between the objects being
Collin Winter google.com> writes:
>
> I don't think it's possible in general to remove any PUTs if the
> pickle is being written to a file-like object.
Does cPickle bytecode have some kind of NOP instruction?
You could keep track of which PUTs weren't necessary and zero them out at the
end. It w
On Fri, Apr 23, 2010 at 1:53 PM, Alexandre Vassalotti
wrote:
> On Fri, Apr 23, 2010 at 3:57 PM, Dan Gindikin wrote:
>> This wouldn't help our use case, your code needs the entire pickle
>> stream to be in memory, which in our case would be about 475mb, this
>> is on top of the 300mb+ data structu
Alexandre Vassalotti peadrop.com> writes:
>
> On Fri, Apr 23, 2010 at 3:57 PM, Dan Gindikin gmail.com>
> wrote:
> > This wouldn't help our use case, your code needs the entire pickle
> > stream to be in memory, which in our case would be about 475mb, this
> > is on top of the 300mb+ data struc
On Fri, Apr 23, 2010 at 3:57 PM, Dan Gindikin wrote:
> This wouldn't help our use case, your code needs the entire pickle
> stream to be in memory, which in our case would be about 475mb, this
> is on top of the 300mb+ data structures that generated the pickle
> stream.
>
In that case, the best w
Collin Winter google.com> writes:
> I should add that, adding the necessary bookkeeping to remove only
> unused PUTs (instead of the current all-or-nothing scheme) should not
> be hard. I'd watch out for a further performance/memory hit; the
> pickling benchmarks in the benchmark suite should help
Nick Coghlan writes:
> Joachim B Haga wrote:
>> There seem to be an inconsistency in the handling of local scopes in
>> exec. [...]
>>
>> The intermediate scope is searched for the variable name if the third
>> argument to exec() is locals(), but not if it is locals().copy().
>
> What actually m
On Fri, Apr 23, 2010 at 3:07 PM, Collin Winter wrote:
> I should add that, adding the necessary bookkeeping to remove only
> unused PUTs (instead of the current all-or-nothing scheme) should not
> be hard. I'd watch out for a further performance/memory hit; the
> pickling benchmarks in the benchma
On Fri, Apr 23, 2010 at 11:53 AM, Collin Winter wrote:
> On Fri, Apr 23, 2010 at 11:49 AM, Alexandre Vassalotti
> wrote:
>> On Fri, Apr 23, 2010 at 2:38 PM, Alexandre Vassalotti
>> wrote:
>>> Collin Winter wrote a simple optimization pass for cPickle in Unladen
>>> Swallow [1]. The code reads th
On Fri, Apr 23, 2010 at 11:49 AM, Alexandre Vassalotti
wrote:
> On Fri, Apr 23, 2010 at 2:38 PM, Alexandre Vassalotti
> wrote:
>> Collin Winter wrote a simple optimization pass for cPickle in Unladen
>> Swallow [1]. The code reads through the stream and remove all the
>> unnecessary PUTs in-place
Alexandre Vassalotti peadrop.com> writes:
> Just put your code on bugs.python.org and I will take a look.
>
Thanks, I'll put it in there.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscr
On Fri, Apr 23, 2010 at 2:38 PM, Alexandre Vassalotti
wrote:
> Collin Winter wrote a simple optimization pass for cPickle in Unladen
> Swallow [1]. The code reads through the stream and remove all the
> unnecessary PUTs in-place.
>
I just noticed the code removes *all* PUT opcodes, regardless if
On Fri, Apr 23, 2010 at 2:11 PM, Dan Gindikin wrote:
> We were having performance problems unpickling a large pickle file, we were
> getting 170s running time (which was fine), but 1100mb memory usage. Memory
> usage ought to have been about 300mb, this was happening because of memory
> fragmentat
On Fri, Apr 23, 2010 at 11:11, Dan Gindikin wrote:
> We were having performance problems unpickling a large pickle file, we were
> getting 170s running time (which was fine), but 1100mb memory usage. Memory
> usage ought to have been about 300mb, this was happening because of memory
> fragmentati
On Fri, Apr 23, 2010 at 09:21, Barry Warsaw wrote:
> On Apr 23, 2010, at 05:44 PM, Victor Stinner wrote:
>
> >I'm not sure that it's a good idea to change the build process after the
> >first beta. It would depend on the issue comments ;-)
>
> OTOH, this doesn't seem like a new feature, so I thin
We were having performance problems unpickling a large pickle file, we were
getting 170s running time (which was fine), but 1100mb memory usage. Memory
usage ought to have been about 300mb, this was happening because of memory
fragmentation, due to many unnecessary "puts" in the pickle stream.
We
On 2010-04-22, at 10:55 PM, Jeroen Ruigrok van der Werven wrote:
> -On [20100423 02:48], Sridhar Ratnakumar (sridh...@activestate.com) wrote:
>>> Ok, I setup a cron job to maintain an internal mirror of the above
>>> mentioned repositories in code.python.org. W
Joachim B Haga wrote:
> There seem to be an inconsistency in the handling of local scopes in
> exec. Consider the following code, which raises NameError if the '#' is
> removed from the second last line.
>
>
> block = """
> b = 'ok'
> def f():
> print(b)# raises NameError here
> f()
> """
On Apr 23, 2010, at 05:44 PM, Victor Stinner wrote:
>I'm not sure that it's a good idea to change the build process after the
>first beta. It would depend on the issue comments ;-)
OTOH, this doesn't seem like a new feature, so I think it should be okay.
Doubly so if it fixes a bug.
-Barry
si
ACTIVITY SUMMARY (2010-04-16 - 2010-04-23)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue
number. Do NOT respond to this message.
2665 open (+58) / 17664 closed (+31) / 20329 total (+89)
Open issues with patches: 1084
Ave
Le jeudi 22 avril 2010 22:14:48, Sridhar Ratnakumar a écrit :
> I am seeing random 403 errors when cloning the mercurial repositories of
> Python.
I don't know if it is related, but I get errors from the bbreport tool:
--
$ python2.6 bbreport.py 3.x
Selected builders: 20 / 80 (branch:
Le vendredi 23 avril 2010 17:26:59, Matthias Klose a écrit :
> configure is still generated by 2.61; would it be possible to update to
> 2.65?
Yes, everything is possible. Open a new issue and write a patch ;-)
> even if 2.7 already is in beta?
I'm not sure that it's a good idea to change the b
configure is still generated by 2.61; would it be possible to update to 2.65?
The cr_lf issue mentioned in [1] seems to be resolved, ac_cr is now defined as
ac_cr=`echo X | tr X '\015'`
Proposing to
- fix some quoting in help strings and code snippets (#8509)
- update to autoconf 2.65 (#85
There seem to be an inconsistency in the handling of local scopes in
exec. Consider the following code, which raises NameError if the '#' is
removed from the second last line.
block = """
b = 'ok'
def f():
print(b)# raises NameError here
f()
"""
scope = locals()#.copy()
exec(block, global
26 matches
Mail list logo
