> On Apr 11, 2020, at 06:52, Paul Sokolovsky wrote:
>
>> And a StringBuilder class would be another way.
>
> StringBuilder would be just subset of functionality of StringIO, and
> would be hard to survive Occam's razor. (Mine is sharp to cut it off
> right away.)
I think _this_ is actually the
Hello,
On Mon, 6 Apr 2020 09:41:46 -0700
Christopher Barker wrote:
> Sorry, this has been sitting in my drafts for a while, and maybe this
> conversation is over. But since I wrote it, I figured I might as well
> post it.
I appreciate it, and it's actually my desired intention to not leave
Hello,
On Tue, 31 Mar 2020 17:01:19 -0700
Christopher Barker wrote:
[]
> However if we're all wrong, and there would be a demand for such a
> "string builder", then why not write one (could be a wrapper around
> StringIO if you want), and put it on PyPi, or even just for own lib,
> and see how
On Mon, Apr 6, 2020 at 9:49 AM Chris Angelico wrote:
> > This all made me think *why* do I type check strings, and virtually
> nothing else? And it's because in most otehr places, if I want a given
> type, I can just try to make it from the input:
> >
> > x = float(x)
> > i = int(i)
> >
> > arr
On Tue, Apr 7, 2020 at 2:42 AM Christopher Barker wrote:
> This all made me think *why* do I type check strings, and virtually nothing
> else? And it's because in most otehr places, if I want a given type, I can
> just try to make it from teh input:
>
> x = float(x)
> i = int(i)
>
> arr =
Sorry, this has been sitting in my drafts for a while, and maybe this
conversation is over. But since I wrote it, I figured I might as well post
it.
On Fri, Apr 3, 2020 at 4:24 AM Paul Sokolovsky wrote:
> > the idea of adding += to the File protocol -- for all file-like
> > objects. I like the
Hello,
On Wed, 1 Apr 2020 19:00:00 -0700
Christopher Barker wrote:
[]
> The funny thing is, in this thread, while I dont really see the need
> for adding += to StringIO to make a string builder, I kind of like
> the idea of adding += to the File protocol -- for all file-like
> objects. I like
Christopher Barker writes:
> On Thu, Apr 2, 2020 at 6:07 AM Stephen J. Turnbull <
> turnbull.stephen...@u.tsukuba.ac.jp> wrote:
> > Yah, but you also get
>
> >
> > outfile.seek(some_random_place)
> > outfile += something
> >
>
> Is that not exactly the same as what you can do
Hello,
On Fri, 3 Apr 2020 09:34:44 +1100
Chris Angelico wrote:
> On Fri, Apr 3, 2020 at 9:20 AM Paul Sokolovsky
> wrote:
> > But not exactly. Let me humbly explain what's really a cost. It's
> > looking at PyObject_HEAD
> > https://swenson.github.io/python-xr/Include/object.h.html#line-78
> >
On Fri, Apr 3, 2020 at 9:20 AM Paul Sokolovsky wrote:
> But not exactly. Let me humbly explain what's really a cost. It's
> looking at PyObject_HEAD
> https://swenson.github.io/python-xr/Include/object.h.html#line-78
> (damn, that's Python2 source, stupid google), and seeing that it's at
> least:
Hello,
On Fri, 3 Apr 2020 08:44:23 +1100
Chris Angelico wrote:
> On Fri, Apr 3, 2020 at 8:26 AM Paul Sokolovsky
> wrote:
> >
> > Hello,
> >
> > On Wed, 1 Apr 2020 21:25:46 -0400
> > Kyle Stanley wrote:
> >
> > > Also, on the point of memory usage: I'd very much like to see some
> > > real
On Apr 1, 2020, at 20:59, Steven D'Aprano wrote:
>
> Nevertheless, you do make a good point. It may be that StringIO is not
> the right place for this. That can be debated without dismissing the
> entire idea.
I think it’s worth separating the two.
There is a legitimate desire for a “better”
On 02/04/2020 22:24, Paul Sokolovsky wrote:
Hello,
On Wed, 1 Apr 2020 21:25:46 -0400
Kyle Stanley wrote:
Paul Sokolovsky wrote:
Roughly speaking, the answer would be about the same in idea as
answers to the following questions:
[snip]
I would say the difference between this proposal so far
Hello,
On Wed, 1 Apr 2020 20:05:33 -0700
Andrew Barnert wrote:
> On Apr 1, 2020, at 14:47, Paul Sokolovsky wrote:
> >
> >> At the moment, the
> >> message is relatively clear - "build a list and join it" (it's very
> >> rare that anyone suggests StringIO currently).
> >
> > I don't know
On Fri, Apr 3, 2020 at 8:26 AM Paul Sokolovsky wrote:
>
> Hello,
>
> On Wed, 1 Apr 2020 21:25:46 -0400
> Kyle Stanley wrote:
>
> > Also, on the point of memory usage: I'd very much like to see some
> > real side-by-side comparisons of the ``''.join(parts)`` memory usage
> > across Python
Hello,
On Wed, 1 Apr 2020 21:25:46 -0400
Kyle Stanley wrote:
> Paul Sokolovsky wrote:
> > Roughly speaking, the answer would be about the same in idea as
> > answers to the following questions:
> > [snip]
>
> I would say the difference between this proposal so far and the ones
> listed are
On Thu, Apr 2, 2020 at 6:07 AM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:
> Christopher Barker writes:
>
> > The funny thing is, in this thread, while I dont really see the need for
> > adding += to StringIO to make a string builder, I kind of like the idea
> of
> >
I'm assuming here that the goal is to make string building easier,
better, and/or more discoverable, and that the io.StringIO discussion is
just one way to achieve this. For example, I don't think (but maybe I'm
wrong) that "must be a file-like object" is a goal here.
If that's not the goal,
Christopher Barker writes:
> The funny thing is, in this thread, while I dont really see the need for
> adding += to StringIO to make a string builder, I kind of like the idea of
> adding += to the File protocol -- for all file-like objects. I like the
> compactness of:
>
> with
(Folks, sorry for letting this spam slip through! It was reasonably clever
compared to most of the stuff we get :)
best,
—titus
> On Apr 2, 2020, at 12:49 AM, gstindianews.i...@gmail.com wrote:
>
> ya that's what I also get for quickly whipping something up and not
> testing it. Good catch
ya that's what I also get for quickly whipping something up and not
testing it. Good catch from the end. I found another similar at
https://gstindianews.info. But you get the idea - a simple wrapper around
a list is going to be way better than a wrapper around StringIO. i
jayesh
Here's a very readable way to concatenate strings using "+="
on lists:
>>> l = []
>>> l += ['abc']
>>> l += ['abc']
>>> l += ['abc']
>>> l += ['abc']
>>> s = ''.join(l)
>>> s
'abcabcabcabc'
Probably not exactly what Paul had in mind, but it's beginners
friendly.
The string version, with O(N²)
On Thu, Apr 2, 2020 at 5:58 PM Steven D'Aprano wrote:
>
> On Thu, Apr 02, 2020 at 03:37:34PM +1100, Chris Angelico wrote:
> > On Thu, Apr 2, 2020 at 3:27 PM Steven D'Aprano wrote:
> > > Or maybe that's just an argument that no solution is going to solve
> > > *every* problem. What do we do about
On Thu, Apr 02, 2020 at 03:37:34PM +1100, Chris Angelico wrote:
> On Thu, Apr 2, 2020 at 3:27 PM Steven D'Aprano wrote:
> > Or maybe that's just an argument that no solution is going to solve
> > *every* problem. What do we do about people who write this:
> >
> > buf = f'{buf}{substring}'
> >
On Thu, 2 Apr 2020 at 04:59, Steven D'Aprano wrote:
>
> On Wed, Apr 01, 2020 at 09:25:46PM -0400, Kyle Stanley wrote:
>
> > While I agree that it's sometimes okay to go outside the strict bounds of
> > "only one way to do it"
>
> The Zen of Python was invented as a joke, not holy writ, and as a
On Thu, Apr 2, 2020 at 3:27 PM Steven D'Aprano wrote:
> Or maybe that's just an argument that no solution is going to solve
> *every* problem. What do we do about people who write this:
>
> buf = f'{buf}{substring}'
>
> inside a loop? We can't fix everyone's code with one change.
>
I don't
On Tue, Mar 31, 2020 at 12:39:56AM -0700, Christopher Barker wrote:
> But while the OP specifically suggested adding += to stringIO, you might
> note that he did not suggest it as an extension to the file protocol, which
> StringIO mimics. Rather, he suggested that there should be something that
On Wed, Apr 01, 2020 at 07:00:00PM -0700, Christopher Barker wrote:
> > But with all that, I don't see why such a "mutable string" would be
> > more suitable for "string builder" pattern.
>
> It would in one small way, which is that it would be usable directly in
> many (not all by any means)
On Wed, Apr 01, 2020 at 09:25:46PM -0400, Kyle Stanley wrote:
> While I agree that it's sometimes okay to go outside the strict bounds of
> "only one way to do it"
The Zen of Python was invented as a joke, not holy writ, and as a series
of koans intended to guide thought, not shut it down.
On Apr 1, 2020, at 14:47, Paul Sokolovsky wrote:
>
>> At the moment, the
>> message is relatively clear - "build a list and join it" (it's very
>> rare that anyone suggests StringIO currently).
>
> I don't know how much you mix with other Pythonistas, but word "clear"
> is an exaggeration.
On Wed, Apr 1, 2020 at 1:31 PM Paul Sokolovsky wrote:
>
> > In short: a mutable string would satisfy the requirements of a "string
> > builder", and more.
>
> Thanks for the detailed explanation. For me, a canonical example of a
> feature of "mutable string" would be:
>
> s = MutStr("foo")
>
On Thu, Apr 2, 2020 at 12:27 PM Kyle Stanley wrote:
> As Chris Angelico mentioned, this can be observed through monitoring the
> before and after RSS (or equivalent on platforms without it). On Linux, I
> typically use something like this:
>
> ```
> def show_rss():
> os.system(f"grep ^VmRSS
Paul Sokolovsky wrote:
> Roughly speaking, the answer would be about the same in idea as answers
> to the following questions:
> [snip]
I would say the difference between this proposal so far and the ones listed
are that they emphasized concrete, real-world examples from existing code
either in
Hello,
On Wed, 1 Apr 2020 10:01:06 +0100
Paul Moore wrote:
> On Wed, 1 Apr 2020 at 02:07, Steven D'Aprano
> wrote:
> > Paul has not suggested making StringIO look and feel like a string.
> > Nobody is going to add 45+ string methods to StringIO. This is a
> > minimal extension to the StringIO
Hello,
On Tue, 31 Mar 2020 17:01:19 -0700
Christopher Barker wrote:
[]
> So that suggested to me that a mutable string type would completely
> satisfy your use case, but be more natural to folks used to strings:
>
> message = MutableString("The start of the message")
> for i in something:
>
On Wed, 1 Apr 2020 at 02:07, Steven D'Aprano wrote:
> Paul has not suggested making StringIO look and feel like a string.
> Nobody is going to add 45+ string methods to StringIO. This is a minimal
> extension to the StringIO class which will allow people to improve their
> string building code
On Tue, Mar 31, 2020 at 07:32:11PM -, jdve...@gmail.com wrote:
> If I understand you are proposing a change from StringIO `write`
> method to `+=` operator. Is it right?
No, that is not correct. The StringIO.write() method will not be changed
or removed. The proposal is to extend the class
On Tue, Mar 31, 2020 at 12:21 PM Paul Sokolovsky wrote:
> Christopher Barker wrote:
> For avoidance of doubt: nothing in my RFC has anything to do, or
> implies, "a mutable string type".
I said "there are some use cases for a mutable string type" I did not say
that's what was asked for in
Hello,
On Tue, 31 Mar 2020 16:28:22 +1100
Steven D'Aprano wrote:
> On Mon, Mar 30, 2020 at 10:08:06PM -0700, Guido van Rossum wrote:
> > On Mon, Mar 30, 2020 at 10:00 PM Steven D'Aprano
> > wrote:
> >
> > > > it’s optimized for a different use case than string building,
> > >
> > > It is?
On Mar 31, 2020, at 12:06, Paul Sokolovsky wrote:
I don’t know why you think being snarky helps make your case. If you make a
mistake and it’s pointed out and you give a sarcastically over-enthusiastic
thanks, that doesn’t change the fact that it‘s wrong, and if your rationale
depends on
Hello,
On Mon, 30 Mar 2020 23:20:28 -0400
David Mertz wrote:
> I have myself been "guilty" of using the problem style for N < 10. In
> fact, I had forgotten about the optimization even, since my uses are
> negligible time.
I personally don't think it's a "problem style" per se. I'm using it
Paul Sokolovsky wrote:
> Hello,
> On Mon, 30 Mar 2020 16:25:07 -0700
> Christopher Barker python...@gmail.com wrote:
> > As others have pointed out, the OP started in a bit
> > of an oblique
> > way, but it maybe come down to this:
> > There are some use-cases for a mutable string type.
> > For
Hello,
On Mon, 30 Mar 2020 16:25:07 -0700
Christopher Barker wrote:
> As others have pointed out, the OP started in a bit of an oblique
> way, but it maybe come down to this:
>
> There are some use-cases for a mutable string type.
For avoidance of doubt: nothing in my RFC has anything to do,
Hello,
On Mon, 30 Mar 2020 13:59:42 -0700
Andrew Barnert wrote:
> On Mar 30, 2020, at 13:06, Paul Sokolovsky wrote:
> >
> > I appreciate expressing it all concisely and clearly. Then let me
> > respond here instead of the very first '"".join() rules!' reply I
> > got.
>
> Ignoring replies
Steven D'Aprano writes:
> On Mon, Mar 30, 2020 at 10:08:06PM -0700, Guido van Rossum wrote:
> > StringIO was created in order to fit code designed to a file,
> > where all you want to do is capture its output and process it
> > further, in the same process.
> But it does that by *building a
Steven D'Aprano writes:
> [I]t ought to be clear from Paul's well-written and detailed post,
> which carefully explains what he wants.
Whose value to Python I still don't understand, because AFAICS it's
something that on the one hand violates TOOWTDI and has no parallels
elsewhere in the io
On Mar 30, 2020, at 22:03, Steven D'Aprano wrote:
>
> On Mon, Mar 30, 2020 at 01:59:42PM -0700, Andrew Barnert via Python-ideas
> wrote:
>
> [...]
>> When you call getvalue() it then builds a Py_UCS4*
>> representation that’s in this case 4x the size of the final string
>> (since your string
On Mon, Mar 30, 2020 at 10:08:06PM -0700, Guido van Rossum wrote:
> On Mon, Mar 30, 2020 at 10:00 PM Steven D'Aprano
> wrote:
>
> > > it’s optimized for a different use case than string building,
> >
> > It is? That's odd. The whole purpose of StringIO is to build strings.
I misspoke: it is not
On Tue, Mar 31, 2020 at 03:01:51PM +1100, Steven D'Aprano wrote:
> > nor the fastest way
>
> It's pretty close though.
>
> On my test, accumulating 500,000 strings into a list versus a StringIO
> buffer, then building a string, took 27.5 versus 31.6 ms. Using a string
> took 36.4 ms. So it's
On Mon, Mar 30, 2020 at 10:00 PM Steven D'Aprano
wrote:
> > it’s optimized for a different use case than string building,
>
> It is? That's odd. The whole purpose of StringIO is to build strings.
>
> What use-case do you believe it is optimized for?
>
Let me tell you, since I was there.
On Mon, Mar 30, 2020 at 01:59:42PM -0700, Andrew Barnert via Python-ideas wrote:
[...]
> When you call getvalue() it then builds a Py_UCS4*
> representation that’s in this case 4x the size of the final string
> (since your string is pure ASCII and will be stored in UCS1, not
> UCS4). And then
On Mon, Mar 30, 2020 at 04:25:07PM -0700, Christopher Barker wrote:
> As others have pointed out, the OP started in a bit of an oblique way, but
> it maybe come down to this:
>
> There are some use-cases for a mutable string type. And one could certainly
> write one.
With respect Christopher,
On Mon, Mar 30, 2020 at 10:07:30AM -0700, Andrew Barnert via Python-ideas wrote:
> Why? What’s the benefit of building a mutable string around a virtual
> file object wrapped around a buffer (with all the extra complexities
> and performance costs that involves, like incremental Unicode
I have myself been "guilty" of using the problem style for N < 10. In
fact, I had forgotten about the optimization even, since my uses are
negligible time.
For stuff like this, it's fast no matter what:
for clause in query_clauses:
sql += clause
Maybe I have a WHERE or two. Maybe an ORDER
Does anyone know if any linters find and warn about the `string += word` in
a loop pattern? It feels like a linter would be the place to do that. I
don't think we could possibly make it an actual interpreter warning given
borderline OK uses (or possibly even preferred ones). But a little nagging
On Mon, Mar 30, 2020 at 12:37:48PM -0700, Andrew Barnert via Python-ideas wrote:
> On Mar 30, 2020, at 12:00, Paul Sokolovsky wrote:
> > Roughly speaking, to support efficient appending, one need to
> > be ready to over-allocate string storage, and maintain bookkeeping for
> > this. Another known
On Mon, Mar 30, 2020 at 10:24:02AM -0700, Andrew Barnert via Python-ideas wrote:
> On Mar 30, 2020, at 10:01, Brett Cannon wrote:
[talking about string concatenation]
> > I don't think characterizing this as a "mis-optimization" is fair.
[...]
> Yes. A big part of the reason there’s so much
On Tue, Mar 31, 2020 at 10:25 AM Christopher Barker wrote:
>
> As others have pointed out, the OP started in a bit of an oblique way, but
> it maybe come down to this:
>
> There are some use-cases for a mutable string type. And one could certainly
> write one.
>
> presto: here is one:
>
>
As others have pointed out, the OP started in a bit of an oblique way, but
it maybe come down to this:
There are some use-cases for a mutable string type. And one could certainly
write one.
presto: here is one:
https://github.com/Daniil-Kost/mutable_strings
Which looks to me to be more a toy
Hello,
On Tue, 31 Mar 2020 07:40:01 +1100
Chris Angelico wrote:
> On Tue, Mar 31, 2020 at 7:04 AM Paul Sokolovsky
> wrote:
> > for i in range(5):
> > v = u"==%d==" % i
> > # All individual strings will be kept in the list and
> > # can't be GCed before teh final
On Mar 30, 2020, at 13:06, Paul Sokolovsky wrote:
>
> I appreciate expressing it all concisely and clearly. Then let me
> respond here instead of the very first '"".join() rules!' reply I got.
Ignoring replies doesn’t actually answer them.
> The issue with "".join() is very obvious:
>
>
On Tue, Mar 31, 2020 at 7:04 AM Paul Sokolovsky wrote:
> for i in range(5):
> v = u"==%d==" % i
> # All individual strings will be kept in the list and
> # can't be GCed before teh final join.
> sz += sys.getsizeof(v)
> sb.append(v)
> s =
Hello,
On Mon, 30 Mar 2020 12:37:48 -0700
Andrew Barnert wrote:
> On Mar 30, 2020, at 12:00, Paul Sokolovsky wrote:
> > Roughly speaking, to support efficient appending, one need to
> > be ready to over-allocate string storage, and maintain bookkeeping
> > for this. Another known optimization
Hello,
On Tue, 31 Mar 2020 04:27:04 +1100
Chris Angelico wrote:
[]
> There's a vast difference between "mutable string" and "string
> builder". The OP was talking about this kind of thing:
>
> buf = ""
> for i in range(5):
> buf += "foo"
> print(buf)
>
> And then suggested using a
On Mar 30, 2020, at 12:00, Paul Sokolovsky wrote:
> Roughly speaking, to support efficient appending, one need to
> be ready to over-allocate string storage, and maintain bookkeeping for
> this. Another known optimization CPython does is for stuff like "s =
> s[off:]", which requires maintaining
On Mar 30, 2020, at 10:18, Joao S. O. Bueno wrote:
>
> That said, anyone could tell about small, efficient,
> well maintained "mutable string" classes on Pypi?
I don’t know of one. But what do you actually want it for? In most cases where
you want “mutable strings”, what you really want is
Hello,
On Mon, 30 Mar 2020 09:58:32 -0700
Brett Cannon wrote:
> On Sun, Mar 29, 2020 at 10:58 AM Paul Sokolovsky
> wrote:
>
> > [SNIP]
> >
> > 1. Succumb to applying the same mis-optimization for string type as
> > CPython3. (With the understanding that for speed-optimized projects,
> >
On Tue, Mar 31, 2020 at 5:10 AM Serhiy Storchaka wrote:
>
> 30.03.20 20:27, Chris Angelico пише:
> > def __iadd__(self, s): self.data.append(s)
>
> __iadd__ should return self.
>
And that's what I get for quickly whipping something up and not
testing it. Good catch. But you get the idea - a
30.03.20 20:27, Chris Angelico пише:
def __iadd__(self, s): self.data.append(s)
__iadd__ should return self.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
30.03.20 20:07, Andrew Barnert via Python-ideas пише:
Sadly, this isn’t possible. Large amounts of C code—including builtins and stdlib—won’t let you duck type as a string; as it will do a type check and expect an actual str (and if you subclass str, it will ignore your methods and use the
On Tue, Mar 31, 2020 at 4:20 AM Joao S. O. Bueno wrote:
>
> Hi Andrew -
>
> I made my previous post before reading your first answer.
>
> So, anyway, what we have is that for a "mutable string like object" one
> is free to build his wrapper - StringIO based or not - put it on pypi, and
>
On Mar 30, 2020, at 10:01, Brett Cannon wrote:
>
> I don't think characterizing this as a "mis-optimization" is fair. There is
> use of in-place add with strings in the wild and CPython happens to be able
> to optimize for it. Someone was motivated to do the optimization so we took
> it
Hi Andrew -
I made my previous post before reading your first answer.
So, anyway, what we have is that for a "mutable string like object" one
is free to build his wrapper - StringIO based or not - put it on pypi, and
remember
calling `str()` on it before having it leave your code.
Thank you
On Mar 30, 2020, at 08:29, Joao S. O. Bueno wrote:
>
>
> I agree with the arguments the OP brings forward.
>
> Maybe, it should be the case of having an `StringIO` and `BytesIO` subclass?
> Or better yet, just a class that wraps those, and hide away the other
> file-like
> methods and
On Sun, Mar 29, 2020 at 10:58 AM Paul Sokolovsky wrote:
> [SNIP]
>
> 1. Succumb to applying the same mis-optimization for string type as
> CPython3. (With the understanding that for speed-optimized projects,
> implementing mis-optimizations will eat into performance budget, and
> for
I agree with the arguments the OP brings forward.
Maybe, it should be the case of having an `StringIO` and `BytesIO` subclass?
Or better yet, just a class that wraps those, and hide away the other
file-like
methods and behaviors?
That would keep the new class semantically as a string,
and they
I completely agree with Andrew Barnert.
I just want to add a little comment about overriding the `+=` (and `+`)
operator for StringIO. Since StringIO is a stream --not a string--, I think
`StringIO` should continue to use the common interface for streams in Python.
`write()` and `read()` are
> On Mar 29, 2020, at 10:57, Paul Sokolovsky wrote:
>
>
> It is a well-known anti-pattern to use a string as a string buffer, to
> construct a long (perhaps very long) string piece-wise. A running
> example is:
>
> buf = ""
> for i in range(5):
> buf += "foo"
> print(buf)
>
> An
78 matches
Mail list logo
