Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Steven D'Aprano]

On Fri, Apr 26, 2019 at 12:17:57AM -0400, Terry Reedy wrote:
> On 4/25/2019 7:12 PM, Greg Ewing wrote:
> >Steven D'Aprano wrote:
> >>I too often forget that reverse() returns an iterator,
> 
> I presume you mean reversed().  list.reverse() is a list

Yes, I meant reversed(), not list.reverse() which is an in-place mutator 
method and returns None.

> >That seems like a mistake. Shouldn't it return a view?
> 
> RL = reversed(somelist) is already partly view-like.  The nth next call 
> returns the nth item at the time of the next call, rather than at the 
> time of the reversed call.  However, the number of items produced by 
> next calls is the length of the list at the time of the reversed call. 


That's not quite correct:

py> L = [1, 2, 3]
py> R = reversed(L)
py> L.clear()
py> next(R)
Traceback (most recent call last):
  File "", line 1, in 
StopIteration


It seems that:

- in-place modifications in the list are reflected in the items
  yielded (so reversed() doesn't make a copy of the list);

- operations which extend the length of the list don't show up 
  in the reversed version;

- and operations which decrease the length of the list decrease
  the number of items yielded.

That suggests to me an implementation similar to:

# untested
def reversed(alist):
N = len(alist)
for i in range(N-1, -1, -1):
try:
yield alist[i]
except IndexError:
break
raise StopIteration

which I suppose is close to what you meant here:

> The first next(RL) is the current somelist[captured_length].



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Steven D'Aprano]

On Fri, Apr 26, 2019 at 11:12:51AM +1200, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >I too often forget that reverse() returns an iterator,
> 
> That seems like a mistake. Shouldn't it return a view?

I don't know what it "should" or "shouldn't" it return, but it actually 
does return an iterator:

py> L = [1, 2, 3]
py> R = reversed(L)
py> hasattr(R, '__iter__') and iter(R) is R
True


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Terry Reedy]

On 4/25/2019 7:12 PM, Greg Ewing wrote:

Steven D'Aprano wrote:

I too often forget that reverse() returns an iterator,


I presume you mean reversed().  list.reverse() is a list


That seems like a mistake. Shouldn't it return a view?


RL = reversed(somelist) is already partly view-like.  The nth next call 
returns the nth item at the time of the next call, rather than at the 
time of the reversed call.  However, the number of items produced by 
next calls is the length of the list at the time of the reversed call. 
The first next(RL) is the current somelist[captured_length].


>>> somelist = [1,2,3]
>>> RL = reversed(somelist)
>>> somelist[-1] = None
>>> somelist.append(4)
>>> list(RL)
[None, 2, 1]



--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Greg Ewing]

Steven D'Aprano wrote:

I too often forget that reverse() returns an iterator,


That seems like a mistake. Shouldn't it return a view?

--
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()? (John Lin)

[Steven D'Aprano]

On Wed, Apr 24, 2019 at 01:50:48AM +0800, Thautwarm Zhao wrote:

> However, the reason why we don't need list.rindex but do for str.rindex is
> simple I'd say: str is immutable and has no O(1) reverse method.
> 
> On the other hand, when it comes to list, you can use list.index after
> list.reverse, and after a bunch of operations you can resume the state by
> invoking list.reverse again.

list reverse is not O(1), and flipping the order, then flipping the 
order back again is not safe if the list could be accessed by two or 
more threads.

(The call to reverse itself is thread-safe, but not the operations in 
between.)


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Steven D'Aprano]

On Tue, Apr 23, 2019 at 10:28:29AM -0700, Brett Cannon wrote:

> Given "abcdefabcdefabcdef", what is the last result of "abc"?
> x.rindex("abc") will tell you.
> 
> Given [1, 2, 3, 4, 5, 1, 2, 3, 4, 5] where is the last result of 3?
> reversed(x).index(3) will tell you (or x[::-1]).

That first version doesn't work, as list_reverseiterator objects don't 
have an index method. You're not the only person to make that error, I 
too often forget that reverse() returns an iterator, not a list.

The second is easy to get wrong, because it returns the wrong index:

  # Get the item following the last instance of spam.
  index = x[::-1].index(spam)
  print(x[index+1])

In your example, the correct index is 7 but the returned value is 2.


> Notice how with lists you can easily reverse them and still get at the
> value since you are searching per index. 

"Easily" hides a lot of copying behind the scenes. If the list is a 
non-trivial size, that can be very wasteful, especially if you're doing 
it in a loop, or hidden in a function. Don't think about the case of a 
ten element list, think of a ten-billion element list.

Personally, I don't think I've every used list.index, let alone needed 
rindex. But I think we underestimate the difficulty and cost of faking 
an rindex method from index for those who need it (if anyone does).


> But with strings, you searching by
> a subslice that can be greater than 1 in which case you can't use a similar
> approach.

Of course you can: you "just" need to reverse the substring as well.

The conversions will be even more fiddly and error-prone:

py> s = "abc spam def spam ghi"
py> s.rindex('spam') == len(s) - s[::-1].index('spam'[::-1]) - len('spam')
True

but it can be done.



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Steven D'Aprano]

On Wed, Apr 24, 2019 at 08:59:18AM +0800, 林自均 wrote:
> Hi all,
> 
> Thanks for the explanation. Now I agree that the need for list.rindex() is
> not as common as str.rindex(). In fact, I only need list.rindex() when
> doing some algorithm problems. I guess that doesn't count as real need here.

Of course it's a "real need", but the question is whether it is common 
enough for somebody to do the work if it doesn't affect them personally.

Can you share an example of one of these algorithms?



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[林自均]

Hi all,

Thanks for the explanation. Now I agree that the need for list.rindex() is
not as common as str.rindex(). In fact, I only need list.rindex() when
doing some algorithm problems. I guess that doesn't count as real need here.

Best,
John Lin

Guido van Rossum  於 2019年4月24日 週三 上午4:20寫道：

> On Tue, Apr 23, 2019 at 1:02 PM MRAB  wrote:
>
>> On 2019-04-23 18:52, Terry Reedy wrote:
>> > On 4/23/2019 2:44 AM, 林自均 wrote:
>> >> Hi all,
>> >>
>> >> I found that there are str.index() and str.rindex(), but there is only
>> >> list.index() and no list.rindex().
>> >
>> > str.index and list.index are related but not the same.  The consistency
>> > argument is better applied to find-rfind, index-rindex,
>> > partition-rpartition, etc.
>> >
>> > It is much more common to process strings right to left or in both
>> > directions, than process lists right to left or in both directions.
>> > Moreover, lists have a reverse method, strings do not.
>> > ''.join(reversed(somestring)) is likely slower, especially if there many
>> > non-ascii chars.  Moreover, somestring.rindex(substring) would have to
>> > have both somestring and substring reversed when substring is more than
>> > one char.
>> >
>> You can reverse a string with somestring[::-1].
>>
>> Personally, I'm not convinced by the "lists can be reversed" argument.
>>
>
> Me neither, though for substring checks, reversing the string would be
> even more cumbersome (you'd have to reverse the query string too).
>
> My money is on "nobody uses this for lists".
>
> Some use cases for rindex() on strings that I found in a large codebase
> here include searching a pathname for the final slash, a list of
> comma-separated items for the last comma, a fully-qualified module name for
> the last period, and some ad-hoc parsing of other things. The "last
> separator" use cases are the most common and here rindex() sounds very
> useful.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him/his **(why is my pronoun here?)*
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Guido van Rossum]

On Tue, Apr 23, 2019 at 1:02 PM MRAB  wrote:

> On 2019-04-23 18:52, Terry Reedy wrote:
> > On 4/23/2019 2:44 AM, 林自均 wrote:
> >> Hi all,
> >>
> >> I found that there are str.index() and str.rindex(), but there is only
> >> list.index() and no list.rindex().
> >
> > str.index and list.index are related but not the same.  The consistency
> > argument is better applied to find-rfind, index-rindex,
> > partition-rpartition, etc.
> >
> > It is much more common to process strings right to left or in both
> > directions, than process lists right to left or in both directions.
> > Moreover, lists have a reverse method, strings do not.
> > ''.join(reversed(somestring)) is likely slower, especially if there many
> > non-ascii chars.  Moreover, somestring.rindex(substring) would have to
> > have both somestring and substring reversed when substring is more than
> > one char.
> >
> You can reverse a string with somestring[::-1].
>
> Personally, I'm not convinced by the "lists can be reversed" argument.
>

Me neither, though for substring checks, reversing the string would be even
more cumbersome (you'd have to reverse the query string too).

My money is on "nobody uses this for lists".

Some use cases for rindex() on strings that I found in a large codebase
here include searching a pathname for the final slash, a list of
comma-separated items for the last comma, a fully-qualified module name for
the last period, and some ad-hoc parsing of other things. The "last
separator" use cases are the most common and here rindex() sounds very
useful.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[MRAB]

On 2019-04-23 18:52, Terry Reedy wrote:

On 4/23/2019 2:44 AM, 林自均 wrote:

Hi all,

I found that there are str.index() and str.rindex(), but there is only 
list.index() and no list.rindex().


str.index and list.index are related but not the same.  The consistency
argument is better applied to find-rfind, index-rindex,
partition-rpartition, etc.

It is much more common to process strings right to left or in both
directions, than process lists right to left or in both directions.
Moreover, lists have a reverse method, strings do not.
''.join(reversed(somestring)) is likely slower, especially if there many
non-ascii chars.  Moreover, somestring.rindex(substring) would have to
have both somestring and substring reversed when substring is more than
one char.


You can reverse a string with somestring[::-1].

Personally, I'm not convinced by the "lists can be reversed" argument.

[snip]
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Terry Reedy]

On 4/23/2019 2:44 AM, 林自均 wrote:

Hi all,

I found that there are str.index() and str.rindex(), but there is only 
list.index() and no list.rindex().


str.index and list.index are related but not the same.  The consistency 
argument is better applied to find-rfind, index-rindex, 
partition-rpartition, etc.


It is much more common to process strings right to left or in both 
directions, than process lists right to left or in both directions. 
Moreover, lists have a reverse method, strings do not.
''.join(reversed(somestring)) is likely slower, especially if there many 
non-ascii chars.  Moreover, somestring.rindex(substring) would have to 
have both somestring and substring reversed when substring is more than 
one char.


 So I filed the issue
https://bugs.python.org/issue36639 to provide list.rindex(). However, 
the issue was rejected and closed with the comment:


 > There were known, strong use cases for str.rindex().  The 
list.rindex() method was intentionally omitted.  AFAICT no compelling 
use cases have arisen, so we should continue to leave it out.  In 
general, we don't grow the core APIs unnecessarily.


However, I am not sure what the known, strong use cases for str.rindex() 
are. Why doesn't the strong use cases apply on list.rindex()? Could 
anyone give me some examples? Thanks.


Omitting tests of rindex and rfind themselves:

Searching 'rindex' in C:\Programs\Python38\lib\*.py ...
C:\Programs\Python38\lib\_markupbase.py: 55: pos = 
rawdata.rindex("\n", i, j) # Should not fail
C:\Programs\Python38\lib\collections\__init__.py: 1260: def 
rindex(self, sub, start=0, end=_sys.maxsize):
C:\Programs\Python38\lib\collections\__init__.py: 1261: return 
self.data.rindex(sub, start, end)
C:\Programs\Python38\lib\pydoc_data\topics.py: 10125: 
'str.rindex(sub[, start[, end]])\n'
C:\Programs\Python38\lib\test\pickletester.py: 2543: 
self.assertEqual((pickled.rindex(b"abcd") + len(b"abcd") -
C:\Programs\Python38\lib\test\pickletester.py: 2574: 
self.assertEqual((pickled.rindex(b"abcd") + len(b"abcd") -
C:\Programs\Python38\lib\test\test_baseexception.py: 46: 
depth = exc_line.rindex('-')
C:\Programs\Python38\lib\test\test_bigmem.py: 310: 
self.assertEqual(s.rindex(_(' ')),
C:\Programs\Python38\lib\test\test_bigmem.py: 311: 
   sublen + size + SUBSTR.rindex(_(' ')))
C:\Programs\Python38\lib\test\test_bigmem.py: 312: 
self.assertEqual(s.rindex(SUBSTR), sublen + size)
C:\Programs\Python38\lib\test\test_bigmem.py: 313: 
self.assertEqual(s.rindex(_(' '), 0, sublen + size - 1),
C:\Programs\Python38\lib\test\test_bigmem.py: 314: 
   SUBSTR.rindex(_(' ')))
C:\Programs\Python38\lib\test\test_bigmem.py: 315: 
self.assertEqual(s.rindex(SUBSTR, 0, sublen + size), 0)
C:\Programs\Python38\lib\test\test_bigmem.py: 316: 
self.assertEqual(s.rindex(_('i')),
C:\Programs\Python38\lib\test\test_bigmem.py: 317: 
   sublen + size + SUBSTR.rindex(_('i')))
C:\Programs\Python38\lib\test\test_bigmem.py: 318: 
self.assertEqual(s.rindex(_('i'), 0, sublen), SUBSTR.rindex(_('i')))
C:\Programs\Python38\lib\test\test_bigmem.py: 319: 
self.assertEqual(s.rindex(_('i'), 0, sublen + size),
C:\Programs\Python38\lib\test\test_bigmem.py: 320: 
   SUBSTR.rindex(_('i')))
C:\Programs\Python38\lib\test\test_bigmem.py: 321: 
self.assertRaises(ValueError, s.rindex, _('j'))
C:\Programs\Python38\lib\test\test_bytes.py: 566: 
self.assertEqual(b.rindex(b'ss'), 5)
C:\Programs\Python38\lib\test\test_bytes.py: 567: 
self.assertRaises(ValueError, b.rindex, b'w')
C:\Programs\Python38\lib\test\test_bytes.py: 568: 
self.assertRaises(ValueError, b.rindex, b'mississippian')
C:\Programs\Python38\lib\test\test_bytes.py: 570: 
self.assertEqual(b.rindex(i), 10)
C:\Programs\Python38\lib\test\test_bytes.py: 571: 
self.assertRaises(ValueError, b.rindex, w)
C:\Programs\Python38\lib\test\test_bytes.py: 573: 
self.assertEqual(b.rindex(b'ss', 3), 5)
C:\Programs\Python38\lib\test\test_bytes.py: 574: 
self.assertEqual(b.rindex(b'ss', 0, 6), 2)
C:\Programs\Python38\lib\test\test_bytes.py: 576: 
self.assertEqual(b.rindex(i, 1, 3), 1)
C:\Programs\Python38\lib\test\test_bytes.py: 577: 
self.assertEqual(b.rindex(i, 3, 9), 7)
C:\Programs\Python38\lib\test\test_bytes.py: 578: 
self.assertRaises(ValueError, b.rindex, w, 1, 3)
C:\Programs\Python38\lib\test\test_bytes.py: 768: 
self.assertEqual(3, b.rindex(l, None))
C:\Programs\Python38\lib\test\test_bytes.py: 769: 
self.assertEqual(3, b.rindex(l, -2, None))
C:\Programs\Python38\lib\test\test_bytes.py: 770: 
self.assertEqual(2, b.rindex(l, None, -2))
C:\Programs\Python38\lib\test\test_bytes.py: 771: 
self.assertEqual(0, b.rindex(h, None, None))
C:\Programs\Python38\lib\test\test_bytes.py: 791: for method in 
(b.count, b.find, b.index, b.rfind, b.rindex):
C:\Programs\Python38\lib\test\test_bytes.py: 806: 
self.assertRaisesRegex(TypeError, r'\brindex\b', b.rindex,
C:\Programs\Python38\lib\test\test_descr.py: 3596: try: 
''.rindex('5')
C:\Programs\Python38\lib\test\test_descr.py: 3598:   

Re: [Python-ideas] What are the strong use cases for str.rindex()? (John Lin)

[Thautwarm Zhao]

IMO, there're lots of use cases in parsing related stuffs, which requires
rindex a lot, say, when you have generated a tokenizer which might across
multiple lines:

line 8:   X """
line 9:
line 10:   """

In this case, we need to extract 2 tokens X and , a multiline whitespace
string. After getting each token we're to compute/update the current column
and line number. If the line number gets advanced then we use rindex('\n')
to help with updating the new column number, otherwise,  col_offset +=
len(current_token) .

However, the reason why we don't need list.rindex but do for str.rindex is
simple I'd say: str is immutable and has no O(1) reverse method.

On the other hand, when it comes to list, you can use list.index after
list.reverse, and after a bunch of operations you can resume the state by
invoking list.reverse again.


On Wed, Apr 24, 2019, 12:11 AM  Send Python-ideas mailing list submissions to
> python-ideas@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/python-ideas
> or, via email, send a message with subject or body 'help' to
> python-ideas-requ...@python.org
>
> You can reach the person managing the list at
> python-ideas-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-ideas digest..."
> Today's Topics:
>
>1. What are the strong use cases for str.rindex()? (???)
>
>
>
> -- Forwarded message --
> From: "林自均" 
> To: python-ideas@python.org
> Cc:
> Bcc:
> Date: Tue, 23 Apr 2019 14:44:25 +0800
> Subject: [Python-ideas] What are the strong use cases for str.rindex()?
> Hi all,
>
> I found that there are str.index() and str.rindex(), but there is only
> list.index() and no list.rindex(). So I filed the issue
> https://bugs.python.org/issue36639 to provide list.rindex(). However, the
> issue was rejected and closed with the comment:
>
> > There were known, strong use cases for str.rindex().  The list.rindex()
> method was intentionally omitted.  AFAICT no compelling use cases have
> arisen, so we should continue to leave it out.  In general, we don't grow
> the core APIs unnecessarily.
>
> However, I am not sure what the known, strong use cases for str.rindex()
> are. Why doesn't the strong use cases apply on list.rindex()? Could anyone
> give me some examples? Thanks.
>
> Best,
> John Lin
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What are the strong use cases for str.rindex()?

[Brett Cannon]

Given "abcdefabcdefabcdef", what is the last result of "abc"?
x.rindex("abc") will tell you.

Given [1, 2, 3, 4, 5, 1, 2, 3, 4, 5] where is the last result of 3?
reversed(x).index(3) will tell you (or x[::-1]).

Notice how with lists you can easily reverse them and still get at the
value since you are searching per index. But with strings, you searching by
a subslice that can be greater than 1 in which case you can't use a similar
approach.

On Mon, Apr 22, 2019 at 11:47 PM 林自均  wrote:

> Hi all,
>
> I found that there are str.index() and str.rindex(), but there is only
> list.index() and no list.rindex(). So I filed the issue
> https://bugs.python.org/issue36639 to provide list.rindex(). However, the
> issue was rejected and closed with the comment:
>
> > There were known, strong use cases for str.rindex().  The list.rindex()
> method was intentionally omitted.  AFAICT no compelling use cases have
> arisen, so we should continue to leave it out.  In general, we don't grow
> the core APIs unnecessarily.
>
> However, I am not sure what the known, strong use cases for str.rindex()
> are. Why doesn't the strong use cases apply on list.rindex()? Could anyone
> give me some examples? Thanks.
>
> Best,
> John Lin
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] What are the strong use cases for str.rindex()?

[林自均]

Hi all,

I found that there are str.index() and str.rindex(), but there is only
list.index() and no list.rindex(). So I filed the issue
https://bugs.python.org/issue36639 to provide list.rindex(). However, the
issue was rejected and closed with the comment:

> There were known, strong use cases for str.rindex().  The list.rindex()
method was intentionally omitted.  AFAICT no compelling use cases have
arisen, so we should continue to leave it out.  In general, we don't grow
the core APIs unnecessarily.

However, I am not sure what the known, strong use cases for str.rindex()
are. Why doesn't the strong use cases apply on list.rindex()? Could anyone
give me some examples? Thanks.

Best,
John Lin
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/