Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[MRAB]

On 2019-10-28 19:42, Terry Reedy wrote:

On 10/28/2019 2:47 PM, MRAB wrote:


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling 


If you're using Windows, you'd just install the different Python 
versions into different folders: Python37 for Python 3.7 and Python38 
for Python 3.8. The installer doesn't that by default.



Oops! Should've been "does that by default".


The installer puts them into separate directories by default.  The
default path depends on whether one installs for all users or just one.
Or one can select the path preceeding 'pythonxy'.  In any case, one can
use the 'py' launcher to chose which PSF installed python to run.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[Terry Reedy]

On 10/28/2019 2:47 PM, MRAB wrote:


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling 


If you're using Windows, you'd just install the different Python 
versions into different folders: Python37 for Python 3.7 and Python38 
for Python 3.8. The installer doesn't that by default.


The installer puts them into separate directories by default.  The 
default path depends on whether one installs for all users or just one. 
Or one can select the path preceeding 'pythonxy'.  In any case, one can 
use the 'py' launcher to chose which PSF installed python to run.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[Terry Reedy]

On 10/28/2019 2:27 PM, DL Neil via Python-list wrote:

Dottore,


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling 
Python 3.8.

Conflicts problems.
Thankyou
Ugo Donini
Inviato da Posta per Windows 10



If the question is about updating Python 3.7 'in place' (and not 
upsetting any applications or systems programs which use Python), then 
"yes, this is possible": you may be able to update r3.7.0 to r3.7.4, for 
example (only the last digit changes). I see few compelling reasons to 
do-so, but YMMV (you have your own reasons).


The reason to update 3.7 or any x.y is to get the bugfixes, which 
eventually number in the hundreds.


If the question is a desire to run both Python 3.7 AND Python 3.8 on the 
same machine, then the PSL's (Python Standard Library) answer is "venv".


On Windows, the answer is to install 3.7 and 3.8 into separate 
directories and use the py launcher, installed by the Windows python 
installer, to select  which to run.



--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[MRAB]

On 2019-10-28 18:27, DL Neil via Python-list wrote:

Dottore,


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling Python 
3.8.
Conflicts problems.
Thankyou
Ugo Donini
Inviato da Posta per Windows 10



If the question is about updating Python 3.7 'in place' (and not
upsetting any applications or systems programs which use Python), then
"yes, this is possible": you may be able to update r3.7.0 to r3.7.4, for
example (only the last digit changes). I see few compelling reasons to
do-so, but YMMV (you have your own reasons).

If the question is a desire to run both Python 3.7 AND Python 3.8 on the
same machine, then the PSL's (Python Standard Library) answer is "venv".
Which you will find described in the docs (documentation) at "12.
Virtual Environments and Packages".

For my own reasons, instead of Python virtual environments, I prefer to
separate different clients/projects by keeping them in their own
VirtualBox VM. Some dislike this particular solution as it lacks "open
source" principles. There are other "container" technologies worth
review. However, I suggest that this type of solution is the preserve of
the more advanced programmer/computer user.


WebRefs:
https://docs.python.org/3/tutorial/venv.html
https://www.virtualbox.org/

If you're using Windows, you'd just install the different Python 
versions into different folders: Python37 for Python 3.7 and Python38 
for Python 3.8. The installer doesn't that by default.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[DL Neil via Python-list]

Dottore,


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling Python 
3.8.
Conflicts problems.
Thankyou
Ugo Donini
Inviato da Posta per Windows 10



If the question is about updating Python 3.7 'in place' (and not 
upsetting any applications or systems programs which use Python), then 
"yes, this is possible": you may be able to update r3.7.0 to r3.7.4, for 
example (only the last digit changes). I see few compelling reasons to 
do-so, but YMMV (you have your own reasons).


If the question is a desire to run both Python 3.7 AND Python 3.8 on the 
same machine, then the PSL's (Python Standard Library) answer is "venv". 
Which you will find described in the docs (documentation) at "12. 
Virtual Environments and Packages".


For my own reasons, instead of Python virtual environments, I prefer to 
separate different clients/projects by keeping them in their own 
VirtualBox VM. Some dislike this particular solution as it lacks "open 
source" principles. There are other "container" technologies worth 
review. However, I suggest that this type of solution is the preserve of 
the more advanced programmer/computer user.



WebRefs:
https://docs.python.org/3/tutorial/venv.html
https://www.virtualbox.org/
--
Ciao =dn
--
https://mail.python.org/mailman/listinfo/python-list

Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[Dott. Ugo Donini]

I cannot use Python 3.8 on my PC with installed Python 3.7. 
Is it possible to update the existing Python 3.7 without reinstalling Python 
3.8.
Conflicts problems.

Thankyou

Ugo Donini

Inviato da Posta per Windows 10

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't figure out how to do something using ctypes (and maybe struct?)

[Peter Otten]

inhahe wrote:

> I need to make a list of instances of a Structure, then I need to make an
> instance of another Structure, one of the fields of which needs to be an
> arbitrary-length array of pointers to the instances in the list. How do I
> do that?
> 
> Just in case it helps, I'll include what I tried that didn't work:
> --
> class NoteEvent(ctypes.Structure):
>   _fields_ = [('type', ctypes.c_int),
>   ('byteSize', ctypes.c_int),
>   ('deltaFrames', ctypes.c_int),
>   ('flags', ctypes.c_int),
>   ('noteLength', ctypes.c_int),
>   ('noteOffset', ctypes.c_int),
>   ('commandCode', ctypes.c_char),
>   ('noteNumber', ctypes.c_char),
>   ('velocity', ctypes.c_char)]
> 
> def mkVstEvents(events):
>   class Events(ctypes.Structure):
> _fields_ = [('numEvents', ctypes.c_int),
> ('reserved', ctypes.c_int),
> ('eventspointerarray', ctypes.c_void_p * len(events))]
>   return Events(len(events), 0, tuple([ctypes.pointer(event) for event in
> events]))

...

> RuntimeError: (c_void_p_Array_1) :
> incompatible types, LP_NoteEvent instance instead of c_void_p instance

Two things that you can try:

> ('eventspointerarray', ctypes.c_void_p * len(events))]

Either change c_void_p to POINTER(NoteEvent) like

https://stackoverflow.com/questions/7015487/ctypes-variable-length-structures

or cast the NoteEvent pointer to void *:

>   return Events(len(events), 0, tuple([ctypes.pointer(event) for event in
> events]))

... cast(pointer(event), c_void_p) ...


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't figure out how to do something using ctypes (and maybe struct?)

[Steven D'Aprano]

On Fri, 10 Aug 2018 18:40:11 -0400, inhahe wrote:

> I need to make a list of instances of a Structure, then I need to make
> an instance of another Structure, one of the fields of which needs to be
> an arbitrary-length array of pointers to the instances in the list. How
> do I do that?
> 
> Just in case it helps, I'll include what I tried that didn't work:

How about simplifying your example to the smallest and simplest example 
of the problem? Your example has:

- two functions;

- one method that seems to have become unattached from its class;

- two classes;

- using 12 different fields.

Surely not all of that detail is specific to the problem you are 
happening. If you can simplify the problem, the solution may be more 
obvious.

It might help to read this: http://sscce.org/

By the way, unrelated to your specific problem but possibly relevant 
elsewhere, you have this function:


> def mkVstEvents(events):
> class Events(ctypes.Structure):
> _fields_ = [ ... ]
> return Events( ... )

You might not be aware of this, but that means that every time you call 
mkVstEvents, you get a singleton instance of a new and distinct class 
that just happens to have the same name and layout.

So if you did this:

a = mkVstEvents( ... )
b = mkVstEvents( ... )

then a and b would *not* be instances of the same class:

isinstance(a, type(b))  # returns False
isinstance(b, type(a))  # returns False
type(a) == type(b)  # also False


Each time you call the function, it creates a brand new class, always 
called Events, creates a single instance of that class, and returns it. 
That is especially wasteful of memory, since classes aren't small.

py> class Events(ctypes.Structure):
... pass
...
py> sys.getsizeof(Events)
508


Unless that's what you intended, you ought to move the class outside of 
the function.


class Events(ctypes.Structure):
_fields_ = [ ... ]

def mkVstEvents(events):
return Events( ... )




-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Can't figure out how to do something using ctypes (and maybe struct?)

[inhahe]

I need to make a list of instances of a Structure, then I need to make an
instance of another Structure, one of the fields of which needs to be an
arbitrary-length array of pointers to the instances in the list. How do I
do that?

Just in case it helps, I'll include what I tried that didn't work:
--
class NoteEvent(ctypes.Structure):
  _fields_ = [('type', ctypes.c_int),
  ('byteSize', ctypes.c_int),
  ('deltaFrames', ctypes.c_int),
  ('flags', ctypes.c_int),
  ('noteLength', ctypes.c_int),
  ('noteOffset', ctypes.c_int),
  ('commandCode', ctypes.c_char),
  ('noteNumber', ctypes.c_char),
  ('velocity', ctypes.c_char)]

def mkVstEvents(events):
  class Events(ctypes.Structure):
_fields_ = [('numEvents', ctypes.c_int),
('reserved', ctypes.c_int),
('eventspointerarray', ctypes.c_void_p * len(events))]
  return Events(len(events), 0, tuple([ctypes.pointer(event) for event in
events]))

def mkNoteEvent(deltaFrames, flags, noteNumber, channel, velocity,
noteLength, noteOffset, detune, noteOffVelocity):
  vstEvent = NoteEvent(type=1, bytesize=11, deltaFrames=deltaFrames,
flags=1, noteLength=0, noteOffset=0,
   commandCode=chr(144+channel),
noteNumber=chr(noteNumber), velocity=chr(velocity))
  return vstEvent

 def onSendMidiButton(self, event):

plugins[0].plugin.process_events(ctypes.pointer(mkVstEvents([mkNoteEvent(deltaFrames=0,
flags=1, noteNumber=60, channel=0,

velocity=127, noteLength=0, noteOffset=0, detune=0,

noteOffVelocity=0)])))
--
Here's the error I got:
--
Traceback (most recent call last):
  File "soundshop.9.1.2.py", line 216, in onSendMidiButton
noteOffVelocity=0)])))
  File "soundshop.9.1.2.py", line 76, in mkVstEvents
return Events(len(events), 0, tuple([ctypes.pointer(event) for event in
events]))
RuntimeError: (c_void_p_Array_1) :
incompatible types, LP_NoteEvent instance instead of c_void_p instance
--
I also tried to use struct.pack:
--
def mkVstEvents(events):
  st = struct.pack("=ii"+"L"*len(events), len(events), 0,
*(ctypes.pointer(event) for event in events))
  return st
---
Here's the error I got:
--
Traceback (most recent call last):
  File "soundshop.9.3.py", line 213, in onSendMidiButton
noteOffVelocity=0)])))
  File "soundshop.9.3.py", line 72, in mkVstEvents
st = struct.pack("=ii"+"p"*len(events), len(events), 0,
*(ctypes.pointer(event) for event in events))
struct.error: argument for 'p' must be a string
--
I also tried replacing "p"*len(events) with "i"*len(events) and also with
"L"*len(events), both of which gave me errors.

Thanks for your help.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to do pd.read_csv with consecutive spaces or semi-colon as delimiters?

[breamoreboy]

On Tuesday, April 18, 2017 at 10:25:51 AM UTC+1, David Shi wrote:
> Any way to do that?
> The link to a sample dataset is as follows:
> http://www.ebi.ac.uk/ena/data/warehouse/search?query=%22geo_circ(-0.587,-90.5713,170)%22=sequence_release=text
> Looking forward to hearing from you.
> Regards.
> David

After less than one minute searching I found 
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html.  
It's the last time I do this for you as your repeated questions show no attempt 
at trying anything for yourself.  You haven't even had the common courtesy to 
respond when I have given you links in response to those questions.

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to do pd.read_csv with consecutive spaces or semi-colon as delimiters?

[Peter Otten]

David Shi via Python-list wrote:

> Any way to do that?
> The link to a sample dataset is as follows:
> 
http://www.ebi.ac.uk/ena/data/warehouse/search?query=%22geo_circ(-0.587,-90.5713,170)%22=sequence_release=text
> Looking forward to hearing from you.
> Regards.
> David

David, please stop reposting this question here again and and again, with 
only minor variations. The data you are trying to process is not in CSV 
format, so you have to look elsewhere to preprocess it.

If there are tools available in Python to achieve your goal they may be part 
of biopython, or there may be domain experts on the biopython mailing list

http://biopython.org/wiki/Mailing_lists

who know how to proceed. I recommend that you ask there, but with a few more 
details on what you are trying to achieve so that you don't come across as 
lazy and careless as with your above posting.

Thank you, and good luck.

-- 
https://mail.python.org/mailman/listinfo/python-list

How to do pd.read_csv with consecutive spaces or semi-colon as delimiters?

[David Shi via Python-list]

Any way to do that?
The link to a sample dataset is as follows:
http://www.ebi.ac.uk/ena/data/warehouse/search?query=%22geo_circ(-0.587,-90.5713,170)%22=sequence_release=text
Looking forward to hearing from you.
Regards.
David
-- 
https://mail.python.org/mailman/listinfo/python-list

How to do Integration testing of a C code/firmware in python

[puravnshah]

I am doing an Internship at a company. I have a multi threaded code/firmware in 
c. I have done its unit testing with cantata in c++ and now want to do 
Integration testing in python with the device connected to my PC. From where do 
I start. How and what to write. The code has various functions with sensor, 
adc,dac, etc

If any more info let me know I am a newbie with no experience so let me know if 
any mistake
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Jussi Piitulainen]

Steven D'Aprano writes:

> On Tue, 5 Jul 2016 07:51 pm, Jussi Piitulainen wrote:
>
>> Chris Angelico writes:
>> 
>>> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten wrote:
 It looks like

 $ python3 -c 'print({1, 2})'
 {1, 2}
 $ python3 -c 'print({2, 1})'
 {1, 2}

 will always print the same output. Can you construct a set from two
 small integers where this is not the case? What's the difference?
>>>
>>> Given that the display (iteration) order of sets is arbitrary, I'm not
>>> sure what the significance would ever be, but my guess is that the
>>> display order would be the same for any given set, if constructed this
>>> way. But it sounds as if you know of a set that behaves differently.
>> 
>> The first thing that came to mind, {-1,-2} and {-2,-1}.
>> 
>> But I haven't a clue. It doesn't happen with -1 and -3, or with another
>> pair that I tried, and it doesn't seem to be about object identity.
>
> The hash of most small ints is equal to the int itself:
>
> py> for i in range(100):
> ... assert hash(i) == i
> ...
> py>
>
> With one exception:
>
> py> hash(-2)
> -2
> py> hash(-1)
> -2

Thanks. That must be the explanation. I tried object identity but I did
not think of comparing hashes directly.

Amusing that I didn't know this, yet I happened to think of just this
one pair of numbers. Literally the first thing that I thought to try,
and it turns out to be the only thing.

> That's because in the C implementation of hash, -1 is used to indicate
> an error.
>
 What happens if you replace the ints with strings? Why?
>>>
>>> Then hash randomization kicks in, and you can run the exact same
>>> line of code multiple times and get different results. It's a coin
>>> toss.
>> 
>> Oh, nice, a new way to generate random bits in shell scripts.
>
> O_o
>
> You're joking, right?

Er, ok, better not escalate this: Of course I am.

> I'll just leave this here...
>
> https://docs.python.org/3.6/library/secrets.html

Ok.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Lawrence D’Oliveiro]

On Tuesday, July 5, 2016 at 9:51:21 PM UTC+12, Jussi Piitulainen wrote:
>
> Chris Angelico writes:
> 
>> Then hash randomization kicks in, and you can run the exact same line
>> of code multiple times and get different results. It's a coin toss.
> 
> Oh, nice, a new way to generate random bits in shell scripts.

Please, don’t do that...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Carl Meyer]

On 07/05/2016 05:50 AM, Chris Angelico wrote:
> On Tue, Jul 5, 2016 at 9:33 PM, Peter Otten <__pete...@web.de> wrote:
>> Chris Angelico wrote:
>>
>>> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten <__pete...@web.de> wrote:
 What will

 $ cat foo.py
 import foo
 class A: pass
 print(isinstance(foo.A(), A))
 $ python -c 'import foo'
 ...
 $ python foo.py
 ...

 print?
[snip]
>> The intended lesson was that there may be two distinct classes
>>
>> __main__.A and foo.A
[snip]
> The two distinct classes problem is a very real one, and comes of
> circular (or not-technically-circular, as in the second case) imports.

It can also come of pathological setups where a path and its parent are
both on sys.path, so all import paths have an "optional" prefix (but you
actually get a different copy of the module depending on whether you use
that prefix).

Carl



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Chris Angelico]

On Tue, Jul 5, 2016 at 9:33 PM, Peter Otten <__pete...@web.de> wrote:
> Chris Angelico wrote:
>
>> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten <__pete...@web.de> wrote:
>>> What will
>>>
>>> $ cat foo.py
>>> import foo
>>> class A: pass
>>> print(isinstance(foo.A(), A))
>>> $ python -c 'import foo'
>>> ...
>>> $ python foo.py
>>> ...
>>>
>>> print?
>>
>> I refuse to play around with isinstance and old-style classes.
>> Particularly when circular imports are involved. Run this under Python
>> 3 and/or explicitly subclass object, and then I'd consider it. :)
>
> The intended lesson was that there may be two distinct classes
>
> __main__.A and foo.A
>
> Even though not just classes, but every object created in the script is
> affected this seems to cause the most subtle bugs.
>
> Maybe the setup can be simplified or the question rephrased to make this
> clearer.

Like I said, change the commands to "python3", or explicitly subclass
object "class A(object): pass", and then it'll be using new-style
classes.

The two distinct classes problem is a very real one, and comes of
circular (or not-technically-circular, as in the second case) imports.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Peter Otten]

Chris Angelico wrote:

> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten <__pete...@web.de> wrote:
>> What will
>>
>> $ cat foo.py
>> import foo
>> class A: pass
>> print(isinstance(foo.A(), A))
>> $ python -c 'import foo'
>> ...
>> $ python foo.py
>> ...
>>
>> print?
> 
> I refuse to play around with isinstance and old-style classes.
> Particularly when circular imports are involved. Run this under Python
> 3 and/or explicitly subclass object, and then I'd consider it. :)

The intended lesson was that there may be two distinct classes

__main__.A and foo.A

Even though not just classes, but every object created in the script is 
affected this seems to cause the most subtle bugs.

Maybe the setup can be simplified or the question rephrased to make this 
clearer.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Steven D'Aprano]

On Tue, 5 Jul 2016 07:51 pm, Jussi Piitulainen wrote:

> Chris Angelico writes:
> 
>> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten wrote:
>>> It looks like
>>>
>>> $ python3 -c 'print({1, 2})'
>>> {1, 2}
>>> $ python3 -c 'print({2, 1})'
>>> {1, 2}
>>>
>>> will always print the same output. Can you construct a set from two
>>> small integers where this is not the case? What's the difference?
>>
>> Given that the display (iteration) order of sets is arbitrary, I'm not
>> sure what the significance would ever be, but my guess is that the
>> display order would be the same for any given set, if constructed this
>> way. But it sounds as if you know of a set that behaves differently.
> 
> The first thing that came to mind, {-1,-2} and {-2,-1}.
> 
> But I haven't a clue. It doesn't happen with -1 and -3, or with another
> pair that I tried, and it doesn't seem to be about object identity.

The hash of most small ints is equal to the int itself:

py> for i in range(100):
... assert hash(i) == i
...
py>

With one exception:

py> hash(-2)
-2
py> hash(-1)
-2

That's because in the C implementation of hash, -1 is used to indicate an
error.


>>> What happens if you replace the ints with strings? Why?
>>
>> Then hash randomization kicks in, and you can run the exact same line
>> of code multiple times and get different results. It's a coin toss.
> 
> Oh, nice, a new way to generate random bits in shell scripts.

O_o

You're joking, right?

I'll just leave this here...

https://docs.python.org/3.6/library/secrets.html




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Steven D'Aprano]

On Tue, 5 Jul 2016 06:36 pm, Peter Otten wrote:

> It looks like
> 
> $ python3 -c 'print({1, 2})'
> {1, 2}
> $ python3 -c 'print({2, 1})'
> {1, 2}
> 
> will always print the same output. Can you construct a set from two small
> integers where this is not the case? What's the difference?

Define "small". According to some mathematicians, any number you can write
down counts as small :-)


> What happens if you replace the ints with strings? Why?

Depends on the version of Python. Recent versions have hash randomisation
turned on, so the order of identical sets/dicts running in identical code
will vary from one run to another.


[steve@ando 3.6]$ ./python -c "print({'a', 'b', 'c', 'd'})"
{'c', 'b', 'd', 'a'}
[steve@ando 3.6]$ ./python -c "print({'a', 'b', 'c', 'd'})"
{'b', 'c', 'a', 'd'}
[steve@ando 3.6]$ ./python -c "print({'a', 'b', 'c', 'd'})"
{'c', 'a', 'd', 'b'}




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Jussi Piitulainen]

Chris Angelico writes:

> On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten wrote:
>> It looks like
>>
>> $ python3 -c 'print({1, 2})'
>> {1, 2}
>> $ python3 -c 'print({2, 1})'
>> {1, 2}
>>
>> will always print the same output. Can you construct a set from two small
>> integers where this is not the case? What's the difference?
>
> Given that the display (iteration) order of sets is arbitrary, I'm not
> sure what the significance would ever be, but my guess is that the
> display order would be the same for any given set, if constructed this
> way. But it sounds as if you know of a set that behaves differently.

The first thing that came to mind, {-1,-2} and {-2,-1}.

But I haven't a clue. It doesn't happen with -1 and -3, or with another
pair that I tried, and it doesn't seem to be about object identity.

>> What happens if you replace the ints with strings? Why?
>
> Then hash randomization kicks in, and you can run the exact same line
> of code multiple times and get different results. It's a coin toss.

Oh, nice, a new way to generate random bits in shell scripts.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Chris Angelico]

On Tue, Jul 5, 2016 at 4:33 PM, Steven D'Aprano
<steve+comp.lang.pyt...@pearwood.info> wrote:
> > Got any other tricky questions to add?
S
P
O
I
L
E
R

S
P
A
C
E



A
N
D

A

B
I
T

M
O
R
E

[Thanks Steven, I just copied and pasted your space. See? You can copy
and paste blank space and use it over and over. Could be a useful tip
for the people who can't find enough newlines to space out their code
or readability.]

> Explain metaclasses, descriptors, the mro, multiple inheritance, and the
> interaction between them. Why is the mro needed?

A metaclass is simply the type of the type of something. By default,
that's type, but if you type "metaclass=X" in a class definition, you
can make your class a type of something else. (Can I use the word
"type" in any other senses? I couldn't work "font" into the above.)

Descriptors are objects stored in the class which, when referenced
from an instance, return something other than themselves. The most
common example is functions; a function in a class becomes a bound
method on an instance, thus proving that descriptors make Python
better than JavaScript.

The MRO is simply the class hierarchy, starting from the current class
and ending with 'object', flattened out into a line. In a pure
single-inheritance situation, this is just the line of parents; when
multiple inheritance comes into play, the MRO becomes more
complicated, but still follows straight-forward rules (children before
parents, and parents in the order they're listed). The MRO's job is
twofold: attributes of classes earlier in the list will shadow those
later, and super() means "next in the MRO".

> obj.spam is a property. How do you get access to the underlying property 
> object
> itself?

Presumably you mean that obj.spam returns the result of calling the
property function, something like this:

class X:
@property
def spam(self):
return "something"

obj = X()

In that case, type(obj).spam is the property object, and
type(obj).spam.{fget,fset,fdel} are the three functions (of which two
will be None in my example).

> Why doesn't the property decorator work inside a Python 2 classic class?

Guido put an explicit check in to prevent you from doing so.
Attempting it will automatically download the latest Python 3.x, point
your web browser to a porting tutorial, and raise SyntaxError("Please
learn to use Python 3.").

Or: Instances of classic classes are all actually instances of
Instance(), so the property would have to be attached to Instance.

Or: It does.

rosuav@sikorsky:~$ python2
Python 2.7.11+ (default, Jun  2 2016, 19:34:15)
[GCC 5.3.1 20160528] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class X:
...   @property
...   def spam(self):
... print("Spamming")
... return "Hello"*5
...
>>> x=X()
>>> x
<__main__.X instance at 0x7f06e44fad88>
>>> x.spam
Spamming
'HelloHelloHelloHelloHello'

Take your pick.

> Explain Python scoping rules, in particular with respect to nested classes and
> nested functions (or even nested classes inside nested functions).

If you have an explicit global or nonlocal directive, the listed
name(s) are looked up in that scope only. Otherwise, Python looks in
local names, then in enclosing function scopes, then the module
globals, and finally the built-ins. While executing a class or
function body, local scope is that class/function. Enclosing class
scopes are not searched.

> Explain attribute lookup rules. When does an instance attribute take priority
> over a class attribute?

With simple attributes, an instance attribute always takes priority,
and then class attributes in method resolution order. With
descriptors... I'd have to go look it up. I've never done any
shadowing of descriptors, at least not deliberately/consciously.

> When is locals() writable? When is locals() writable, AND the effect shows up
> in the local scope? Explain how exec works inside a function, and the
> interaction with locals().

In CPython, locals() is always writable. It's just a dictionary. I'm
not sure whether this is a language guarantee or not.

When does the effect show up in local scope? When locals() is
globals(). Otherwise, the answer has to be "undefined" or at best
"implementation defined". To go further than that, I have to actually
experiment, rather than going from memory.

rosuav@sikorsky:~$ python3
Python 3.6.0a2+ (default:57f3514564f6, Jun 29 2016, 16:27:34)
[GCC 5.3.1 20160528] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...x = 1
...print(id(locals()), id(globals()))
...locals()["x"] = 2
...print(x)
...
>>> f()
140191368813960 140191369099208
1
>>

Re: How well do you know Python?

[Chris Angelico]

On Tue, Jul 5, 2016 at 6:36 PM, Peter Otten <__pete...@web.de> wrote:
> What will
>
> $ cat foo.py
> import foo
> class A: pass
> print(isinstance(foo.A(), A))
> $ python -c 'import foo'
> ...
> $ python foo.py
> ...
>
> print?

I refuse to play around with isinstance and old-style classes.
Particularly when circular imports are involved. Run this under Python
3 and/or explicitly subclass object, and then I'd consider it. :)

> It looks like
>
> $ python3 -c 'print({1, 2})'
> {1, 2}
> $ python3 -c 'print({2, 1})'
> {1, 2}
>
> will always print the same output. Can you construct a set from two small
> integers where this is not the case? What's the difference?

Given that the display (iteration) order of sets is arbitrary, I'm not
sure what the significance would ever be, but my guess is that the
display order would be the same for any given set, if constructed this
way. But it sounds as if you know of a set that behaves differently.

> What happens if you replace the ints with strings? Why?

Then hash randomization kicks in, and you can run the exact same line
of code multiple times and get different results. It's a coin toss.

rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'2', '1'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'2', '1'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'2', '1'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'2', '1'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}
rosuav@sikorsky:~$ python3 -c 'print({"1", "2"})'
{'1', '2'}

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Christian Gollwitzer]

Am 05.07.16 um 10:22 schrieb Steven D'Aprano:

On Tuesday 05 July 2016 16:38, Chris Angelico wrote:


On Tue, Jul 5, 2016 at 4:33 PM, Steven D'Aprano
 wrote:

What happens in this code snippet?

L = [1]
t = (L,)
t[0] += 1

Explain what value t has, and why.


Not sure you have that question right, because it simply gives a
TypeError. You can't add an integer to a list.


D'oh!


Try this instead:

t = ([0],)
t[0] += [1]



and after that, try:

>>> a=[0]
>>> t=(a,)
>>> a+=[1]


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Peter Otten]

Chris Angelico wrote:

> After some discussion with a Ruby on Rails programmer about where Ruby
> ends and where Rails begins (and it's definitely not where I'd have
> expected... Rails does a ton of monkey-patching, including of built-in
> types, to provide functionality that is strangely absent from the core
> language), I tried to come up with some somewhat-challenging Python
> questions. But to make them hard, I had to go a smidge more esoteric
> than the Ruby questions did Anyhow, see how you go. Assume Python
> 3.x unless stated.
> 
> 1) Under what circumstances can str.upper() return a string of
> different length to its input?
> 2) What exception do you get when you craft an impossible class hierarchy?
> a. ValueError b. TypeError c. types.ClassInheritanceError d.
> SyntaxError
> 3) What does `from __future__ import braces` do?
> 4) Which operator, removed from Python 3.0, can be reinstated with a
> 'joke' future directive?
> 5) What is the difference between the `/` and `//` operators in Python
> 2.7? In Python 3.x?
> 
> Got any other tricky questions to add?

What will 

$ cat foo.py 
import foo
class A: pass
print(isinstance(foo.A(), A))
$ python -c 'import foo'
...
$ python foo.py
...

print?

It looks like

$ python3 -c 'print({1, 2})'
{1, 2}
$ python3 -c 'print({2, 1})'
{1, 2}

will always print the same output. Can you construct a set from two small 
integers where this is not the case? What's the difference?

What happens if you replace the ints with strings? Why?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Steven D'Aprano]

On Tuesday 05 July 2016 16:38, Chris Angelico wrote:

> On Tue, Jul 5, 2016 at 4:33 PM, Steven D'Aprano
>  wrote:
>> What happens in this code snippet?
>>
>> L = [1]
>> t = (L,)
>> t[0] += 1
>>
>> Explain what value t has, and why.
> 
> Not sure you have that question right, because it simply gives a
> TypeError. You can't add an integer to a list.

D'oh!


Try this instead:

t = ([0],)
t[0] += [1]


-- 
Steve



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Chris Angelico]

On Tue, Jul 5, 2016 at 4:33 PM, Steven D'Aprano
 wrote:
> What happens in this code snippet?
>
> L = [1]
> t = (L,)
> t[0] += 1
>
> Explain what value t has, and why.

Not sure you have that question right, because it simply gives a
TypeError. You can't add an integer to a list.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Steven D'Aprano]

On Tuesday 05 July 2016 14:02, Chris Angelico wrote:

> After some discussion with a Ruby on Rails programmer about where Ruby
> ends and where Rails begins (and it's definitely not where I'd have
> expected... Rails does a ton of monkey-patching, including of built-in
> types, to provide functionality that is strangely absent from the core
> language), I tried to come up with some somewhat-challenging Python
> questions. But to make them hard, I had to go a smidge more esoteric
[...]
> Got any other tricky questions to add?


Explain metaclasses, descriptors, the mro, multiple inheritance, and the 
interaction between them. Why is the mro needed?

obj.spam is a property. How do you get access to the underlying property object 
itself?

Why doesn't the property decorator work inside a Python 2 classic class?

Explain Python scoping rules, in particular with respect to nested classes and 
nested functions (or even nested classes inside nested functions).

Explain attribute lookup rules. When does an instance attribute take priority 
over a class attribute?

When is locals() writable? When is locals() writable, AND the effect shows up 
in the local scope? Explain how exec works inside a function, and the 
interaction with locals().

Name all the falsey builtins.

Apart from exceptions, list the builtins, from memory. You can list the 
exceptions as well.

An easy one: list the Python keywords.

What happens in this code snippet?

L = [1]
t = (L,)
t[0] += 1

Explain what value t has, and why.

Explain what "yield from it" does and how it is different from:

for item in it:
yield item




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How well do you know Python?

[Steven D'Aprano]

On Tuesday 05 July 2016 14:02, Chris Angelico wrote:

> After some discussion with a Ruby on Rails programmer about where Ruby
> ends and where Rails begins (and it's definitely not where I'd have
> expected... Rails does a ton of monkey-patching, including of built-in
> types, to provide functionality that is strangely absent from the core
> language), I tried to come up with some somewhat-challenging Python
> questions. But to make them hard, I had to go a smidge more esoteric
> than the Ruby questions did Anyhow, see how you go. Assume Python
> 3.x unless stated.




S
P
O
I
L
E
R
 
S
P
A
C
E



A
N
D
 
A
 
B
I
T
 
M
O
R
E
 


> 1) Under what circumstances can str.upper() return a string of
> different length to its input?

Potentially any string might uppercase to a different length, depending on the 
content of the string and the case conversion rules for letters from that 
script. For Europeans, the simplest example is probably the German eszett, or 
\N{LATIN SMALL LETTER SHARP S}. There's an uppercase version, \N{LATIN CAPITAL 
LETTER SHARP S} but for historical reasons (and the idiosyncrasies of German) 
"ß" uppercases to "SS".


> 2) What exception do you get when you craft an impossible class hierarchy?
> a. ValueError b. TypeError c. types.ClassInheritanceError d. SyntaxError

Without checking, I would expect TypeError (90% confident) or SyntaxError 
(unlikely).


> 3) What does `from __future__ import braces` do?

Report your IP address and username to a secret database of people who will be 
sent to the gulags for re-education when the revolution comes. Or at least, 
they would be, if the Python Secret Underground, which most definitely doesn't 
exist, existed, which it doesn't.

Otherwise it prints the final word on the question of braces in Python.


> 4) Which operator, removed from Python 3.0, can be reinstated with a
> 'joke' future directive?

The <> not-equal operator.


> 5) What is the difference between the `/` and `//` operators in Python
> 2.7? In Python 3.x?

In Python 3, / means "true division" (i.e. what a calculator will do), and // 
means "integer division". E.g. 11/2 gives 5.5, while 11//2 gives 5.

In 2.7, there are two cases: with, or without "from __future__ import 
division". With the __future__ directive, they mean the same as in Python 3. 
Without, // means "integer division", but / depends on its arguments. If both 
arguments are ints, it performs integer division. If at least one is a float, 
it performs "true division" like a calculator.

Although to be pedantic, in all cases it depends on operator overloading :-)



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

How well do you know Python?

[Chris Angelico]

After some discussion with a Ruby on Rails programmer about where Ruby
ends and where Rails begins (and it's definitely not where I'd have
expected... Rails does a ton of monkey-patching, including of built-in
types, to provide functionality that is strangely absent from the core
language), I tried to come up with some somewhat-challenging Python
questions. But to make them hard, I had to go a smidge more esoteric
than the Ruby questions did Anyhow, see how you go. Assume Python
3.x unless stated.

1) Under what circumstances can str.upper() return a string of
different length to its input?
2) What exception do you get when you craft an impossible class hierarchy?
a. ValueError b. TypeError c. types.ClassInheritanceError d. SyntaxError
3) What does `from __future__ import braces` do?
4) Which operator, removed from Python 3.0, can be reinstated with a
'joke' future directive?
5) What is the difference between the `/` and `//` operators in Python
2.7? In Python 3.x?

Got any other tricky questions to add?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about how to do something in BeautifulSoup?

[Cody Piersall]

On Fri, Jan 22, 2016 at 8:01 AM, inhahe <inh...@gmail.com> wrote:
> Say I have the following HTML (I hope this shows up as plain text here
> rather than formatting):
>
> "Is
> today the day?"
>
> And I want to extract the "Is today the day?" part. There are other places
> in the document with  and , but this is the only place that
> uses color #00, so I want to extract anything that's within a color
> #00 style, even if it's nested multiple levels deep within that.
>
> - Sometimes the color is defined as RGB(0, 0, 0) and sometimes it's
defined
> as #00
> - Sometimes the  is within the  and sometimes the  is
> within the .
> - There may be other discrepancies I haven't noticed yet
>
> How can I do this in BeautifulSoup (or is this better done in lxml.html)?

I hope this helps you get started:

This may help you get started:

from bs4 import BeautifulSoup
from itertools import chain
soup = BeautifulSoup('''\
"Is
today the day?"
"Is
tomorrow the day?"''')

# We're going to get all the tags that specify the color, either using hex
or RGB.
# If you only want to get the span tags, just give the positional argument
'span' to
# find_all:
# for tag in chain(soup.find_all('span', style='color: #00;'),
#  soup.find_all('span', style='color: RGB(0, 0, 0);')):

for tag in chain(soup.find_all(style='color: #00;'),
 soup.find_all(style='color: RGB(0, 0, 0);')):
try:
print(tag.em.strong.text)
except AttributeError:
try:
print(tag.strong.em.text)
except AttributeError:
print('ooh no no text')

Cody
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about how to do something in BeautifulSoup?

[Peter Otten]

inhahe wrote:

> I hope this is an appropriate mailing list for BeautifulSoup questions,
> it's been a long time since I've used python-list and I don't remember if
> third-party modules are on topic. I did try posting to the BeautifulSoup
> mailing list on Google groups, but I've waited a day or two and my message
> hasn't been approved yet.
> 
> Say I have the following HTML (I hope this shows up as plain text here
> rather than formatting):
> 
> "Is today the day?"
> 
> And I want to extract the "Is today the day?" part. There are other places
> in the document with  and , but this is the only place that
> uses color #00, so I want to extract anything that's within a color
> #00 style, even if it's nested multiple levels deep within that.
> 
> - Sometimes the color is defined as RGB(0, 0, 0) and sometimes it's
> defined as #00
> - Sometimes the  is within the  and sometimes the  is
> within the .
> - There may be other discrepancies I haven't noticed yet
> 
> How can I do this in BeautifulSoup (or is this better done in lxml.html)?
> Thanks

I don't see how to do this with a lot of glue code, but it may get you 
started:

def recursive_attr(elem, path):
path = path.split("/")
for name in path:
if elem is None:
break
elem = getattr(elem, name)
return elem

def find(soup):
for outer in soup.find_all(
"span",
style=re.compile(r"color:\s*(RGB\(0,\s*0,\s* 0\)|#00)")):
for inner in [
recursive_attr(outer, "strong/em"),
recursive_attr(outer, "em/strong"),]:
if inner is not None:
yield inner.string

def normalize_ws(s):
return " ".join(s.split())

html = ...
soup = bs4.BeautifulSoup(html)
for match in find(soup):
print(normalize_ws(match))


-- 
https://mail.python.org/mailman/listinfo/python-list

Question about how to do something in BeautifulSoup?

[inhahe]

I hope this is an appropriate mailing list for BeautifulSoup questions,
it's been a long time since I've used python-list and I don't remember if
third-party modules are on topic. I did try posting to the BeautifulSoup
mailing list on Google groups, but I've waited a day or two and my message
hasn't been approved yet.

Say I have the following HTML (I hope this shows up as plain text here
rather than formatting):

"Is
today the day?"

And I want to extract the "Is today the day?" part. There are other places
in the document with  and , but this is the only place that
uses color #00, so I want to extract anything that's within a color
#00 style, even if it's nested multiple levels deep within that.

- Sometimes the color is defined as RGB(0, 0, 0) and sometimes it's defined
as #00
- Sometimes the  is within the  and sometimes the  is
within the .
- There may be other discrepancies I haven't noticed yet

How can I do this in BeautifulSoup (or is this better done in lxml.html)?
Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about how to do something in BeautifulSoup?

[Mario R. Osorio]

I think you'd do better using the pyparsing library


On Friday, January 22, 2016 at 9:02:00 AM UTC-5, inhahe wrote:
> I hope this is an appropriate mailing list for BeautifulSoup questions,
> it's been a long time since I've used python-list and I don't remember if
> third-party modules are on topic. I did try posting to the BeautifulSoup
> mailing list on Google groups, but I've waited a day or two and my message
> hasn't been approved yet.
> 
> Say I have the following HTML (I hope this shows up as plain text here
> rather than formatting):
> 
> "Is
> today the day?"
> 
> And I want to extract the "Is today the day?" part. There are other places
> in the document with  and , but this is the only place that
> uses color #00, so I want to extract anything that's within a color
> #00 style, even if it's nested multiple levels deep within that.
> 
> - Sometimes the color is defined as RGB(0, 0, 0) and sometimes it's defined
> as #00
> - Sometimes the  is within the  and sometimes the  is
> within the .
> - There may be other discrepancies I haven't noticed yet
> 
> How can I do this in BeautifulSoup (or is this better done in lxml.html)?
> Thanks

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to do integers to binary lists and back

[John Pote]

On 21/05/2015 23:31, MRAB wrote:

On 2015-05-21 23:20, John Pote wrote:

Hi everyone.
I recently had the problem of converting from an integer to its
representation as a list of binary bits, each bit being an integer 1 or
0, and vice versa. E.G.
0x53
becomes
[ 0, 1, 0, 1, 0, 0, 1, 1 ]

This I wanted to do for integers of many tens, if not hundreds, of bits.
Python very nicely expands integers to any size required, great feature.

Just wondered if there was a neat way of doing this without resorting to
a bit bashing loop.

Looking forward to some interesting answers,
John



I don't know how efficient you want it to be, but:

 number = 0x53
 bin(number)
'0b1010011'
 bin(number)[2 : ]
'1010011'
 list(map(int, bin(number)[2 : ]))
[1, 0, 1, 0, 0, 1, 1]

Thanks for the replies. Interesting that the offered solutions involve 
converting to a binary text string and then the individual chars back to 
ints. I had thought this would be a route to solve this problem but it 
seemed a bit 'heavy' hence I thought it worthwhile asking the question.


My solution to converting a list of 1s and 0s back to an int is
listLen = len( binList )
n = sum( [ binList[i]*( 2**(listLen-1 - i) ) for i in 
range(listLen)] )


In response to Ben Finney's question, I haven't done homework for 40 
years! Genuine problem, I had decided that the clearest way to write the 
algorithm I was working on was to use lists of 1s and 0s rather than 
normal ints.


Thanks for the help,
John
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to do integers to binary lists and back

[Ian Kelly]

On Thu, May 21, 2015 at 4:31 PM, Ben Finney ben+pyt...@benfinney.id.au wrote:
 {foo:d}.format(foo=foo)
 '4567'
 {foo:b}.format(foo=foo)
 '1000111010111'

Which since there's nothing else in the format string can be simplified to:

 format(foo, b)
'1000111010111'
-- 
https://mail.python.org/mailman/listinfo/python-list

How to do integers to binary lists and back

[John Pote]

Hi everyone.
I recently had the problem of converting from an integer to its 
representation as a list of binary bits, each bit being an integer 1 or 
0, and vice versa. E.G.

0x53
becomes
[ 0, 1, 0, 1, 0, 0, 1, 1 ]

This I wanted to do for integers of many tens, if not hundreds, of bits. 
Python very nicely expands integers to any size required, great feature.


Just wondered if there was a neat way of doing this without resorting to 
a bit bashing loop.


Looking forward to some interesting answers,
John


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to do integers to binary lists and back

[Ben Finney]

John Pote johnhp...@o2.co.uk writes:

 I recently had the problem of converting from an integer to its
 representation as a list of binary bits, each bit being an integer 1
 or 0, and vice versa.

Is this a homework assignment?

 E.G.
 0x53
 becomes
 [ 0, 1, 0, 1, 0, 0, 1, 1 ]

 foo = 4567

 foo
4567
 {foo:d}.format(foo=foo)
'4567'
 {foo:b}.format(foo=foo)
'1000111010111'

 foo_binary_text = {foo:b}.format(foo=foo)
 foo_binary_digits = list(foo_binary_text)
 foo_binary_digits
['1', '0', '0', '0', '1', '1', '1', '0', '1', '0', '1', '1', '1']

 Just wondered if there was a neat way of doing this without resorting
 to a bit bashing loop.

Python's string formatting and sequence types are quite powerful.

-- 
 \   “As far as the laws of mathematics refer to reality, they are |
  `\not certain, and as far as they are certain, they do not refer |
_o__)  to reality.” —Albert Einstein, 1983 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to do integers to binary lists and back

[MRAB]

On 2015-05-21 23:20, John Pote wrote:

Hi everyone.
I recently had the problem of converting from an integer to its
representation as a list of binary bits, each bit being an integer 1 or
0, and vice versa. E.G.
0x53
becomes
[ 0, 1, 0, 1, 0, 0, 1, 1 ]

This I wanted to do for integers of many tens, if not hundreds, of bits.
Python very nicely expands integers to any size required, great feature.

Just wondered if there was a neat way of doing this without resorting to
a bit bashing loop.

Looking forward to some interesting answers,
John



I don't know how efficient you want it to be, but:

 number = 0x53
 bin(number)
'0b1010011'
 bin(number)[2 : ]
'1010011'
 list(map(int, bin(number)[2 : ]))
[1, 0, 1, 0, 0, 1, 1]

--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Johannes Bauer]

On 05.06.2014 20:52, Ryan Hiebert wrote:
 2014-06-05 13:42 GMT-05:00 Johannes Bauer dfnsonfsdu...@gmx.de:
 
 On 05.06.2014 20:16, Paul Rubin wrote:
 Johannes Bauer dfnsonfsdu...@gmx.de writes:
 line = line[:-1]
 Which truncates the trailing \n of a textfile line.

 use line.rstrip() for that.

 rstrip has different functionality than what I'm doing.
 
 How so? I was using line=line[:-1] for removing the trailing newline, and
 just replaced it with rstrip('\n'). What are you doing differently?

Ah, I didn't know rstrip() accepted parameters and since you wrote
line.rstrip() this would also cut away whitespaces (which sadly are
relevant in odd cases).

Thanks for the clarification, I'll definitely introduce that.

Cheers,
Johannes

-- 
 Wo hattest Du das Beben nochmal GENAU vorhergesagt?
 Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Johannes Bauer]

On 05.06.2014 22:18, Ian Kelly wrote:

 Personally I tend toward rstrip('\r\n') so that I don't have to worry
 about files with alternative line terminators.

Hm, I was under the impression that Python already took care of removing
the \r at a line ending. Checking that right now:

(DOS encoded file y)
 for line in open(y, r): print(line.encode(utf-8))
...
b'foo\n'
b'bar\n'
b'moo\n'
b'koo\n'

Yup, the \r was removed automatically. Are there cases when it isn't?

Cheers,
Johannes

-- 
 Wo hattest Du das Beben nochmal GENAU vorhergesagt?
 Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Tim Chase]

On 2014-06-06 10:47, Johannes Bauer wrote:
  Personally I tend toward rstrip('\r\n') so that I don't have to
  worry about files with alternative line terminators.
 
 Hm, I was under the impression that Python already took care of
 removing the \r at a line ending. Checking that right now:
 
 (DOS encoded file y)
  for line in open(y, r): print(line.encode(utf-8))
 ...
 b'foo\n'
 b'bar\n'
 b'moo\n'
 b'koo\n'
 
 Yup, the \r was removed automatically. Are there cases when it
 isn't?

It's possible if the file is opened as binary:

 f = file('delme.txt', 'wb')
 f.write('hello\r\nworld\r\n')
 f.close()
 f = file('delme.txt', 'rb')
 for row in f: print repr(row)
... 
'hello\r\n'
'world\r\n'
 f.close()


-tkc

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Steven D'Aprano]

On Fri, 06 Jun 2014 10:47:44 +0200, Johannes Bauer wrote:

 Hm, I was under the impression that Python already took care of removing
 the \r at a line ending. Checking that right now:
[snip example]

This is called Universal Newlines. Technically it is a build-time 
option which applies when you build the Python interpreter from source, 
so, yes, some Pythons may not implement it at all. But I think that it 
has been on by default for a long time, and the option to turn it off may 
have been removed in Python 3.3 or 3.4. In practical terms, you should 
normally expect it to be on.


Here's the PEP that introduced it: 
http://legacy.python.org/dev/peps/pep-0278/


The idea is that when universal newlines support is enabled, by default 
will convert any of \n, \r or \r\n into \n when reading from a file in 
text mode, and convert back the other way when writing the file.

In binary mode, newlines are *never* changed.

In Python 3, you can return end-of-lines unchanged by passing newline='' 
to the open() function.

https://docs.python.org/2/library/functions.html#open
https://docs.python.org/3/library/functions.html#open




-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Grant Edwards]

On 2014-06-06, Roy Smith r...@panix.com wrote:

 Roy is using MT-NewsWatcher as a client.

 Yes.  Except for the fact that it hasn't kept up with unicode, I find 
 the U/I pretty much perfect.  I imagine at some point I'll be force to 
 look elsewhere, but then again, netnews is pretty much dead.

There are still a few active groups, but reading e-mail lists via NNTP
(in my case using slrn) via gmane is a huge reason to have an
efficient, well-designed news client.

If usenet does really pack it in someday and I have to switch from
comp.lang.python to the mailing list, it will be done by pointing slrn
at new.gmane.org -- not by having all those e-mails sent to me so I
can try to sort through them...

-- 
Grant Edwards   grant.b.edwardsYow! My NOSE is NUMB!
  at   
  gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Larry Hudson]

On 06/06/2014 01:42 AM, Johannes Bauer wrote:
snip

Ah, I didn't know rstrip() accepted parameters and since you wrote
line.rstrip() this would also cut away whitespaces (which sadly are
relevant in odd cases).



No problem.  If a parameter is used in the strip() family, than _only_ those characters are 
stripped.  Example:


 s = 'some text \n'
 print('{}'.format(s.rstrip()))  #  No parameter, strip all whitespace
some text
 print('{}'.format(s.rstrip('\n')))  #  Parameter is newline, only strip 
newlines
some text 

 -=- Larry

BTW, the strip() parameter (which must be a string) is not limited to whitespace, it can be used 
with any set of characters.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Marko Rauhamaa]

wxjmfa...@gmail.com:

 Unicode ?
 I have the feeling is similar as explaining,
 i (the imaginary number) is not equal to
 sqrt(-1).

 jmf

 PS Once I gave you a link pointing
 to unicode.org doc, you obviously did not read it.

Sir, you are an artist, a poet even!

With admiration,


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[alister]

On Thu, 05 Jun 2014 00:06:54 -0700, wxjmfauth wrote:

 Le mercredi 4 juin 2014 16:50:59 UTC+2, Michael Torrie a écrit :
 On 06/04/2014 12:50 AM, wxjmfa...@gmail.com wrote:
 
  Like many, you are not understanding unicode because
 
  you do not understand the coding of characters.
 
 
 
 If that is true, then I'm sure a well-written paragraph or two can set
 
 him straight.  You continually berate people for not understanding
 
 unicode, but you've posted nothing to explain anything, nor demonstrate
 
 your own understanding.  That's one reason your posts are so
 frustrating
 
 and considered trolling.  You never ever explain yourself, instead just
 
 flailing around and muttering about folks not understanding unicode,
 
 just as you've done here, true to form.
 
 
 
 
  
  You do not understand the coding of the characters
 
  because you do not understand the mathematics behind it.
 
 
 
 flamebaiting here... FSR *is* UTF-32 internally, compresses off leading
 
 zero bits during string creation.
 
 
 
  You focussed on the wrong problem.
 
 
 
 Frankly it is you who is focused on the wrong problem, at least with
 
 this particular thread.  I think you got distracted by the subject
 line.
 
  Chris's original post really has nothing to do with unicode at all.
 
 He's simply asking for use cases for string indexing where O(1) is
 
 desired or necessary.  Could be old Python 2 byte strings, or Python 3
 
 unicode strings.  It does not matter.  Unicode is orthogonal to his
 
 question.
 
 
 
 Maybe his purpose in asking the question is to justify a fixed-length
 
 encoding scheme (which is what FSR actually is), or maybe it is to
 
 explore the costs of using a much slower, but more compact,
 
 variable-length encoding scheme like UTF-8.  Particularly in the
 context
 
 of low-memory applications where unicode support would be nice, but
 
 memory is at a premium.  But either way, you got hung up on the wrong
 thing.
 
 
 
 
  
  (All this stuff has been discussed, tested and worked on
 
  20 (twenty) years ago.)
 
 
  
  Sorry.
 
 
 
 As am I.
 
 =
 
 Unicode ?
 I have the feeling is similar as explaining,
 i (the imaginary number) is not equal to sqrt(-1).
 
 jmf
 
 PS Once I gave you a link pointing to unicode.org doc, you obviously did
 not read it.



And you have may time been given a link explaining the problems with 
posting g=from google groups but deliberately choose to not make your 
replys readable.

-- 
If you're not part of the solution, you're part of the precipitate.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Mark H Harris]

On 6/5/14 10:39 AM, alister wrote:

{snipped all the mess}

And you have may time been given a link explaining the problems with
posting g=from google groups but deliberately choose to not make your
replys readable.



The problem is that thing look fine in google groups. What helps is 
getting to see what the mess looks like from Thunderbird or equivalent.



--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Johannes Bauer]

On 04.06.2014 02:39, Chris Angelico wrote:

 I know the collective experience of python-list can't fail to bring up
 a few solid examples here :)

Just also grepped lots of code and have surprisingly few instances of
index-search. Most are with constant indices. One particular example
that comes up a lot is

line = line[:-1]

Which truncates the trailing \n of a textfile line.

Then some indexing in the form of

negative = (line[0] == -)

All in all I'm actually a bit surprised this isn't too common.

Cheers,
Johannes


-- 
 Wo hattest Du das Beben nochmal GENAU vorhergesagt?
 Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Mark Lawrence]

On 05/06/2014 16:57, Mark H Harris wrote:

On 6/5/14 10:39 AM, alister wrote:

{snipped all the mess}

And you have may time been given a link explaining the problems with
posting g=from google groups but deliberately choose to not make your
replys readable.



The problem is that thing look fine in google groups. What helps is
getting to see what the mess looks like from Thunderbird or equivalent.



Wrong.  99.99% of people when asked politely take action so there is no 
problem.  The remaining 0.01% consists of one complete ignoramus.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Joshua Landau]

On 4 June 2014 15:50, Michael Torrie torr...@gmail.com wrote:
 On 06/04/2014 12:50 AM, wxjmfa...@gmail.com wrote:
 [Things]

 [Reply to things]

Please. Just don't.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[alister]

On Thu, 05 Jun 2014 18:15:31 +0100, Mark Lawrence wrote:

 The problem is that thing look fine in google groups. What helps is
 getting to see what the mess looks like from Thunderbird or equivalent.


 Wrong.  99.99% of people when asked politely take action so there is no
 problem.  The remaining 0.01% consists of one complete ignoramus.

Who has actively stated he will not change.
pretty much the same attitude he has constantly saying pythons unicode 
implementation is broken* without any valid supporting evidence.  


* Not just incomplete or inefficient but irrevocably broken.
 


-- 
Yow!  It's some people inside the wall!  This is better than mopping!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Paul Rubin]

Johannes Bauer dfnsonfsdu...@gmx.de writes:
 line = line[:-1]
 Which truncates the trailing \n of a textfile line.

use line.rstrip() for that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Johannes Bauer]

On 05.06.2014 20:16, Paul Rubin wrote:
 Johannes Bauer dfnsonfsdu...@gmx.de writes:
 line = line[:-1]
 Which truncates the trailing \n of a textfile line.
 
 use line.rstrip() for that.

rstrip has different functionality than what I'm doing.

Cheers,
Johannes

-- 
 Wo hattest Du das Beben nochmal GENAU vorhergesagt?
 Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ryan Hiebert]

2014-06-05 13:42 GMT-05:00 Johannes Bauer dfnsonfsdu...@gmx.de:

 On 05.06.2014 20:16, Paul Rubin wrote:
  Johannes Bauer dfnsonfsdu...@gmx.de writes:
  line = line[:-1]
  Which truncates the trailing \n of a textfile line.
 
  use line.rstrip() for that.

 rstrip has different functionality than what I'm doing.


How so? I was using line=line[:-1] for removing the trailing newline, and
just replaced it with rstrip('\n'). What are you doing differently?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Chris Angelico]

On Fri, Jun 6, 2014 at 4:52 AM, Ryan Hiebert r...@ryanhiebert.com wrote:
 2014-06-05 13:42 GMT-05:00 Johannes Bauer dfnsonfsdu...@gmx.de:

 On 05.06.2014 20:16, Paul Rubin wrote:
  Johannes Bauer dfnsonfsdu...@gmx.de writes:
  line = line[:-1]
  Which truncates the trailing \n of a textfile line.
 
  use line.rstrip() for that.

 rstrip has different functionality than what I'm doing.


 How so? I was using line=line[:-1] for removing the trailing newline, and
 just replaced it with rstrip('\n'). What are you doing differently?

 line = Hello,\nworld!\n\n
 line[:-1]
'Hello,\nworld!\n'
 line.rstrip('\n')
'Hello,\nworld!'

If it's guaranteed to end with exactly one newline, then and only then
will they be identical.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Paul Rubin]

Ryan Hiebert r...@ryanhiebert.com writes:
 How so? I was using line=line[:-1] for removing the trailing newline, and
 just replaced it with rstrip('\n'). What are you doing differently?

rstrip removes all the newlines off the end, whether there are zero or
multiple.  In perl the difference is chomp vs chop.  line=line[:-1]
removes one character, that might or might not be a newline.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ryan Hiebert]

On Thu, Jun 5, 2014 at 2:59 PM, Chris Angelico ros...@gmail.com wrote:

 On Fri, Jun 6, 2014 at 4:52 AM, Ryan Hiebert r...@ryanhiebert.com wrote:
  2014-06-05 13:42 GMT-05:00 Johannes Bauer dfnsonfsdu...@gmx.de:
 
  On 05.06.2014 20:16, Paul Rubin wrote:
   Johannes Bauer dfnsonfsdu...@gmx.de writes:
   line = line[:-1]
   Which truncates the trailing \n of a textfile line.
  
   use line.rstrip() for that.
 
  rstrip has different functionality than what I'm doing.
 
 
  How so? I was using line=line[:-1] for removing the trailing newline, and
  just replaced it with rstrip('\n'). What are you doing differently?

  line = Hello,\nworld!\n\n
  line[:-1]
 'Hello,\nworld!\n'
  line.rstrip('\n')
 'Hello,\nworld!'

 If it's guaranteed to end with exactly one newline, then and only then
 will they be identical.

  OK, that's not an issue for my case, and additionally I'm using the
open(_, 'U') file iterable, so I shouldn't see multiple trailing newlines
anyway.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ian Kelly]

On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin no.email@nospam.invalid wrote:
 Ryan Hiebert r...@ryanhiebert.com writes:
 How so? I was using line=line[:-1] for removing the trailing newline, and
 just replaced it with rstrip('\n'). What are you doing differently?

 rstrip removes all the newlines off the end, whether there are zero or
 multiple.  In perl the difference is chomp vs chop.  line=line[:-1]
 removes one character, that might or might not be a newline.

Given the description that the input string is a textfile line, if
it has multiple newlines then it's invalid.

Personally I tend toward rstrip('\r\n') so that I don't have to worry
about files with alternative line terminators.

If you want to be really picky about removing exactly one line
terminator, then this captures all the relatively modern variations:
re.sub('\r?\n$|\n?\r$', line, '', count=1)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Albert-Jan Roskam]

- Original Message -
 From: Ian Kelly ian.g.ke...@gmail.com
 To: Python python-list@python.org
 Cc: 
 Sent: Thursday, June 5, 2014 10:18 PM
 Subject: Re: Unicode and Python - how often do you index strings?
 
 On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin no.email@nospam.invalid 
 wrote:
  Ryan Hiebert r...@ryanhiebert.com writes:
  How so? I was using line=line[:-1] for removing the trailing newline, 
 and
  just replaced it with rstrip('\n'). What are you doing 
 differently?
 
  rstrip removes all the newlines off the end, whether there are zero or
  multiple.  In perl the difference is chomp vs chop.  line=line[:-1]
  removes one character, that might or might not be a newline.
 
 Given the description that the input string is a textfile line, if
 it has multiple newlines then it's invalid.
 
 Personally I tend toward rstrip('\r\n') so that I don't have 
 to worry
 about files with alternative line terminators.

I tend to use: s.rstrip(os.linesep)

 If you want to be really picky about removing exactly one line
 terminator, then this captures all the relatively modern variations:
 re.sub('\r?\n$|\n?\r$', line, '', count=1)

or perhaps: re.sub([^ \S]+$, , line)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Roy Smith]

In article mailman.10767.1402000635.18130.python-l...@python.org,
 Albert-Jan Roskam fo...@yahoo.com wrote:

 





- Original Message -
 From: Ian Kelly ian.g.ke...@gmail.com

  To: Python python-list@python.org
 Cc: 
 Sent: Thursday, June 5, 2014 
 10:18 PM
 Subject: Re: Unicode and Python - how often do you index strings?

  
 On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin no.email@nospam.invalid 
 
 wrote:
  Ryan Hiebert r...@ryanhiebert.com writes:
  How so? I was 
 using line=line[:-1] for removing the trailing newline, 
 and
  just 
 replaced it with rstrip('\n'). What are you doing 
 differently?
 
  
 rstrip removes all the newlines off the end, whether there are zero or
  
 multiple.? In perl the difference is chomp vs chop.? line=line[:-1]
  
 removes one character, that might or might not be a newline.
 
 Given the 
 description that the input string is a textfile line, if
 it has multiple 
 newlines then it's invalid.
 
 Personally I tend toward rstrip('\r\n') so 
 that I don't have 
 to worry
 about files with alternative line 
 terminators.

I tend to use: s.rstrip(os.linesep)

 If you want to be really 
 picky about removing exactly one line
 terminator, then this captures all 
 the relatively modern variations:
 re.sub('\r?\n$|\n?\r$', line, '', 
 count=1)

or perhaps: re.sub([^ \S]+$, , line)

Just for fun, I took a screen-shot of what this looks like in my 
newsreader.  URL below.  Looks like something chomped on unicode pretty 
hard :-)

http://www.panix.com/~roy/unicode.pdf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Rustom Mody]

On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote:
 Just for fun, I took a screen-shot of what this looks like in my 
 newsreader.  URL below.  Looks like something chomped on unicode pretty 
 hard :-)
  
 http://www.panix.com/~roy/unicode.pdf

Yii
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ned Deily]

In article 8681edf0-7a1f-4110-9f87-a8cd0988c...@googlegroups.com,
 Rustom Mody rustompm...@gmail.com wrote:

 On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote:
  Just for fun, I took a screen-shot of what this looks like in my 
  newsreader.  URL below.  Looks like something chomped on unicode pretty 
  hard :-)
   
  http://www.panix.com/~roy/unicode.pdf
 
 Yii

Roy is using MT-NewsWatcher as a client.  Because its codebase's origins 
are back in classic MacOS (= 9), it has its own *interesting* ways to 
deal with encodings.  BTW, don't upgrade to OS X 10.9 Mavericks if 
you're dependent on MT-NW; it finally stops working there because what 
was left of Open Transport support in OS X has finally been ripped out 
of 10.9.

-- 
 Ned Deily,
 n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ian Kelly]

On Thu, Jun 5, 2014 at 2:34 PM, Albert-Jan Roskam fo...@yahoo.com wrote:
 If you want to be really picky about removing exactly one line
 terminator, then this captures all the relatively modern variations:
 re.sub('\r?\n$|\n?\r$', line, '', count=1)

 or perhaps: re.sub([^ \S]+$, , line)

That will remove more than one terminator, plus tabs. Points for
including \f and \v though.

I suppose if we want to be absolutely correct, we should follow the
Unicode standard:
re.sub(r'\r?\n$|[\r\v\f\x85\u2028\u2029]$', line, '', count=1)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Roy Smith]

In article mailman.10781.1402009056.18130.python-l...@python.org,
 Ned Deily n...@acm.org wrote:

 In article 8681edf0-7a1f-4110-9f87-a8cd0988c...@googlegroups.com,
  Rustom Mody rustompm...@gmail.com wrote:
 
  On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote:
   Just for fun, I took a screen-shot of what this looks like in my 
   newsreader.  URL below.  Looks like something chomped on unicode pretty 
   hard :-)

   http://www.panix.com/~roy/unicode.pdf
  
  Yii
 
 Roy is using MT-NewsWatcher as a client.

Yes.  Except for the fact that it hasn't kept up with unicode, I find 
the U/I pretty much perfect.  I imagine at some point I'll be force to 
look elsewhere, but then again, netnews is pretty much dead.

 BTW, don't upgrade to OS X 10.9 Mavericks if you're dependent on 
 MT-NW; it finally stops working there because what was left of Open 
 Transport support in OS X has finally been ripped out of 10.9.

Hmmm, good to know.  I'm still on 10.7, and don't see any reason to 
move.  But, then again, you'd expect that from somebody who's still on 
Python 2.x, wouldn't you?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ned Deily]

In article roy-2a9d82.20100705062...@news.panix.com,
 Roy Smith r...@panix.com wrote:
 In article mailman.10781.1402009056.18130.python-l...@python.org,
  Ned Deily n...@acm.org wrote:
  Roy is using MT-NewsWatcher as a client.
 Yes.  Except for the fact that it hasn't kept up with unicode, I find 
 the U/I pretty much perfect.  I imagine at some point I'll be force to 
 look elsewhere, but then again, netnews is pretty much dead.

I agree about the U/I, although I'm sure a lot of that has to do with 
familiarity. However, netnews isn't dead, it has just morphed a bit.  A 
newsreader, like MT-NW, is great for following mailing lists like this 
(and most other Python-related lists) via gmane.org's bi-directional 
mailing list - NNTP gateways.  And for this list it's usually better to 
read the mailing list variant via gmane.org NNTP than the Usenet group 
variant via a traditional USENET NNTP server because there's less spam 
with the former.
 
  BTW, don't upgrade to OS X 10.9 Mavericks if you're dependent on 
  MT-NW; it finally stops working there because what was left of Open 
  Transport support in OS X has finally been ripped out of 10.9.
 Hmmm, good to know.  I'm still on 10.7, and don't see any reason to 
 move.  But, then again, you'd expect that from somebody who's still on 
 Python 2.x, wouldn't you?

Heh. Well, both 10.8 and 10.9 proved various improvements, both feature 
and performance, over 10.7.  Alas, Apple won't likely be supporting 10.7 
with security updates for as long as the PSF will be supporting 2.7.x.  
But, by then, you'll have had a chance to re-implement MT-NW in Python.

-- 
 Ned Deily,
 n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Gregory Ewing]

Chris Angelico wrote:

On Wed, Jun 4, 2014 at 11:18 AM, Roy Smith r...@panix.com wrote:


sarcasm style=regex-pedantUm, you mean cent(er|re), don't you?  The
pattern you wrote also matches centee and centrr./sarcasm


Maybe there's someone who spells it that way!


Come visit Pirate Island, the centrr of the universe!

--
Pegleg Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Mark Lawrence]

On 04/06/2014 01:39, Chris Angelico wrote:

A current discussion regarding Python's Unicode support centres (or
centers, depending on how close you are to the cent[er]{2} of the
universe) around one critical question: Is string indexing common?

Python strings can be indexed with integers to produce characters
(strings of length 1). They can also be iterated over from beginning
to end. Lots of operations can be built on either one of those two
primitives; the question is, how much can NOT be implemented
efficiently over iteration, and MUST use indexing? Theories are great,
but solid use-cases are better - ideally, examples from actual
production code (actual code optional).

I know the collective experience of python-list can't fail to bring up
a few solid examples here :)

Thanks in advance, all!!

ChrisA



Single characters quite often, iteration rarely if ever, slicing all the 
time, but does that last one count?


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Chris Angelico]

On Wed, Jun 4, 2014 at 6:22 PM, Mark Lawrence breamore...@yahoo.co.uk wrote:
 Single characters quite often, iteration rarely if ever, slicing all the
 time, but does that last one count?

Yes, slicing counts. What matters here is the potential impact of
internally representing strings as UTF-8 streams; when you ask for the
Nth character, it would have to scan from either the beginning or end
(more likely beginning) of the string and count, instead of doing what
CPython 3.3+ does and simply look up the header to find out the kind,
bit-shift the index by one less than that, and use that as a memory
location.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Peter Otten]

Mark Lawrence wrote:

 On 04/06/2014 01:39, Chris Angelico wrote:
 A current discussion regarding Python's Unicode support centres (or
 centers, depending on how close you are to the cent[er]{2} of the
 universe) around one critical question: Is string indexing common?

 Python strings can be indexed with integers to produce characters
 (strings of length 1). They can also be iterated over from beginning
 to end. Lots of operations can be built on either one of those two
 primitives; the question is, how much can NOT be implemented
 efficiently over iteration, and MUST use indexing? Theories are great,
 but solid use-cases are better - ideally, examples from actual
 production code (actual code optional).

 I know the collective experience of python-list can't fail to bring up
 a few solid examples here :)

 Thanks in advance, all!!

 ChrisA

 
 Single characters quite often, iteration rarely if ever, slicing all the
 time, but does that last one count?

The indices used for slicing typically don't come out of nowhere. A simple 
example would be

def strip_prefix(text, prefix):
if text.startswith(prefix):
text = text[len(prefix):] 
return text

If both prefix and text use UTF-8 internally the byte offset is already 
known. The question is then how we can preserve that information.

The first approach that comes to mind is an int subtype:

 for i, c in enumerate(123αλφα):
... print(i, byteoffset(i), c)
... 
0 0 1
1 1 2
2 2 3
3 3 α
4 5 λ
5 7 φ
6 9 α

This would work in the strip_prefix() example, but lead to data corruption 
in most other cases unless limited to a specific string -- in which case it 
would no longer work with strip_prefix().

So a new interface would be needed. My second try, an object with two byte 
offsets linked to a specific string:

 span(foobar).startswith(oob)
 p = span(foobar).startswith(foo)
 p.replace(baz)
'bazbar'
 p.before()
''
 p.after()
'bar'
 span(foo bar baz).find(bar).replace(spam)
'foo spam bar'

I have no idea if that could work out...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Chris Angelico]

On Wed, Jun 4, 2014 at 8:10 PM, Peter Otten __pete...@web.de wrote:
 The indices used for slicing typically don't come out of nowhere. A simple
 example would be

 def strip_prefix(text, prefix):
 if text.startswith(prefix):
 text = text[len(prefix):]
 return text

 If both prefix and text use UTF-8 internally the byte offset is already
 known. The question is then how we can preserve that information.

Almost completely useless. First off, it solves only the problem of
operating on the string at exactly some point where you just got an
index; and secondly, you don't always get that index from a string
method. Suppose, for instance, that you iterate over a string thus:

for i, ch in enumerate(string):
if ch=='{': start = i
elif ch=='}': return string[start:end+1]

Okay, so this could be done by searching, but for something more
complicated, I can imagine it being better to enumerate. (But I can
imagine is much weaker than Here's code that we use in production,
which is why I asked the question.)

Incidentally, the above code highlights the first problem too. With
direct indexing, you can ask for inclusive or exclusive slicing by
adding or subtracting one from the index. If you do that with a
byte-position-retaining special integer, you lose the byte position.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[alister]

On Tue, 03 Jun 2014 21:18:12 -0400, Roy Smith wrote:

 In article mailman.10656.1401842403.18130.python-l...@python.org,
  Chris Angelico ros...@gmail.com wrote:
 
 A current discussion regarding Python's Unicode support centres (or
 centers, depending on how close you are to the cent[er]{2} of the
 universe)
 
 sarcasm style=regex-pedantUm, you mean cent(er|re), don't you?  The
 pattern you wrote also matches centee and centrr./sarcasm

super pedant mode
The language is ENGLISH so the correct spelling is Centre regional 
variations my be common but they are incorrect
/super pedant mode
:-)
-- 
Prepare for tomorrow -- get ready.
-- Edith Keeler, The City On the Edge of Forever,
   stardate unknown
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[alister]

On Wed, 04 Jun 2014 18:48:29 +1200, Gregory Ewing wrote:

 Chris Angelico wrote:
 On Wed, Jun 4, 2014 at 11:18 AM, Roy Smith r...@panix.com wrote:
 
sarcasm style=regex-pedantUm, you mean cent(er|re), don't you?  The
pattern you wrote also matches centee and centrr./sarcasm
 
 Maybe there's someone who spells it that way!
 
 Come visit Pirate Island, the centrr of the universe!

that should be Cent-argh



-- 
I hope the ``Eurythmics'' practice birth control ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Rustom Mody]

On Wednesday, June 4, 2014 4:20:01 PM UTC+5:30, alister wrote:
 The language is ENGLISH so the correct spelling is Centre regional 
 variations my be common but they are incorrect

my?

O mee Oo my -- cockney (or Aussie) pedant??
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[wxjmfauth]

Le mercredi 4 juin 2014 02:39:54 UTC+2, Chris Angelico a écrit :
 A current discussion regarding Python's Unicode support centres (or
 
 centers, depending on how close you are to the cent[er]{2} of the
 
 universe) around one critical question: Is string indexing common?
 
 
 
 Python strings can be indexed with integers to produce characters
 
 (strings of length 1). They can also be iterated over from beginning
 
 to end. Lots of operations can be built on either one of those two
 
 primitives; the question is, how much can NOT be implemented
 
 efficiently over iteration, and MUST use indexing? Theories are great,
 
 but solid use-cases are better - ideally, examples from actual
 
 production code (actual code optional).
 
 
 
 I know the collective experience of python-list can't fail to bring up
 
 a few solid examples here :)
 
 
 
 Thanks in advance, all!!
 
 
 
 ChrisA

=

Like many, you are not understanding unicode because
you do not understand the coding of characters.

You do not understand the coding of the characters
because you do not understand the mathematics behind it.

You focussed on the wrong problem.

(All this stuff has been discussed, tested and worked on
20 (twenty) years ago.)

Sorry.

jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[alister]

On Wed, 04 Jun 2014 05:52:24 -0700, Rustom Mody wrote:

 On Wednesday, June 4, 2014 4:20:01 PM UTC+5:30, alister wrote:
 The language is ENGLISH so the correct spelling is Centre regional
 variations my be common but they are incorrect
 
 my?
 
 O mee Oo my -- cockney (or Aussie) pedant??

I made no claims about my typing or spelling being correct.
That post was actually quite good fro me usually my typing is worse.
 



-- 
The difference between genius and stupidity is that genius has its limits.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Michael Torrie]

On 06/04/2014 12:50 AM, wxjmfa...@gmail.com wrote:
 Like many, you are not understanding unicode because
 you do not understand the coding of characters.

If that is true, then I'm sure a well-written paragraph or two can set
him straight.  You continually berate people for not understanding
unicode, but you've posted nothing to explain anything, nor demonstrate
your own understanding.  That's one reason your posts are so frustrating
and considered trolling.  You never ever explain yourself, instead just
flailing around and muttering about folks not understanding unicode,
just as you've done here, true to form.

 
 You do not understand the coding of the characters
 because you do not understand the mathematics behind it.

flamebaiting here... FSR *is* UTF-32 internally, compresses off leading
zero bits during string creation.

 You focussed on the wrong problem.

Frankly it is you who is focused on the wrong problem, at least with
this particular thread.  I think you got distracted by the subject line.
 Chris's original post really has nothing to do with unicode at all.
He's simply asking for use cases for string indexing where O(1) is
desired or necessary.  Could be old Python 2 byte strings, or Python 3
unicode strings.  It does not matter.  Unicode is orthogonal to his
question.

Maybe his purpose in asking the question is to justify a fixed-length
encoding scheme (which is what FSR actually is), or maybe it is to
explore the costs of using a much slower, but more compact,
variable-length encoding scheme like UTF-8.  Particularly in the context
of low-memory applications where unicode support would be nice, but
memory is at a premium.  But either way, you got hung up on the wrong thing.

 
 (All this stuff has been discussed, tested and worked on
 20 (twenty) years ago.)
 
 Sorry.

As am I.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Dave Angel]

Chris Angelico ros...@gmail.com Wrote in message:
 On Wed, Jun 4, 2014 at 8:10 PM, Peter Otten __pete...@web.de wrote:
 The indices used for slicing typically don't come out of nowhere. A simple
 example would be

 def strip_prefix(text, prefix):
 if text.startswith(prefix):
 text = text[len(prefix):]
 return text

 If both prefix and text use UTF-8 internally the byte offset is already
 known. The question is then how we can preserve that information.
 
 Almost completely useless. First off, it solves only the problem of
 operating on the string at exactly some point where you just got an
 index; and secondly, you don't always get that index from a string
 method. Suppose, for instance, that you iterate over a string thus:
 
 for i, ch in enumerate(string):
 if ch=='{': start = i
 elif ch=='}': return string[start:end+1]
 
 Okay, so this could be done by searching, but for something more
 complicated, I can imagine it being better to enumerate. (But I can
 imagine is much weaker than Here's code that we use in production,
 which is why I asked the question.)
 
 Incidentally, the above code highlights the first problem too. With
 direct indexing, you can ask for inclusive or exclusive slicing by
 adding or subtracting one from the index. If you do that with a
 byte-position-retaining special integer, you lose the byte position.
 
 ChrisA
 

A string could have two extra fields in it that hold index and
 offset for the most recent substring reference.  Even though the
 string is immutable,  nothing prevents mutable elements that are
 externally visible only by performance measurement.
 

So a loop using a subscript of a string would tend to be faster
 even if written in a naive way.

It's also conceivable to build an array of such pairs in strings
 over a threshold size. So if you had a megabyte string, there
 might be 100 evenly spaced pairs, calculated when the string
 object is first created.

And naturally there can be flags indicating that the particular
 string is pure ASCII.

Clearly this breaks down if there are two alternating references
 at different offsets, but I think this would be exceeding
 rare.

-- 
DaveA

-- 
https://mail.python.org/mailman/listinfo/python-list

Unicode and Python - how often do you index strings?

[Chris Angelico]

A current discussion regarding Python's Unicode support centres (or
centers, depending on how close you are to the cent[er]{2} of the
universe) around one critical question: Is string indexing common?

Python strings can be indexed with integers to produce characters
(strings of length 1). They can also be iterated over from beginning
to end. Lots of operations can be built on either one of those two
primitives; the question is, how much can NOT be implemented
efficiently over iteration, and MUST use indexing? Theories are great,
but solid use-cases are better - ideally, examples from actual
production code (actual code optional).

I know the collective experience of python-list can't fail to bring up
a few solid examples here :)

Thanks in advance, all!!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Tim Chase]

On 2014-06-04 10:39, Chris Angelico wrote:
 A current discussion regarding Python's Unicode support centres (or
 centers, depending on how close you are to the cent[er]{2} of the
 universe) around one critical question: Is string indexing common?
 
 Python strings can be indexed with integers to produce characters
 (strings of length 1). They can also be iterated over from beginning
 to end. Lots of operations can be built on either one of those two
 primitives; the question is, how much can NOT be implemented
 efficiently over iteration, and MUST use indexing? Theories are
 great, but solid use-cases are better - ideally, examples from
 actual production code (actual code optional).

Many of my string-indexing uses revolve around a sliding window which
can be done with itertools[1], though I often just roll it as
something like

  n = 3
  for i in range(1 + len(s) - n):
do_something(s[i:i+n])

So that could be supplanted by the SO iterator linked below.

The other use big case I have from production code involves a
column-offset delimited file where the headers have a row of
underscores under them delimiting the field widths, so it looks
something like

  EmpID NameCost Center
  - --- -
  314159Longstocking, Pippi RJ45
  265358Davis, MilesJA22
  979328Bell, Alexander RJ15

I then take row 2 and use it to make a mapping of header-name to a
slice-object for slicing the subsequent strings:

  import re
  r = re.compile('-+') # a sequence of 1+ dashes
  f = file(data.txt)
  headers = next(f)
  lines = next(f)
  header_map = dict((
  headers[i.start():i.end()].strip().upper(),
  slice(i.start(), i.end())
  )
for i in r.finditer(lines)
)
  for row in f:
print(EmpID = %s % row[header_map[EMPID]].strip())
print(Name = %s % row[header_map[NAME]].strip())
# ...

which I presume uses string indexing under the hood.

Perhaps there's a better way of doing that, but it's what I currently
use to process these large-ish files (largest max out at 10-20MB each)

There might be other use-cases I've done, but those two leap to mind.

-tkc


[1]
http://stackoverflow.com/questions/6822725/rolling-or-sliding-window-iterator-in-python




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Roy Smith]

In article mailman.10656.1401842403.18130.python-l...@python.org,
 Chris Angelico ros...@gmail.com wrote:

 A current discussion regarding Python's Unicode support centres (or
 centers, depending on how close you are to the cent[er]{2} of the
 universe)

sarcasm style=regex-pedantUm, you mean cent(er|re), don't you?  The 
pattern you wrote also matches centee and centrr./sarcasm

 around one critical question: Is string indexing common?

Not in our code.  I've got 80008 non-blank lines of Python (2.7) source 
handy.  I tried a few heuristics to find patterns which might be string 
indexing.

$ find . -name '*.py' | xargs egrep '\[[^]][0-9]+\]'

and then looked them over manually.  I see this pattern a bunch of times 
(in a single-use script):

data['shard_key'] = hashlib.md5(str(id)).hexdigest()[:4]  

We do this once:

if tz_offset[0] == '-':

We do this somewhere in some command-line parsing:

process_match = args.process[:15]

There's this little gem:

return [dedup(x[1:-1].lower()) for x in 
re.findall('(\[[^\]\[]+\]|\([^\)\(]+\))',title)]

It appears I wrote this one, but I don't remember exactly what I had in 
mind at the time...

withhyphen = number if '-' in number else (number[:-2] + '-' + 
number[-2:]) # big assumption here

Anyway, there's a bunch more, but the bottom line is that in our code, 
indexing into a string (at least explicitly in application source code) 
is a pretty rare thing.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Ethan Furman]

On 06/03/2014 05:39 PM, Chris Angelico wrote:


A current discussion regarding Python's Unicode support centres (or
centers, depending on how close you are to the cent[er]{2} of the
universe) around one critical question: Is string indexing common?


I use it quite a bit, but the strings are usually quite small (well under 100 characters) so an implementation that 
wasn't O(1) would not hurt me much.


--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Chris Angelico]

On Wed, Jun 4, 2014 at 11:18 AM, Roy Smith r...@panix.com wrote:
 In article mailman.10656.1401842403.18130.python-l...@python.org,
  Chris Angelico ros...@gmail.com wrote:

 A current discussion regarding Python's Unicode support centres (or
 centers, depending on how close you are to the cent[er]{2} of the
 universe)

 sarcasm style=regex-pedantUm, you mean cent(er|re), don't you?  The
 pattern you wrote also matches centee and centrr./sarcasm

Maybe there's someone who spells it that way! Let's not be excluding
people. That'd be rude.

 around one critical question: Is string indexing common?

 Not in our code.  I've got 80008 non-blank lines of Python (2.7) source
 handy.  I tried a few heuristics to find patterns which might be string
 indexing.

 $ find . -name '*.py' | xargs egrep '\[[^]][0-9]+\]'

 and then looked them over manually.  I see this pattern a bunch of times
 (in a single-use script):

 data['shard_key'] = hashlib.md5(str(id)).hexdigest()[:4]

Slicing is a form of indexing too, although in this case (slicing from
the front) it could be implemented on top of UTF-8 without much
problem.

 withhyphen = number if '-' in number else (number[:-2] + '-' +
 number[-2:]) # big assumption here

This *definitely* counts; if strings were represented internally in
UTF-8, this would involve two scans (although a smart implementation
could probably count backward rather than forward). By the way, any
time you slice up to the third from the end, you win two extra awesome
points, just for putting [:-3] into your code and having it mean
something. But I digress.

 Anyway, there's a bunch more, but the bottom line is that in our code,
 indexing into a string (at least explicitly in application source code)
 is a pretty rare thing.

Thanks. Of course, the pattern you searched for is looking only for
literals; it's a bit harder to find cases where the index (or slice
position) comes from a variable or expression, and those situations
are also rather harder to optimize (the MD5 prefix is clearly better
scanned from the front, the number tail is clearly better scanned from
the back - but with a variable?).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Chris Angelico]

On Wed, Jun 4, 2014 at 11:11 AM, Tim Chase
python.l...@tim.thechases.com wrote:
 I then take row 2 and use it to make a mapping of header-name to a
 slice-object for slicing the subsequent strings:

   slice(i.start(), i.end())

 print(EmpID = %s % row[header_map[EMPID]].strip())
 print(Name = %s % row[header_map[NAME]].strip())

 which I presume uses string indexing under the hood.

Yes, it's definitely going to be indexing. If strings were represented
internally in UTF-8, each of those calls would need to scan from the
beginning of the string, counting and discarding characters until it
finds the place to start, then counting and retaining characters until
it finds the place to stop. Definite example of what I'm looking for,
thanks!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode and Python - how often do you index strings?

[Tim Chase]

On 2014-06-04 12:16, Chris Angelico wrote:
 On Wed, Jun 4, 2014 at 11:11 AM, Tim Chase
 python.l...@tim.thechases.com wrote:
  I then take row 2 and use it to make a mapping of header-name to a
  slice-object for slicing the subsequent strings:
 
slice(i.start(), i.end())
 
  print(EmpID = %s % row[header_map[EMPID]].strip())
  print(Name = %s % row[header_map[NAME]].strip())
 
  which I presume uses string indexing under the hood.
 
 Yes, it's definitely going to be indexing. If strings were
 represented internally in UTF-8, each of those calls would need to
 scan from the beginning of the string, counting and discarding
 characters until it finds the place to start, then counting and
 retaining characters until it finds the place to stop. Definite
 example of what I'm looking for, thanks!

For what it's worth, most of the lines in each file are under ~2k, so
even O(N) or O(log N) indexing wouldn't be grievous.  Noticeable, but
not grievous.

Glad my example could give you some fodder.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21003] how to do batch processing using python

[Eric V. Smith]

Eric V. Smith added the comment:

This bug tracker is for reporting bugs in python. For questions on using 
python, please use the python-list mailing list: 
https://mail.python.org/mailman/listinfo/python-list

--
nosy: +eric.smith
resolution:  - invalid
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21003
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21003] how to do batch processing using python

[Pramod Jadhav]

New submission from Pramod Jadhav:

i am new in python.i have created twitter data anlysis script.how to run this 
script dailly using python script.

--
messages: 214327
nosy: pramod.jadhav
priority: normal
severity: normal
status: open
title: how to do batch processing using python

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21003
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Survey - how much do you rely on Free/Open Source Software?

[Tony]

Hi,

I'm conducting a survey that aims to measure the importance that Open 
Source/Free Software has to people and organizations around the world.
Answering is very quick (mostly one click per answer)
Please answer it by clicking the link below:
https://docs.google.com/forms/d/1bY5KQgsuPeGMwRoTY2DWU6q-210l7RQ1kcVLRUfLlsA/viewform

After answering, you'll be able to see the results of all answers given so far.

Please help us to obtain accurate data by forwarding this survey to many 
different types of people (meaning, beyound the geek circles!)

NOTE: All answers are optional. Answering only one or two questions is 
perfectly ok.
-- 
http://mail.python.org/mailman/listinfo/python-list

How to do a Lispy-esque read?

[zeta . convex]

Suppose I want to read an object from some stream. How do I do it?

For example, if the input stream contained the text:
[1, # python should ignore this comment
2]

and I do a read on it, I should obtain the result
[1, 2]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Lispy-esque read?

[Barrett Lewis]

 For example, if the input stream contained the text:
 [1, # python should ignore this comment
 2]

 and I do a read on it, I should obtain the result
 [1, 2]
 --


I don't know much about lisp but given that input and the desired output
you can write functions like the following

def strtolist(l):
if l.startswith('['):
l = l[1:]
if l.endswith(']'):
l = l[:-1]

# treat newlines as if they are commas so we can easily split
l = l.replace('\n', ',').split(',')
# list comprehension
# strip to remove whitespace and aggregate all elements without the
comment
# you can further refine this to create ints by using the int() method
return [x for x in l if not x.strip().startswith('#')]

you would have to use input() to read the input or open a file. Either way
you can pass that value to strtolist and it will convert it to a list of
strings.
However, this implementation is not very robust and doesn't cover a lot of
edge cases (more a demo of how it might be done, I don't know your python
experience so forgive me if this is all self evident).

The real solution that I would use would be to use the json module.
Docs: http://docs.python.org/3.3/library/json.html
It allows you to take a string and turn it into a native dict or list in
python. The great part is that it is fairly robust and it has a simple
interface. You could take input() and send it to loads and it will return
an array. The downside is json doesn't allow comments.

If you really want comments, you could look into some of the yaml
libraries, a quick google search shoes PyYaml is out there. I however don't
have any knowledge of that module or other yaml modules as I find json is
enough for anything I've ever attempted.

I hope that helps
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Lispy-esque read?

[Benjamin Kaplan]

There is no read in a stream until it's a valid literal function as
far as I know, but ast.literal_eval will turn your string into an
object.

On Mon, Apr 8, 2013 at 12:45 AM,  zeta.con...@gmail.com wrote:
 Suppose I want to read an object from some stream. How do I do it?

 For example, if the input stream contained the text:
 [1, # python should ignore this comment
 2]

 and I do a read on it, I should obtain the result
 [1, 2]
 --
 http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Lispy-esque read?

[Arnaud Delobelle]

On 8 April 2013 08:45,  zeta.con...@gmail.com wrote:
 Suppose I want to read an object from some stream. How do I do it?

 For example, if the input stream contained the text:
 [1, # python should ignore this comment
 2]

 and I do a read on it, I should obtain the result
 [1, 2]

You might be interested in code.compile_command()
(http://docs.python.org/2/library/code.html)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

How to do this?

[Ana Dionísio]

So I have this script:


from numpy import array

vt=[0]*20
vt = array(vt, dtype=dict)

for t in range(20):
if t == 4:
vt[t]=1

else:
vt[t]=0


And have this output:

[0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

What I need is when t == 4 I need to put 1 in that position and in the next 3, 
for example the following output:

[0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0]

Do you have any suggestions?


I'm sorry if I can't explain this in a better way, English is not my first 
language

Thank you
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do this?

[Vincent Vande Vyvre]

Le 01/04/13 12:14, Ana Dionísio a écrit :
 for t in range(20):
 if t == 4:
 vt[t]=1
Sets only the needed values:

for t in range(4, 8):
vt[t]=1



-- 
Vincent V.V.
Oqapy https://launchpad.net/oqapy . Qarte
https://launchpad.net/qarte . PaQager https://launchpad.net/paqager
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do this?

[Ana Dionísio]

Nice! Thank you!

And if I need something like this?

[0 0 0 0 0.17 0.17 0.17 0.17 0 0 0 0.17 0.17 0.17 0 0 0 0 0 0 0]

How can I do this?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do this?

[Peter Otten]

Ana Dionísio wrote:

 Nice! Thank you!
 
 And if I need something like this?
 
 [0 0 0 0 0.17 0.17 0.17 0.17 0 0 0 0.17 0.17 0.17 0 0 0 0 0 0 0]
 
 How can I do this?

With vanilla Python:

 vt = [0] * 20
 for i, v in enumerate(vt):
... if 4 = i  8 or 13 = i  16:
... vt[i] = .17
... 
 vt
[0, 0, 0, 0, 0.17, 0.17, 0.17, 0.17, 0, 0, 0, 0, 0, 0.17, 0.17, 0.17, 0, 0, 
0, 0]

With vanilla Python using slices:

 vt = [0] * 20
 vt[4:8] = [.17]*4
 vt[13:16] = [.17]*3
 vt
[0, 0, 0, 0, 0.17, 0.17, 0.17, 0.17, 0, 0, 0, 0, 0, 0.17, 0.17, 0.17, 0, 0, 
0, 0]

With numpy:

 vt = numpy.zeros(20)
 vt[4:8] = vt[13:16] = .17
 vt
array([ 0.  ,  0.  ,  0.  ,  0.  ,  0.17,  0.17,  0.17,  0.17,  0.  ,
0.  ,  0.  ,  0.  ,  0.  ,  0.17,  0.17,  0.17,  0.  ,  0.  ,
0.  ,  0.  ])


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do this?

[Dave Angel]

On 04/01/2013 07:08 AM, Ana Dionísio wrote:

[0 0 0 0 0.17 0.17 0.17 0.17 0 0 0 0.17 0.17 0.17 0 0 0 0 0 0 0]



I'd do
res = [0 0 0 0 0.17 0.17 0.17 0.17 0 0 0 0.17 0.17 0.17 0 0 0 0 0 0 0]

Unless there's a pattern you're trying to accomplish (like maybe next 
time you want to do it for a list of two million items), there's 
little point in writing code to do what you've already predetermined. 
And if there is a pattern, you'd probably better let us know.


But if it's for fun,



vt = [0] * 21
for index, val in enumerate(vt):
if 3index8 or 10index14:
vt[index] = 0.17


or


vt = [0] * 21
for index, val in enumerate(vt):
if index in (4,5,6,7,11,12,13):
vt[index] = 0.17


--
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to do this?

[Mark Lawrence]

On 01/04/2013 11:14, Ana Dionísio wrote:

So I have this script:


from numpy import array


Are you aware that this overrides the Python builtin array?  It's 
usually but not always better to do this


import numpy as np
vt = np.array(vt, dtype=dict)



vt=[0]*20
vt = array(vt, dtype=dict)

for t in range(20):
 if t == 4:
 vt[t]=1

 else:
 vt[t]=0


Why the loop, you've already initialise the array to zeroes?




And have this output:

[0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

What I need is when t == 4 I need to put 1 in that position and in the next 3, 
for example the following output:

[0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0]

Do you have any suggestions?


Use slicing.
vt[4:8] = 1




I'm sorry if I can't explain this in a better way, English is not my first 
language

Thank you



--
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.


Mark Lawrence

--
http://mail.python.org/mailman/listinfo/python-list

Re: SQLAlchemy: How to do Table Reflection and MySQL?

[Nick Sabalausky]

On Mon, 22 Oct 2012 14:35:23 -0700 (PDT)
darnold darnold992...@yahoo.com wrote:
 
 i'm not brave enough to dig too deeply into SQLAlchemy, but maybe this
 will help? :
 
 http://kashififtikhar.blogspot.com/2010/07/using-sqlalchemy-reflection-with-pylons.html
 
 that came up from googling sqlalchemy table reflection tutorial.

Thanks, your view of Google seems to be far better tailored for Python
than mine is, that doesn't come up for me anywhere on the first five
pages of results for that query.

Unfortunately the info on that page doesn't seem to work for me:

--
from sqlalchemy import *
from sqlalchemy.orm import sessionmaker

engine = create_engine(my connection string)
meta = MetaData()
meta.bind = engine
meta.reflect()

Session = sessionmaker(bind=engine)
session = Session()

res = session.query(user).filter(user.name==bert).first()
print res.name
--

That just gives me:

NameError: name 'user' is not defined

(And yes, the code given on that page to print out the table info
*does* indicate a table named 'user' was found.)

I also tried this which also fails:

res =
session.query(meta.tables[user]).filter(meta.tables[user].name==bert).first()

sqlalchemy.exc.ArgumentError: filter() argument must be of type
sqlalchemy.sql.ClauseElement or string

The page you linked to appears to get around the matter by manually
setting up tables filled with the reflected info, but that seems to
defeat much of the point for me. I may as well just set up the tables
manually without the reflection, which is what I'll probably do.

Maybe I just misunderstood what was meant in the SQLAlchemy docs here?:

but note that SA can also “import” whole sets of Table objects
automatically from an existing database (this process is called table
reflection).  --
http://docs.sqlalchemy.org/en/rel_0_7/core/tutorial.html

It said that but then didn't say how and didn't link to any info on how.

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: SQLAlchemy: How to do Table Reflection and MySQL?

[Prasad, Ramit]

Nick Sabalausky wrote:
 On Mon, 22 Oct 2012 14:35:23 -0700 (PDT)
 darnold darnold992...@yahoo.com wrote:
 
  i'm not brave enough to dig too deeply into SQLAlchemy, but maybe this
  will help? :
 
  http://kashififtikhar.blogspot.com/2010/07/using-sqlalchemy-reflection-with-pylons.html
 
  that came up from googling sqlalchemy table reflection tutorial.
 
 Thanks, your view of Google seems to be far better tailored for Python
 than mine is, that doesn't come up for me anywhere on the first five
 pages of results for that query.
 
 Unfortunately the info on that page doesn't seem to work for me:
 
 --
 from sqlalchemy import *
 from sqlalchemy.orm import sessionmaker
 
 engine = create_engine(my connection string)
 meta = MetaData()
 meta.bind = engine
 meta.reflect()
 
 Session = sessionmaker(bind=engine)
 session = Session()
 
 res = session.query(user).filter(user.name==bert).first()
 print res.name
 --
 
 That just gives me:
 
 NameError: name 'user' is not defined
 
 (And yes, the code given on that page to print out the table info
 *does* indicate a table named 'user' was found.)

This does not seem to be a SQLAlchemy problem. Instead it seems
there is not a variable called `name`. If you define 
the appropriate user it does it work? (From that page it seems 
like user should be a blank class like the following).

class user(object): pass

 
 I also tried this which also fails:
 
 res =
 session.query(meta.tables[user]).filter(meta.tables[user].name==bert).first()
 
 sqlalchemy.exc.ArgumentError: filter() argument must be of type
 sqlalchemy.sql.ClauseElement or string
 
 The page you linked to appears to get around the matter by manually
 setting up tables filled with the reflected info, but that seems to
 defeat much of the point for me. I may as well just set up the tables
 manually without the reflection, which is what I'll probably do.
 
 Maybe I just misunderstood what was meant in the SQLAlchemy docs here?:
 
 but note that SA can also “import” whole sets of Table objects
 automatically from an existing database (this process is called table
 reflection).  --
 http://docs.sqlalchemy.org/en/rel_0_7/core/tutorial.html
 
 It said that but then didn't say how and didn't link to any info on how.

Ramit Prasad



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list