Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[Avi Gross]

Since this topic is not focused on efficiency, why exactly does it matter if
your function should check if a file exists and then avoid a collision?

What you are describing sounds a bit like a hash function. In many cases,
when you perform the computations you may find the slot you hash to is
already occupied. A common solution is to check and if needed make some
adjustment like adding 1 or using a second hash function or changing the
thing being hashed in a deterministic way like changing "key" to "keykey"
and even "keykeykey" before hashing till you get an empty slot. Note in
Python, you can simply use a dictionary and have no idea how the hash is
done.

Since you are describing a need for a permanent file, many suggested ways of
making a unique file name can repeat and thus not be unique. The process ID
wraps around and maybe restarts every time the machine boots, I suspect.
Dates repeat unless they include the year as in 2018101903 to the
nearest second. If you are concerned about two files being created in the
same second, there is a trivial solution. Before or after creating the file,
sleep for a time period like a second so any "second" saved file is
guaranteed to come at least a second later, in the above scheme. 

I heard a concern about what happens if you just use sequence numbers as in
file3 then file4 if the user later deletes some. The concern was
subtle about what happens if a file is later deleted and your algorithm
later will search for the first available empty slot. To each their own but
it strikes me as fairly easy in Python to not check EVERY filename but skip
past the last one.

You have functions that return a list of filenames in a directory as a list
of strings. In my example, you could take the list and replace "file" with
nothing to get [ "1", "2", "4", ... ]

Then in memory, efficiently, without opening a single file, you can sort the
strings, pop off the last one, convert it to an int, add one, make a new
file name like "file00111" that is guaranteed not to exist unless some other
program sneaks it in, and continue.

There are lots of ideas you can try. This would only need to happen perhaps
once per game and you can encapsulate the logic in your own function so all
your code will say is something like:
save_game_info(...)

Of course if someone else had already come up with similar functionality,
use it. 

Oddly, you may not be aware that your method indicates thinking too much
inside the box. Instead of saving files on your own, consider creating a
data structure and letting known modules save it in the file system and
retrieve it later. There are packages like pickle and shelve.

https://docs.python.org/3/library/pickle.html
https://docs.python.org/3/library/shelve.html

So if you load an existing object, such as a dictionary or list before a
game starts, and after each game extend the object and then save it to disk
this way, then you don't need to deal with details as long as the program
knows where it is stored. The person running the program may not trivially
be able to access the data and is not likely to delete easily. The actual
file names may depend on what the package does and you may find other such
packages better fit your needs and let you easily save all kinds of info
like scores and time elapsed and number of moves as you are saving entire
data structures you create and control.

Just some thoughts.

Avi

-Original Message-
From: Tutor  On Behalf Of boB
Stepp
Sent: Monday, October 22, 2018 8:31 PM
To: tutor 
Subject: Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be
used to create *permanent* uniquely named files?

On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann  wrote:
>
> On 10/22/18 8:24 AM, boB Stepp wrote:
> > Forwarding to the Tutor list.  Herr Maier offers a good idea that 
> > would take away much of a remaining issue -- the name "Temporary".  
> > I need to look into the functools library to see what "partial" does.
>
>
> if you really don't care what the file is called because you will 
> maintain a map which leads you to the filename, you might as well use 
> a uuid.

Wouldn't I have to write a check to ensure such a named file (However
unlikely that would be.) did not exist before creating it?  And if yes,
would not that get into the same potential security issue that cause
tempfile.mktemp() to be deprecated?


--
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Installing python

[Avi Gross]

Alan,

Many people have machines with a single login that is shared with nobody. 

I install lots of software on multiple machines that only I use and have
often wondered what the purpose for me was to install for everyone. On some
machines you then need to log in as whatever variant of superuser you have
to be if you want to be able to write in restricted directories or make
changes to the operating system or global configuration files.

There can be tradeoffs in how long it takes to find a program on your PATH
or PYTHONPATH or other such things. 

But, of course, if you have many users who might otherwise install their own
personal copies of software on the same machine, especially one where many
people may log in at once, then it may make more sense to share.


But I wonder about the many modern languages that can be extended by also
installing one of thousands of packages above and beyond the standard ones
routinely in place. Do you want people to replace earlier versions for
everyone or be able to install them in a global area at all? Among other
considerations, if they have write access they might be able to modify
something others use, like numpy, which is a serious security risk when you
can sneak in a call to pop open a shell that reformats the hard disk or
whatever.

Avi

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Friday, November 2, 2018 9:16 PM
To: tutor@python.org
Subject: Re: [Tutor] Installing python

On 02/11/2018 23:44, Mats Wichmann wrote:

> that's actually exactly the right path for a Python 3.5+ if you chose 
> a  "user install", which you usually should.

Ah, I always install for all users. That's why I've not seen that path
presumably?

But why *should* you install for a single user? I only ever do that for
programs that I'm still writing and don't want other users to accidentally
start. But I want everyone to be able to run my Python programs so I always
install for everyone.

> the "modern" answer for Windows is supposed to be the Python Launcher,

I keep forgetting that thing exists. I really must spend some time getting
to grips with it...


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] TESTING

[Avi Gross]

I am not totally up on the exact purposes of this group but have noted how
many of the questions asked have been frustrating.

I wonder if it would make any sense to provide a template people might fill
out to suit their needs.

An example might be people who want you to pretty much do their homework for
them from beginning to end. The purpose of a tutor is to guide someone in
learning how to do something for themselves. If they ask for an idea on an
algorithm in English or pseudocode on how to solve the problem, hey might be
given hints but not actual python code. If they wrote a snippet of code on
their own and wonder how to test it, again, there can be advice. If they ask
for the names of packages that might be useful, ditto. If they encounter an
error and can isolate it to a PART of the code, others might be willing to
point out that $ is not valid in Python user names (other than in regular
expressions or strings) and they should consider writing in a language like
PERL or a shell command language that uses it, albeit for another purpose.

But to do it ALL for someone generally is not the role of a tutor in the
real world.

On the other hand, when someone discusses how to optimize their code, and
shows the ways they already tried, it may be valid to present them with a
complete program that illustrates it assuming this is not a homework problem
but a more advanced person that may be interested.

But, to make this message more meaningful, I have a question. In my reading
I see Python has been bifurcated into often incompatible releases we call 2X
and 3X. Many times if you want something to be portable, you have to forgo
using some of the niftier new features. But for a language to evolve, you
sometimes need to bite the bullet. I was one of the earliest uses of C++
when I was at Bell Labs and the decision was then made to not just call it C
(or D or add-one-to-C or even ++C) but make a clean break with C and give it
a new name. The aforementioned PERL also split into incompatible languages. 

So, does anyone see any further breaks in Python such as when 4.0 comes
along or will we have some backward compatibility for a while? As I see it,
one of the earlier guiding principles of Python was to have mainly one
obvious way to do something and in my reading, I often see MANY ways to do
it. Much of this is caused by adding new features while retaining the old.
Yes, I also see places where a new feature is accompanied by deprecating an
older one and eventually removing it. But that too causes incompatibilities
especially when you use a package written by someone and then abandoned. I
recently ran into a problem with an R script I wrote a year or so ago that I
needed to modify in a minor way (re-scale the limits of the Y axis on a
graph) which I might have done in a few minutes. But the package I used to
do part of the work had been modified and the function just printed out that
I should use a replacement function which did not work. Reloading old
versions of the package still did not work because it had many dependencies
and getting them all correct was more work than rewriting my code after
reading what their code used to run.

I wonder if others have written tools (such as in frozen programs) that
gather up an exact snapshot of the environment needed when you submit a
paper so it can be replicated!

Avi

-Original Message-
From: Tutor  On Behalf Of
Mats Wichmann
Sent: Thursday, November 1, 2018 1:54 PM
To: tutor@python.org
Subject: Re: [Tutor] TESTING

On 11/1/18 11:24 AM, Mario Radomirovic wrote:
> All good
> 
> On Thu, 1 Nov 2018 6:17 pm Alan Gauld via Tutor  wrote:
> 
>> On 01/11/2018 14:01, richard mwenya via Tutor wrote:
>>> Hello.
>>> Everyone is quiet or is it my email thats not working?
>>
>> Just quiet, I've seen nothing in the moderators queue for 5 days. But 
>> that happens sometimes.
>>
>> I guess it just means nobody is having any Python issues right now.

or the ones who do gave up on us because people say they won't solve
homework problems :)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to find optimisations for code

[Avi Gross]

There are many ways to do some things. I present a fairly straightforward
method for consideration. 

 

Synopsis. Use two dictionaries to keep track of how many times a letter can
be found in the source word and the target word.

 

Loop over the target letters and as soon as you find a letter that needs
more copies of that letter than are available in the source, return False.

 

Otherwise, return True.

 

Slight optimization, return False immediately if the target is too long.

 

Note: Remove the print statements used to show what it does when trying for
efficiency.

 

Note I do not claim this is fast. But for longer strings this requires one
pass over each string to set the dictionary counters and often less than one
pass over the target string to determine Truth. I can visualize many
variants but efficiency of each may vary with the data, version of Python
and so on. 

 

Python CODE using version 3.7.0 :

 

"""

Function to check if string str_target is a strict subset

of str_source - but with multiple copies of a letter being unique

in what is thus not a set. In English, can the letters in str_target

be found in the required quantities with possibly some left over

when examining str_source. For example, 'level' has two l's and two e's

and can be made from 'multilevel' but not from 'evil' which has the needed

e v and l but not two copies.

"""

 

def sub_word(str_source, str_target):

"""

Return True/False if letters in source are enough to make target

Method: make dictionaries of number of letters in each. See if enough.

"""

 

# Check for trivial False condition: word too long

if len(str_target) > len(str_source) : return False

 

# Initialize empty dictionaries to hold leter counts.

source = {}

target = {}

 

# Count letters in source thgen target, initializing

# to zero if not yet present.

for letter in str_source.lower():

   source[letter] = source.get(letter, 0) + 1

 

   

for letter in str_target.lower():

   target[letter] = target.get(letter, 0) + 1

 

# During bebug phase, show what the dictionaries contain

print("SOURCE dict:", source)

print("TARGET dict:", target)

 

# Iterate over letters in target and return False

# if the number of instances needed exceeds what

# can be found in the source.

for key in target:

if target[key] > source.get(key, 0) : return False

 

# if you reached here, return True as all letters needed

# have been found.

return True

 

Examples of use:

 

>>> sub_word("multilevel", "level")

SOURCE dict: {'m': 1, 'u': 1, 'l': 3, 't': 1, 'i': 1, 'e': 2, 'v': 1}

TARGET dict: {'l': 2, 'e': 2, 'v': 1}

True

 

>>> sub_word("shorT", "MuchTooLong")

False

>>> sub_word("supercalifragilisticexpialidocious", "california")

SOURCE dict: {'s': 3, 'u': 2, 'p': 2, 'e': 2, 'r': 2, 'c': 3, 'a': 3, 'l':
3, 'i': 7, 'f': 1, 'g': 1, 't': 1, 'x': 1, 'd': 1, 'o': 2}

TARGET dict: {'c': 1, 'a': 2, 'l': 1, 'i': 2, 'f': 1, 'o': 1, 'r': 1, 'n':
1}

False

 

>>> sub_word("supercalifragilisticexpialidocious", "fragile")

SOURCE dict: {'s': 3, 'u': 2, 'p': 2, 'e': 2, 'r': 2, 'c': 3, 'a': 3, 'l':
3, 'i': 7, 'f': 1, 'g': 1, 't': 1, 'x': 1, 'd': 1, 'o': 2}

TARGET dict: {'f': 1, 'r': 1, 'a': 1, 'g': 1, 'i': 1, 'l': 1, 'e': 1}

True

 

>>> sub_word("devil", "vile")

SOURCE dict: {'d': 1, 'e': 1, 'v': 1, 'i': 1, 'l': 1}

TARGET dict: {'v': 1, 'i': 1, 'l': 1, 'e': 1}

True

 

 

-Original Message-
From: Tutor  On Behalf Of
Peter Otten
Sent: Saturday, October 20, 2018 3:02 AM
To: tutor@python.org
Subject: Re: [Tutor] How to find optimisations for code

 

Steven D'Aprano wrote:

 

> We don't need to check that the individual letters are the same, 

> because checking the counts will suffice. If they are not the same, 

> one string will have (let's say) two A's while the other will have 

> none, and the counts will be different.

 

Another great optimisation is solving the actual problem ;)

 

The OP isn't looking for anagrams, he wants to know if string2 can be spelt
with the characters in string1:

 

abba, abacus --> True (don't care about the extra b, c, u, s) abba, baab
--> True (don't care about character order) abba, bab --> False (bab is
missing an a)

 

 

___

Tutor maillist  -    Tutor@python.org

To unsubscribe or change subscription options:

 
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Avi Gross]

inivasan
Sent: Wednesday, November 7, 2018 4:22 AM
To: tutor@python.org; python-l...@python.org
Subject: Re: [Tutor] SyntaxError: can't assign to literal while using
""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess
module in Python

After changing the line to *"cmd = "blkid -o export %s | grep \'TYPE\' | cut
-d\"=\" -f3" % fs"*, Now I dont see the error "SyntaxError: can't assign to
literal"
This is not returning exactly "*vfat*" instead of this, it is returning as
"*
/dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"* "

Could you please help me  as it seems to be like grep and cut commands are
not working in the above line ie., on *cmd = "blkid -o export %s | grep
\'TYPE\' | cut -d\"=\" -f3" % fs*?

On Wed, Nov 7, 2018 at 9:41 AM Avi Gross  wrote:

> I may be missing something but it looks like the embedded double 
> quotes may be a problem in this:
>
> cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % ...
>
> if you use single quotes as in:
>
> cut -d"="
>
> becomes
>
> cut -d'='
>
> or escape the double quotes with \" and so on ...
>
> The above seems to be seen as:
>
> cmd=ALPHA=BETA % ...
>
> where ALPHA="blkid -o export %s | grep 'TYPE' | cut -d"
> and BETA=" -f3"
>
> Other issues may also be there.
> -Original Message-
> From: Tutor  On Behalf 
> Of Alan Gauld via Tutor
> Sent: Tuesday, November 6, 2018 7:37 PM
> To: tutor@python.org
> Cc: python-...@python.org
> Subject: Re: [Tutor] SyntaxError: can't assign to literal while using 
> ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using 
> subprocess module in Python
>
> On 06/11/2018 18:07, srinivasan wrote:
>
> > bash command in python using subprocess module, I ma seeing the below
> > *cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" %
> > (fs)*
>
> In general you should try to do as little as possible using bash and 
> subprocess. Especially try to avoid long pipelines since you are 
> starting a new OS process for every element in the pipeline. That 
> means, in your case, you are running 4 processes to get your result - 
> Python, blkid, grep and cut
>
> Python is designed to do much of what the shell command can do almost 
> as easily and much more efficiently (no new processes being started).
>
> In this case just execute the blkid bit in bash because its too 
> difficult to replicate simply in Python. Then use Python to search for 
> the TYPE lines and slice them to size.
>
> That will, in turn, simplify your command string and remove the issue 
> of multiple quotes.
>
>
> --
> Alan G
> Author of the Learn to Program web site http://www.alan-g.me.uk/ 
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Displaying Status on the Command Line

[Avi Gross]

What Alan wrote makes sense if you just want to put out one mark per second
till you stop.

But if you want a percentage of progress, you need some way to estimate what
percent of the way you are to being done. You need to determine how many
marks represent 100% such as 50 periods. You need to have your code pause
periodically and assess if the current percentage requires more characters
than have been put out so far. If you were at 10% last time and had put out
5 dots and are now at 16% and need 3 more dots, you might say something like
this with n=3, and ch="." 

print(ch*n, end="", flush=True), sep="")

Note you want to make sure the buffers are flushed to make it real time and
when done may want to print a newline so any further output is not left
dangling.

The above print statement is for Python 3.x, and as Alan explained, in
version 2.X you may need to use alternate syntax. Not sure how you flush
buffers but you can skip the print statement and write in other ways to
sys.stdout.

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Wednesday, November 7, 2018 2:15 PM
To: tutor@python.org
Subject: Re: [Tutor] Displaying Status on the Command Line

On 07/11/2018 16:22, Chip Wachob wrote:

> What I would like to do is display, on a single line, in the terminal 
> / command line a progress percentage, or, simply a sequence of - / - 
> \, etc.. or even, accumulating period characters.
> 
> What would the escape codes be, or is there a better way to handle this?

It depends on your Python version.
In Python v2 you simply put a comma after your output character to turn off
the auto newline

while someProcess():# should probably be in a separate thread...
   time.sleep(1)  # 1 second pause
   print '.', # comma suppresses newline

If you want to suppress the spaces tyoo things get a tad more complex and
you are probably best writing direct to sys.stdout

In Python 3 there are parameters to print()

while someProcess():
   time.sleep(1)
   print('.', end='', sep='')   # no newline and no spaces

You shouldn't need any special escape codes.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Avi Gross]

I have been reading the replies and wonder sometimes if we understand the
real question as intended.

Classes in Python can be changed in all kinds of ways even after they have
been defined and the changes take effect on any new instances created
afterward. So can instances in multiple ways. If you want to store the names
of a hundred columns in a variable or even a hundred variables, you have
ways to assign them. You can even change methods on the fly.

If what you want is even more flexibility to design the class later after
receiving more data such as the names and types of the columns in a data
table, you can either write the description as text into a temporary file
and import it, if that makes sense, or make a string to be evaluated in
memory. Both can be dangerous if you do not trust the parts added as the
code is going to be run at runtime and can do malicious things.

Python often has so many ways to do things that various ones may work better
for you. In your case, one example would be to intercept the ability to set
and get (unknown) components of a class or instance by using the right
dunder function such as __getattr__ and have it KNOW about your dynamic
variable names and control access to them. There are many ways to do this,
CAREFULLY, and some work only or differently in new style classes. Heck, you
can put all the important code in an external function called by the above
that can dynamically be made in Python at a later time. One architecture
might be to store your new info in one or more dictionaries and have that
functionality check if a valid request is made and return it. Obviously it
matters where you want the data held as in per instance or per class or
superclass and so on.

Of course, I may misunderstand your issue. But from what it sounds like,
your main request is a way to associate multiple items to be stored after a
class is created but before it is used. There are an amazing number of ways
even before you loom at more advanced methods like decorators.

-Original Message-
From: Tutor  On Behalf Of
Oscar Benjamin
Sent: Wednesday, November 7, 2018 5:33 PM
To: tutor@python.org
Subject: Re: [Tutor] best way to dynamically set class variables?

On Wed, 7 Nov 2018 at 18:35, Alan Gauld via Tutor  wrote:
>
> On 07/11/2018 14:48, Albert-Jan Roskam wrote:
>
> > What is the best way to dynamically set class variables?
>
> I think I'm maybe missing the point of your question?

I think you are as well :)

IIUC then the question is: how can I programatically/dynamically create a
class that has some attributes derived from data that is known at runtime?

Am I understanding this correctly Albert?

> > # ---
> > class Parent: pass
> > class_vars = dict(col1='str', col2='int')
> >
> > # approach 1
> > Child = type('Child', (Parent,), class_vars)

This seems fine to me. It may seem cryptic but that's only because it's
unusual to do this. You are creating a "type" and that is the constructor
for type objects.

--
Oscar
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Require Python assistance

[Avi Gross]

As I see others have replied, the question is not easy to understand and seems 
so broad as to be overwhelming to actually do without further info.

It would be easier if someone asked if we could direct him to resources that 
either explain the algorithms needed or to full-blown packages/modules they 
could easily incorporate. From the text shown, does it not seem like some kind 
of HW assignment perhaps to compare and contrast various methods?

As usual, attachments cannot be seen. If someone is interested, you can ask, in 
situations like these, to be sent such things directly to your mail. 
Alternately, there are services out there that let you share files in the cloud 
for retrieval, sometimes even quite large files. Sending a link like that with 
your message might work, in moderation.

The message indicates they want to improve their computing skills. That may 
suggest wanting to implement algorithms in basic Python from scratch, as 
compared to wanting packages that do it all for you. 

If I were asking for help, I have some things I would include that include 
spelling out what I need but also a synopsis of what I have done, such as go a 
search for: " python encryption package " or whatever. If it was an assignment 
in  on page  that might be worth mentioning.

As I see it, not speaking for Alan, I see several  kinds of useful purposes in 
tutoring. One is to help someone get started when they have no idea what 
resources to even look for. You may also want to ask questions to get them to 
spell out their need more clearly and perhaps figure out what to do. You then 
expect them to go away and do the work. If they come back, it should be with a 
more focused need and again, they should get advice and hints and maybe a brief 
example but not have the work done for them. If they have a bug, and have 
narrowed it down using print statements or debuggers or local help, and can 
reproduce it in a small example, maybe they can ask if others can detect some 
syntax error or logic error. I recently heard that Python 3.8 or so may add a 
new assignment operator (:=) so if they tried it and failed under existing 
versions, we might tell them to either not use it now or wait.

This group is open and welcoming and should remain so. But it is reasonable to 
expect that answers and suggestions will not be given unless enough information 
is presented. I understand that attachments are not allowed for an assortment 
of reasons. The forum is hosted on python.org and space is limited. I could see 
creating a companion group such as pythontutorattachme...@groups.yahoo.com 
where you could send messages with attachments for those who wish to subscribe 
with the main interaction remaining here or one-on-one.

Back to the case in point. Can I assume all the encryption methods have been 
implemented in anything from algorithm pseudocode to in some programming 
language (perhaps other than Python) in a form you can read the source code? If 
so, assuming multilingual abilities, the problem becomes one of translating it 
into Python. But, of course, if the need is to make it work in all reasonable 
versions of Python, good luck!

But before we are asked, SEARCH for things. A few well placed keywords can 
locate so much. Part of learning to be a programmer is to do as much as you can 
by yourself.

-Original Message-
From: Tutor  On Behalf Of TCY 
via Tutor
Sent: Friday, November 9, 2018 8:13 AM
To: tutor@python.org
Subject: [Tutor] Require Python assistance

 


Dear
May I know how to solve the cryptography with Python programming language 
as below -
(1) Implement Elgamal Method(2) Implement Elliptic Curve Cryptography method(3) 
Implement Rabin Method(4) Implement RSA Method Find (a) Prime test (b) Inverse 
function Please help me by provide your advice and suggestion so that I can 
improve my computing skills (please see the attached file) Prayerfully  


Tron Orino Yeong tcynoteb...@yahoo.com 0916643858











___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Require Python assistance

[Avi Gross]

WARNING to any that care:

As the following letter  is a repeat request without any hint they read the 
earlier comments here, I did a little searching and see very much the same 
request on another forum asking how to do this in MATLAB:

https://forum.allaboutcircuits.com/threads/matlab-programming-language-for-cryptography.153704/

"""
Dear 

May I know how to solve the cryptography
with Matlab programming language as below -

(1) Implement Elgamal Method
(2) Implement Elliptic Curve Cryptography method
(3) Implement RSA Method
(4) Implement Rabin Method

Find
(a) Prime test
(b) Inverse function

Please help me by provide your advice and suggestion so that
I can improve my computing skills (please see the attached file)

Prayerfully 
"""
I note someone there also wonders if this is HW. And, it is formatted better on 
multiple lines as compared to our version.

If I had to guess, it might be another sort of HW as in a Cryptography course 
or Analysis of Algorithms and the person is wanting a way to implement it, not 
practice any particular language.

Many people do not speak English as a native language and we can make 
allowances while trying to understand what they want. But the above looks like 
a form-letter starting with DEAR and leaving room for the name(s) of the 
intended recipients and ends with the slightly different ending of Prayerfully 
as compared to Sincerely or Thanks in Advance and so on and no signature.

I am not sure what to make of this and am simply going to ignore the request as 
it seems to be a shotgun approach ...

Many problems can be solved in multiple languages. But if the request had 
simply been to point them to information on the cryptographic methods, it would 
not fit well in a list about problems with doing things in Python.


-Original Message-
From: Tutor  On Behalf Of TCY 
via Tutor
Sent: Saturday, November 10, 2018 10:43 AM
To: tutor@python.org
Subject: [Tutor] Require Python assistance

 

  
  
Dear
May I know how to solve the cryptography with Python programming language 
as below -
(1) Implement Elgamal Method(2) Implement Elliptic Curve Cryptography method(3) 
Implement Rabin Method(4) Implement RSA Method Find (a) Prime test (b) Inverse 
function Please help me by provide your advice and suggestion so that I can 
improve my computing skills (please see the attached file) Prayerfully  


Tron Orino Yeong tcynoteb...@yahoo.com 0916643858













  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Bufferin

[Avi Gross]

After my earlier message (snipped out for space savings) focused on choosing 
among various methods to retain a buffer of recent lines from a file, I 
realized that for many, the best method is simply to import a solution others 
have created, often in the form of an object. Many of the methods I discussed 
(and undoubtedly many other ways) are likely to be incorporated using 
object-oriented aspects in a class. I mean if you created a class called 
Bufferin and set methods like:

__init__ takes an argument "n" with say a default of 5 and initializes whatever 
internal data structure will hold the info.

__add__ might overlay the + sign operator to take the string (or any object) 
passed to it and add it to the internal representation for your ordered 
collection of n objects and perhaps delete something to make room.

__iadd__ and __radd__ might be set to allow += to work as an overlay or to 
accept adding from the other side.

__str__ or perhaps __repr__ could be overlaid to print the results in whatever 
way you want.

And, you could just make normal named methods to allow you to say 
Bufferin.insert() and Bufferin.display() or whatever interface you want to use.

Using an instance of a class this way would let you have multiple active 
buffers active containing the last N lines (or any objects) from multiple 
sources.

Subclassing such an object might even let you be more selective. For example, a 
log may contain entries you do not want buffered such as just warnings or 
entries that say what time something  started. You want the last N lines for 
context to be mostly things that are useful in debugging. Any of the above can 
include specific logic that intercepts the call to add an item and only calls 
the class method if it is deemed to be worth keeping.

In any case, buffering is an extremely common phenomenon, and a search for 
something like "python circular buffer class" can provide code you might just 
include and use for free and that may be well tested or even more efficient. 
Some are probably parts of packages normally available in built-in modules. 

So, again, the solution Alan suggested as a small part of the program he was 
discussing, is fine. I like to get more abstract and see if there is a 
reasonable solution with more power for other needs. My earlier message focused 
on ways to either do it simpler from a human perspective or maybe even faster 
and then on making the size of the buffer variable. Using a class object may 
help not only by having a way to maintain the data without passing it 
repeatedly in functional calls but by allowing multiple buffers running 
independently. Heck, I can even imagine a log where multiple processes or 
threads write to the same log file in an interspersed way. Your goal in the 
program that began this discussion might then be to scan for a problem in ANY 
process or session and as soon as found, display the last N lines for just that 
entity. 

So you might read each line, find something like a process ID, and throw the 
line into an appropriate buffer. One way would be to have a master dictionary 
whose keys are that ID and the values are instances of your buffer object. When 
a failure is encountered, get the object for that key and print the contents as 
discussed above. You might then continue processing the log after optionally 
removing that object or setting it back to empty and process any other such 
errors till the log is completed.

Just some thoughts but also a comment I must add. To those that don't know me 
(meaning almost everyone) I have been known to play games that range from humor 
to sarcasm to word games. I chose the name Bufferin because of the similarity 
in wordplay but also because there is an analgesic product called Bufferin that 
is named that for a somewhat different reason having more to do with Chemistry. 
It is basically Aspirin but with added ingredients that buffer it (Magnesium 
and Calcium carbonates and oxides)  that are antacids so that some people 
experience less gastric discomfort when taking plain aspirin. Alan used "buff" 
as a variable, which has other meanings like someone in good physical shape or 
phrases like being in the buff meaning without clothes. Similar ideas in the 
word "buffed." I generally do not stop and explain my puns or wordplay but if 
something seems phrased oddly, I may be bending things a bit. I did exert some 
self-control by not using the name of my favorite Vampire Slayer 

Last comment. If the purpose of the limited Buffer is to save space by not 
reading the entire file at once and retaining just enough context when needed, 
you may want to remember to release this space too when done using it. Python 
garbage collection may kick in for say closing the file opened in a "for line 
in open(...):" or a context manager context like "with open(...) as line:" but 
you may want to delete things like the buffers manually when done unless they 
too are set up to be temporary. For a 

Re: [Tutor] saveLine decked

[Avi Gross]

Peter,

Appreciated. I wrote something like this in another message before reading
yours. Indeed one of the things I found was the deque class in the
collections module. 

But I was not immediately clear on whether that would be directly
applicable. Their maximum sounded like if you exceeded it, it might either
reject the addition or throw an error. The behavior I wanted was sort of a
sliding window protocol where the oldest entry scrolled off the screen or
was simply removed. Sort of like what you might do with a moving average
that takes the average of just the last 20 days of a stock price.

But I searched some more and stand corrected. 

"New in version 2.4.

If maxlen is not specified or is None, deques may grow to an arbitrary
length. Otherwise, the deque is bounded to the specified maximum length.
Once a bounded length deque is full, when new items are added, a
corresponding number of items are discarded from the opposite end. Bounded
length deques provide functionality similar to the tail filter in Unix. They
are also useful for tracking transactions and other pools of data where only
the most recent activity is of interest."

That sounds exactly like what is needed. As long as you keep adding at the
end (using the append method) it should eventually remove from the beginning
automatically.  No need to use count and selectively remove or pop manually.

And despite all the additional functionality, I suspect it is tuned and
perhaps has parts written in C++ for added speed. I do note that any larger
log file used in the application discussed may throw things on the deque
many times but only ask it to display rarely so the former should be
optimized.

But one question before I go, Columbo style. The manual page
(https://docs.python.org/2/library/collections.html ) suggest you call deque
with an iterator. That would not necessarily meet our need as giving it the
entire file as an iterator would just grind away without any logic and keep
just the last N lines. We could arrange the logic in our own iterator, such
as a function that reads a line at a time using its own open iterator and
yields the line over but that too is problematic as to how and when you stop
and print the results. But on second look, the iterator is optional and I
tried creating a deque using just a maxlen=3 argument for illustration.

>>> from collections import deque

>>> a=deque(maxlen=3)
>>> a
deque([], maxlen=3)
>>> a.append('line 1\n')
>>> a
deque(['line 1\n'], maxlen=3)
>>> a.append('line 2\n')
>>> a.append('line 3\n')
>>> a
deque(['line 1\n', 'line 2\n', 'line 3\n'], maxlen=3)
>>> a.append('line N\n')
>>> a
deque(['line 2\n', 'line 3\n', 'line N\n'], maxlen=3)

OK, that looks right so all you need to figure out is how to print it in a
format you want.

As it happens, deque has an str and a repr that seem the same when I try to
print:

>>> a.__str__()
"deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)"
>>> a.__repr__()
"deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)"

So you either need to subclass deque to get your own printable version (or
use an amazing number of other Python tricks since you can, or do something
manually. 

>>> for line in a: print(line)

line 2

line 3

line N

OK, that works but my \n characters at the end of some items might suggest
using end='' in the 3.X version of print for a smaller display.

Summary: the method Peter mentions is a decent solution with no programming
or debugging overhead. It is even flexible enough, if you choose, to store
or display the lines backwards as in showing the last line that showed the
error, followed by successively earlier lines. 

Why use a limited solution when you can play with a full deck?





-Original Message-
From: Tutor  On Behalf Of
Peter Otten
Sent: Sunday, November 11, 2018 2:43 PM
To: tutor@python.org
Subject: Re: [Tutor] saveLine

Avi Gross wrote:

> Alan and others have answered the questions posed and what I am asking 
> now is to look at the function he proposed to keep track of the last 
> five lines.
> 
> There is nothing wrong with it but I wonder what alternatives people 
> would prefer. His code is made for exactly 5 lines to be buffered and 
> is quite efficient. But what if you wanted N lines buffered, perhaps 
> showing a smaller number of lines on some warnings or errors and the 
> full N in other cases?

The standard library features collections.deque. With that:

buffer = collections.deque(maxlen=N)
save_line = buffer.append

This will start with an empty buffer. To preload the buffer:

buffer = collections.deque(itertools.repeat("", N), maxlen=N)

To print the buffer:

print_buffer = sys.stdout.writelines

or, more general:

def print_buffer(items, end=""):
for item in items:
print(item, end=end)

Also, for s

Re: [Tutor] Example for read and readlines() (Asad)

[Avi Gross]

Asad,

Like many projects, there may be many ways to do things BUT some rules do
apply.

You can only read an open file ONCE unless you seek back to the beginning or
reopen it.

string = f3.read()
string1 = f3.readlines()

The first line reads the entire file into a single buffer.

The second program line won't work as intended. The first consumed the
entire file.

Much of the rest is not organized well enough for me to understand what you
want to do. I find it important for people to try some simple things like
examining the values step by step. Had you typed

print (string)
print (string1)

on a small sample file, you might have fixed that before continuing. Then
each step along the way you could examine and verify it made sense up to
that point.

Try writing the outline of the logic of your program first in English or
your native language as an algorithm. Then see what tools are needed. Look
at a sample of the log you are evaluating and see what it takes to locate
the lines you want and then to break out the parts you want to keep for
further use. 

What I see looks like this:

If you find one instance of the string "ERR1"
Then 
You want to find ALL (nonoverlapping) regions consisting of an upper-case
letter followed by two lower-case letters and a space and either a space or
digits 1 to 3 and digits 0-9 and a space and ...

Fairly complex pattern.

But you are searching the contents of the ENTIRE file for this and since you
seem to have wanted to replace all newlines by spaces and your pattern
includes spaces, this would match something that wrapped around from line to
line. Is this what you wanted?

You then switch gears to using the readlines version and I decided to get
back to my regularly scheduled life. As noted, that probably is an empty
string or worse. Good luck.

-Original Message-
From: Tutor  On Behalf Of
Asad
Sent: Sunday, November 11, 2018 8:54 PM
To: tutor@python.org
Subject: Re: [Tutor] Example for read and readlines() (Asad)

Hi All ,

   Thanks for the reply . I am building a framework for the two error
conditions, therefore I need to read and readlines because in one only regex
is required and in other regex+ n-1 line is required to process :

#Here we are opening the file and substituting space " " for each \n
encountered
f3 = open  (r"D:\QI\log.log", 'r')
string = f3.read()
string1 = f3.readlines()
regex = re.compile ( "\n" )
st = regex.sub ( " ", string )

if re.search('ERR1',st):
y=re.findall("[A-Z][a-z][a-z] [ 123][0-9]
[012][0-9]:[0-5][0-9]:[0-5][0-9] [0-9][0-9][0-9][0-9]",st)
print y

patchnumber = re.compile(r'(\d+)\/(\d+)')==> doesnot
work it only works if I use  #string = f3.read() for j in
range(len(string1)):
if re.search ( r'ERR2', string1[j] ):
print "Error line \n", string1[j - 1]
mo = patchnumber.search (string1[j-1])
a = mo.group()
print a
print os.getcwd()
break

Please advice how to proceed.

Thanks,



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Avi Gross]

I may be missing something but it looks like the embedded double quotes may be 
a problem in this:

cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % ...

if you use single quotes as in:

cut -d"="

becomes 

cut -d'='

or escape the double quotes with \" and so on ...

The above seems to be seen as:

cmd=ALPHA=BETA % ...

where ALPHA="blkid -o export %s | grep 'TYPE' | cut -d"
and BETA=" -f3"

Other issues may also be there. 
-Original Message-
From: Tutor  On Behalf Of Alan 
Gauld via Tutor
Sent: Tuesday, November 6, 2018 7:37 PM
To: tutor@python.org
Cc: python-...@python.org
Subject: Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid 
-o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in 
Python

On 06/11/2018 18:07, srinivasan wrote:

> bash command in python using subprocess module, I ma seeing the below
> *cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % 
> (fs)*

In general you should try to do as little as possible using bash and 
subprocess. Especially try to avoid long pipelines since you are starting a new 
OS process for every element in the pipeline. That means, in your case, you are 
running 4 processes to get your result - Python, blkid, grep and cut

Python is designed to do much of what the shell command can do almost as easily 
and much more efficiently (no new processes being started).

In this case just execute the blkid bit in bash because its too difficult to 
replicate simply in Python. Then use Python to search for the TYPE lines and 
slice them to size.

That will, in turn, simplify your command string and remove the issue of 
multiple quotes.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] python commandos

[Avi Gross]

Dev,

There are many ways to learn a programming language but a list of all
commands is not necessarily a good way. I suspect your real question is how
do you learn Python as either a beginner at this language or with little
knowledge of computers in general.

There are tons of books and classes both online and in the real world that
are a better way to start.

Python has a very short list of commands as in reserved words. Many are the
same or similar to other languages such as if/elif/else/while/for and so on.
What matters more is what they do and how they are used and the CONCEPTS
manipulated and even the philosophies involved in what commands are there
and even what commands are NOT there.

There are also lists of operators like + += / * >= and so on, some of which
are also reserved words in the first list like and/in/not. Again, what
matters is not the exact names but what they do and don't do.

There are tons of names for built-in functions but you can trivially replace
them or import literally tens of thousands or more other functions. There
are all kinds of objects, including the built in, with various names and
many have all kinds of internal parts with names you can access and
manipulate.

The answer to your question is that there are more names than your combined
vocabulary in all the human languages you speak but you don't need to know
them till you decide to use them.

If you want some simpler books or tutorials, many libraries (or book
sellers) let you search for key words like "Python". There are many free
sources including what you can find here:

https://www.python.org/

Specifically, you may want to look here:

https://www.python.org/doc/

If you got to this mailing list, you may already have been there. Make sure
you get educated especially in whatever version of Python you are using as
there are changes with each version with a serious disconnect between
version 3 and anything before.


-Original Message-
From: Tutor  On Behalf Of Dev
Pratap Singh
Sent: Wednesday, November 14, 2018 8:21 AM
To: tutor@python.org
Subject: [Tutor] Request to join subscriber list

Sir
I am really excited to learn about programming languages and as python
becoming very famous these days i just wanted a list of all commands of it.
Please if you are able to send it
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Avi Gross]

An interesting discussion that is outside the scope of a group like this is
HOW malicious things can be done and perhaps how to avoid them.

Obviously some contexts are totally uncontrolled. If you write a
"calculator" that asks the user to type in an arbitrary string like
"2*(3+5)" or "sin(30)" and execute that string and show the result, then
they can slip in anything like a shell command to reformat the hard disk.

What can you do to minimize risks in such situations? Obviously you might
want to scan the string before executing and look for things like carriage
returns and semi-colons that might be used to add continuation commands
beyond what is asked for. You might limit the length of the string. You
might scan for keywords like def and lambda. But note that a cursory scan
like that has false positives as well as false negatives. You might
recognize something within a character string context that is harmless or
you might reject a valid SQL query because it used a word you disallow that
is actually a harmless name of a data column.

And, realistically, Python has so many ways to get around things that it
gets silly. Given some room and ingenuity, you can create code that
assembles individual characters and then executes them into a program so a
scan may not reveal anything.

Heck, if you can simply create a module on the disk somewhere, all you need
to do is insert enough code to IMPORT the file and you can do pretty much
anything. If asked to enter a calculator entry, for example, and you simply
say:

5+3;import mymodule

You then have an exec("5+3;import mymodule")

Some such things may generate an error but only after the side effect is
done.

Python code is often wide open, by design, so subtle messing with internals
is easy. As an example, you can change the search path for modules with an
assignment statement and then any subsequent call for importing a named
module gets the one you substituted.

So, yes, executing random code can be dangerous. But life is dangerous

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Thursday, November 8, 2018 5:52 AM
To: tutor@python.org
Subject: Re: [Tutor] best way to dynamically set class variables?

On 08/11/2018 07:46, Peter Otten wrote:

> By the way I don't think exec() is bad as long as you control its 
> input and as long as this input is fairly simple.

Yes, but reading arbitrary column names from a database is not exactly
controlled input...


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Avi Gross]

In replying to what "Stealth Fleet" asked, I have too many comments,
starting with a suggestion to put a SUBECT on the Subject: line.

Your first error was not converting the text input to an integer or floating
point number.

tokenAmount = input( "How many tokens would you like to buy or cash
in?:  ")

Fix that and then see if other things fail.

BTW, you created functions which take no arguments and use a global variable
but then create local variables that would not persist in a real
application. If your assignment was just to print what would have been
calculated, that may suffice but that may be another area you think more
about.


-Original Message-
From: Tutor  On Behalf Of
Stealth Fleet
Sent: Sunday, November 11, 2018 6:00 PM
To: tutor@python.org
Subject: [Tutor] (no subject)

tokenAmount = input( "How many tokens would you like to buy or cash in?:  ")

print (tokenAmount)



def buy ():

if tokenAmount <= 400:

buy = tokenAmount * .2099

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount > "400" <= "549":

buy = tokenAmount * .3999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount >= "550" <= "749":

buy = tokenAmount * .4999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount >= "750" <= "999":

buy = tokenAmount * .6299

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

else:

buy = tokenAmount * .7999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")



def cashIn ():

cashIn = tokenAmount * .05

print( "The amount of money you will receive is $"+ (cashIn))



tokenAmount works but the buy and cashIn are not being ran why? When I put
"print(buy, cashIn)" it gives me a long message that ends in an error any
and all help is greatly appreciated. Sent from Mail
 for Windows 10
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] saveLine

[Avi Gross]

Alan and others have answered the questions posed and what I am asking now
is to look at the function he proposed to keep track of the last five lines.

There is nothing wrong with it but I wonder what alternatives people would
prefer. His code is made for exactly 5 lines to be buffered and is quite
efficient. But what if you wanted N lines buffered, perhaps showing a
smaller number of lines on some warnings or errors and the full N in other
cases?

Here is Alan's code for comparison:

buffer = ['','','','','']

def saveLine(line, buff):
buff[0] = buff[1]
buff[1] = buff[2]
buff[2] = buff[3]
buff[3] = buff[4]
buff[4] = line

Again, that works fine. If N was not 5, I would suggest initializing the
buffer might look like this:

buffed = 5
buffer = [''] * buffed

Then instead of changing the list in place, I might change the function to
return the new string created by taking the substring containing all but the
first that is then concatenated with the new entry:

def saveLineN(line, buff):
buff = buff[1:] + [line]
return buff

Clearly less efficient but more general. And, yes, the return statement
could be the entire function as in:

def saveLineN(line, buff):
return  buff[1:] + [line]


Here is a transcript of it running:

>>> buffer
['', '', '', '', '']
>>> saveLineN('a', buffer)
['', '', '', '', 'a']
>>> buffer = saveLineN('a', buffer)
>>> buffer
['', '', '', '', 'a']
>>> buffer = saveLineN('b', buffer)
>>> buffer = saveLineN('c', buffer)
>>> buffer = saveLineN('d', buffer)
>>> buffer = saveLineN('e', buffer)
>>> buffer
['a', 'b', 'c', 'd', 'e']
>>> buffer = saveLineN('6th', buffer)
>>> buffer
['b', 'c', 'd', 'e', '6th']

So perhaps using in-line changes might make sense.

Buff.pop(0) would remove the zeroeth item with the side effect of returning
the first item to be ignored. 

>>> buffer = ['a', 'b', 'c', 'd', 'e']
>>> buffer.pop(0)
'a'
>>> buffer
['b', 'c', 'd', 'e']

And it can be extended in-line:

>>> buffer.append('6th')
>>> buffer
['b', 'c', 'd', 'e', '6th']

Sorry, I mean appended, not extended! LOL!

So to make this compact and less wasteful, I consider using del buffer[0]:

>>> del buffer[0]
>>> buffer
['b', 'c', 'd', 'e']

So here is this version that might be more efficient. It deletes the first
item/line of the buffer then adds a new  nth in-line:

def saveLineN2(line, buff):
del buff[0]
buff.append(line)

Here is a transcript of it in use, using N=3 to be different:

>>> buffed = 3
  
>>> buffer = [''] * buffed
  
>>> buffer
  
['', '', '']
>>> saveLineN2('First Line\n', buffer)
  
>>> buffer
  
['', '', 'First Line\n']
>>> saveLineN2('Second Line\n', buffer)
  
>>> saveLineN2('Third Line\n', buffer)
  
>>> buffer
  
['First Line\n', 'Second Line\n', 'Third Line\n']
>>> saveLineN2('nth Line\n', buffer)
  
>>> buffer
  
['Second Line\n', 'Third Line\n', 'nth Line\n']

I can think of many other ways to do this, arguably some are more weird than
others. There is the obvious one which does all the changes in one line as
in:

buff[0],buff[1],buff[2] = buff[1],buff[2],line

Of course, for 5 you change that a bit. Might even be a tad more efficient.

There is also the odd concept of not scrolling along but dealing with things
at print time. I mean you can have a list with 1:N entries and a variable
that holds an index from 1 to N. You store a new line at buffer[index] each
time then you increment index modulo N. This resets it to 0 periodically. At
print time, you print buffer[index:] + buffer[:index] and you have the same
result.

Does anyone have comments on what methods may be better for some purposes or
additional ways to do this? I mean besides reading all lines into memory and
holding on to them and indexing backwards.

For that matter, you can revisit the question of using a list of lines and
consider a dictionary variant which might work better for larger values of
N. No need to slide a buffer window along, just maintain a modular index and
overwrite the key value as in buffer[index] = line

One more comment, if I may. 

Alan mentions but does not define a printBuffer() function. Using any of the
methods above, you can end up with an error happening early on so the kind
of fixed-length buffer mentioned above contains blank entries (not even a
'\n') so it probably should suppress printing items of zero length or that
are empty. And, in some of the methods shown above, it may be worth starting
with an empty buffer and adding lines up to some N and only then removing
the first entry each time. That would complicate the code a bit but make
printing trivial.





___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] look back comprehensively

[Avi Gross]

I have been thinking about the thread we have had where the job seemed to be
to read in a log file and if some string was found, process the line before
it and generate some report. Is that generally correct?

 

The questioner suggested they needed both the entire file as one string but
also as a list of strings. Suggestions were made, IF SO, to read the entire
file twice, once as a whole and once as lines. Another suggestion was to
read either version ONCE and use cheaper Python methods to make the second
copy.

 

We also looked at a similar issue about using a buffer to keep the last N
lines.

 

I thought of another tack that may be in between but allow serious
functionality. 

 

OUTLINE:

 

Use just the readlines version to get a list of strings representing each
line. Assuming the searched text is static, no need for regular expressions.
You can ask if 

 

'something' in line

 

But if you find it, you may not have an index so a use of enumerate (or zip)
to make a tuple might be of use. You can do a list comprehension on an
enumerate object to get both the indexes where the requested 'something' was
found and also optionally the line contents (ignored) and be able to use the
indices to look at the index (or more) before.

 

Here is an example using a fake program where I create the four lines and
generate only the tuples needed for further processing, or just the indices
for the line ABOVE where it was found. It can be done with simple list
comprehensions or made into a generator expression.

 

-CODE-

"""

Sample code showing how to read a (simulated) file

and search for a fixed string and return the item

number in a list of strings for further processing

including of earlier lines.

"""

 

# Make test data without use of file

 

fromfile =  str1 = """alpha line one

beta line two

gamma line three

alphabet line four"""

 

lines= fromfile.split('\n')

print("RAW data: ", lines) # just for illustration

 

errors = [(index,line)

  for (index, line) in enumerate(lines)

  if 'bet' in line]

 

just_indices = [index - 1

for (index, line) in enumerate(lines)

if 'bet' in line]

 

from pprint import pprint

print("ERROR tuples:") # just for illustration

pprint(errors)

 

print("Just error indices:")

pprint(just_indices)

-END-CODE-

 

-OUTPUT-

RAW data:  ['alpha line one', 'beta line two', 'gamma line three', 'alphabet
line four']

ERROR tuples:

[(1, 'beta line two'), (3, 'alphabet line four')]

Just error indices:

[0, 2]

-END-OUTPUT-

 

Again, this did two ways, and only one is needed. But the next step would be
to iterate over the results and process the earlier line to find whatever it
is you need to report. Many ways to do that such as:

 

for (index, ignore) in errors

 

or

 

for index in just_indices

 

You can use a regular expression on a line at a time. And so on.

 

Again, other methods mentioned work fine, and using a deque to store earlier
lines in a limited buffer while not reading the entire file into memory
would also be a good way.

 

Warning: the above assumes the text found will never be in the zeroeth line.
Otherwise, you need to check as accessing line -1 may actually return the
last line!

 

As stated many times, there seem to be an amazing number of ways to do
anything. As an example, I mentioned using zip above. One obvious method is
to zip it with a range statement making it look just like enumerate. A more
subtle one would be to make a copy of the set of lines the same length but
each line content shifted by one. Zip that to the original and you get a
tuple with (line 0, null) then (line 1, line 0) up to (line n, line n-1)

 

Yes, that doubles memory use but you can solve so much more in one somewhat
more complicated list comprehension. Anyone want to guess how?

 

If you recall, some regular expression matches something on the previous
line. Let us make believe you wrote a function called do_the_match(line, re)
that applies the regular expression to the line of text and returns the
matching text or perhaps an empty string if not found. So if you define that
function then the following code will work.

 

First, make my funny zip:

 

I make lines_after as copy of lines shifted over one. Or more exactly
circularly permuted by one. The goal is to search in line_after and if
found, do the regular expression match in line before.

 

>>> lines_after = lines[1:]

>>> lines_after.append(lines[0])

>>> lines_after

['beta line two', 'gamma line three', 'alphabet line four', 'alpha line
one']

>>> lines

['alpha line one', 'beta line two', 'gamma line three', 'alphabet line
four']

>>> list(zip(lines_after, lines))

[('beta line two', 'alpha line one'), ('gamma line three', 'beta line two'),
('alphabet line four', 'gamma line three'), ('alpha line one', 'alphabet
line four')]

 

So the list comprehension looks something like this:

 

matches = [ do_theMatch(line, re)

   for (line_after, line) 

Re: [Tutor] Regarding "IDLE Subprocess Didn't Make Connection" Error

[Avi Gross]

BREAK fast ,

You certainly haven't been idle!

Until you get IDLE working ideally, you can still use one of many text editors 
and programming environments and perhaps run python on the files manually. 

Have you verified that the rest of your installation worked?


Sent from AOL Mobile Mail
On Friday, November 16, 2018 Break fast  wrote:
Hello Python Tutors,

First off, thank you for existing! I hope I am emailing the right queue in
this situation.

I have installed Python from the python.org website, but have found myself
unable to use IDLE due to an error that is as follows:

"IDLE subprocess didn't make connection. Either IDLE can't start a
subprocess, or personal Firewall software is blocking the connection"

I have been researching this issue nonstop the last two days, and haven't
had any luck in resolving the issue.


I am running Windows 7 (x64), and have installed Python 3.7.1 and 3.6.7
numerous times now.

>I have shut off Avast (the only firewall on this pc).
>Scoured the Python root folder for any .py files, and there are none.
>Restarted PC after each install.
>Installed Python to a different harddrive.
>Ran IDLE as Admin

And I am still encountering the same issue.

If anybody has any insight on how to proceed from here, I would be
extremely grateful for any advice you could offer.

Thank you again, and hope you all have a good day!

--Logan McDonald


Virus-free.
www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

Steven,

As I head out the door, I will sketch it.

Given a data.frame populated with N rows and columns you want to break it
into training and test data sets.

In a data.frame, you can refer to a row by using an index like 5 or 2019.
You can ask for the number of rows currently in existence. You can also
create an array/vector of length N consisting of instructions that can tell
which random rows of the N you want and which you don't. For the purposes of
this task, you choose random numbers in the range of N and either keep the
numbers as indices or as a way to mark True/False in the vector. You then
ask for a new data.frame made by indexing the existing one using the vector.
You can then negate the vector and ask for a second new data.frame indexing
it.

Something close to that.

Or, you can simply add the vector as a new column in the data.frame in some
form. It would then mark which rows are to be used for which purpose. Later,
when using the data, you include a CONDITION that row X is true, or
whatever.



-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 28, 2018 11:12 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On Fri, Dec 28, 2018 at 10:39:53PM -0500, Avi Gross wrote:
> I will answer this question then head off on vacation.

You wrote about 140 or more lines, but didn't come close to answering the
question: how to randomly split data from a dictionary into training data
and reserved data.



--
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

Steven,

A more practical answer about splitting a data frame is to import modules such 
as for machine learning. 

Import sklearn.model_selection

Then use train_test_split() to return 4 parts. Not sure what answers you need 
and why here. Plenty of ways and tools exist to specify choosing percentages to 
partition by or other ways.


Sent from AOL Mobile Mail
On Saturday, December 29, 2018 Avi Gross  wrote:
Steven,

As I head out the door, I will sketch it.

Given a data.frame populated with N rows and columns you want to break it
into training and test data sets.

In a data.frame, you can refer to a row by using an index like 5 or 2019.
You can ask for the number of rows currently in existence. You can also
create an array/vector of length N consisting of instructions that can tell
which random rows of the N you want and which you don't. For the purposes of
this task, you choose random numbers in the range of N and either keep the
numbers as indices or as a way to mark True/False in the vector. You then
ask for a new data.frame made by indexing the existing one using the vector.
You can then negate the vector and ask for a second new data.frame indexing
it.

Something close to that.

Or, you can simply add the vector as a new column in the data.frame in some
form. It would then mark which rows are to be used for which purpose. Later,
when using the data, you include a CONDITION that row X is true, or
whatever.



-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 28, 2018 11:12 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On Fri, Dec 28, 2018 at 10:39:53PM -0500, Avi Gross wrote:
> I will answer this question then head off on vacation.

You wrote about 140 or more lines, but didn't come close to answering the
question: how to randomly split data from a dictionary into training data
and reserved data.



--
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

[Mark Lawrence please press DELETE now in case the rest of this message is
all about you.]
[[If that is not working, if on Windows, try Control-ALT-DELETE as that will
really get rid of my message.]]

Back to replying to Steven,

Of course I want to be corrected when wrong.

I think everyone here knows I tend to be quite expansive in my thoughts and
sometimes to the point where they suggest I am free-associating. I am trying
to get to the point faster and stay there.

So if what I write is not wrong as a general point and you want to bring up
every exception, fine. I reserve the right not to follow you there,
especially not on the forum. I may continue a discussion with you in
private, of course.

I often have a problem in real life (not talking about you, let alone
whoever Mark is) where I think I said something clearly by using phrases
like "if" and find the other person simply acts as if I had left that out.
You know, we can go to the park IF it is not raining tomorrow. Reply is to
tell me the weather report says it will rain so why am I suggesting we go to
the park. Duh. I was not aware of the weather report directly BUT clearly
suggested it was a consideration we should look at before deciding. 

Now a more obvious error should be pointed out. EXAMPLE, I am driving to
Pennsylvania this weekend not far from a National Park and will have some
hours to kill. I suggested we might visit Valley Forge National Historic
Park and did not say only if it was open. Well, in the U.S. we happen to
have the very real possibility the Park will be closed due to it being
deemed optional during a so-called Government Shutdown so such a reply IS
reasonable. I did not consider that and stand corrected.

But Chris, you point out I reacted similarly to what you said. Indeed, you
said that sometimes we don't need to focus on efficiency as compared to
saying we should always ignore it or something like that. I think we
actually are in relative agreement in how we might approach a problem like
this. We might try to solve it in a reasonable way first and not worry at
first about efficiency especially now that some equipment runs so fast and
with so much memory that results appear faster than we can get to them. But,
with experience, and need, we may fine tune code that is causing issues. As
I have mentioned, I have applications that regularly need huge samples taken
at random so a list of millions being created millions of times and the
above being done thousands of times, adds up. Many cheaper methods might
then be considered including, especially, just switching to a better data
structure ONCE.

I will stop this message here as I suspect Mark is still reading and fuming.
Note, I do not intend to mention Mark again in future messages. I do not
actually want to annoy him and wish he would live and let live.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Thursday, December 27, 2018 5:38 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On Wed, Dec 26, 2018 at 11:02:07AM -0500, Avi Gross wrote:

> I often find that I try to make a main point ad people then focus on 
> something else, like an example.

I can't speak for others, but for me, that could be because of a number of
reasons:

- I agree with what you say, but don't feel like adding "I agree" 
after each paragraph of yours;

- I disagree, but can't be bothered arguing;

- I don't understand the point you intend to make, so just move on.

But when you make an obvious error, I tend to respond. This is supposed to
be a list for teaching people to use Python better, after all.


> So, do we agree on the main point that choosing a specific data structure
or
> algorithm (or even computer language) too soon can lead to problems that
can
> be avoided if we first map out the problem and understand it better?

Sure, why not? That's vague and generic enough that it has to be true.

But if its meant as advice, you don't really offer anything concrete. 
How does one decide what is "too soon"? How does one avoid design 
paralysis?


> I do not concede that efficiency can be ignored because computers are
fast.

That's good, but I'm not sure why you think it is relevant as I never 
suggested that efficiency can be ignored. Only that what people *guess* 
is "lots of data" and what actually *is* lots of data may not be the 
same thing.


> I do concede that it is often not worth the effort or that you can
> inadvertently make things worse and there are tradeoffs.

Okay.


> Let me be specific. The side topic was asking how to get a random key from
> an existing dictionary. If you do this ONCE, it may be no big deal to make
a
> list of all keys, index it by a random number, and move on. I did supply a
> solution that might(or might not) run faster by using a generator to get
one
> item at a time and stopping when found. Less space but not sure if less
> time.

Re: [Tutor] dangerous class neighborhood

[Avi Gross]

My apologies. This reply went to the wrong forum.

Hopefully it contains little to be debated. An even shorter version would
be:

If at first you don't succeed ...

-Original Message-
From: Tutor  On Behalf Of Avi
Gross
Sent: Thursday, December 27, 2018 9:48 PM
To: tutor@python.org
Subject: Re: [Tutor] dangerous class neighborhood

Sometimes when I post something I get back comments and evaluate them and
learn quite a bit. I then reply and debate every little point and it can
continue for a few rounds.

I don't seem to be in that mood today so let me simply restate my entire
post in a few sentences with no examples, no lectures, no advice on what
anyone else can do and very little for anyone to bother replying to. Here
goes:

Sometimes when I run up against a wall and find that a solution to a problem
does not work because things may not work as I expected, I pause. I
reconsider what I actually need to get done. Then I look to see if I can
come up with other ways to do it that will work while still getting the
important parts done. Failing that, I ask if perhaps there is another tool,
such as another programming language that is a better fit for the task. And,
if the work needed seems excessive, I ask if perhaps the problem does not
really need to be solved by me and I move on.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Interpreter pasting Question

[Avi Gross]

This is a serious question. I have tried things and searched and remain
stumped. It is about python and perhaps just the interpreter.

 

Copying and pasting multiple lines into the interpreter fails in mysterious
ways, unless they are a logical single entity.

 

Is there a way to change this behavior, or perhaps an editor/environment
that feeds multiple lines more carefully to the interpreter?

 

I am used to using R Studio and other tools with R. This is not about R but
please permit me an example.

 

I like to be able to tell it in the EDIT window that I want a single line or
a group of lines to be run on the console. That is especially useful when I
am debugging and want to run some code and go to the console and examine
things then continue. I may want to run one line or a region. Clearly
indentation is not an issue there.

 

The python interpreter I use (cpython 3.7.0) on windows lets me copy/paste a
simple single line of code. If it is a complex construct like an if/else
that is syntactically seen as a single statement, it allows it to be copied
in as a unit.

 

But something as simple as this:

 

X=5

Y=6

 

When copied in looks like this:

 

>>> X=5

Y=6

 

SyntaxError: multiple statements found while compiling a single statement

 

Yet I have seen programs that build up a string and give that to eval. Can
those be multiple statements? My tests using eval confuse me. But things I
can type in to the interpreter could be the same after a paste but perhaps
there are reasons they can't be?

 

It may be something as silly as python being too fast and reading all the
input. When I type and hit CARRIAGE RETURN/ENTER it may finish evaluating
before I even start typing the next line. 

 

The current behavior makes it hard to copy code from something like an email
message. Yes, I can (and do) insert the code in a file using IDLE or other
tools then ask to run the file but that is far from the same. Having to copy
one line at a time is beyond frustrating for me.

 

Feel free to tell me I am doing it wrong or of a way that works for others. 

 

Python has an eval() and an exec() and I would assume the latter would be a
way to see what works.

 

Here are three lines using \n:

 

>>> exec("x=6\ny=7\nprint(x+y)")



13

 

That seems to work. Again, if I copied and pasted the same as three lines,
it fails.

 

This works too:

 

>>> dothis = """x=6

y=7

print(x+y)

"""



>>> exec(dothis)



13

 

I did some HW before writing this and some suggest ipython has a better
repr.

 

Some solutions are arguably weird even if they work. One suggestion is to
wrap it all in an if that is always true and indent all subsequent lines:

 

if 1:

x = 6

y = 7

print(x+y)

 

When I do a copy and paste of that including the otherwise meaningless if
statement, it works:

 

>>> if 1:

x = 6

y = 7

print(x+y)

 

13

 

So, sure, I can stick nonsense in but wonder if there is a reasonable trick
like setting options to the interpreter or using an editor that feeds lines
slowly or .

 

But, yes, I can live with this or perhaps learn how to use a debugger than
might let me run things with breakpoints or .

 

 

 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Re Module

[Avi Gross]

Asad,

After reading replies to you by Alan and Steven I want to ask you if you can
first tell us in normal words what the exact outline of the program does. If
you only want help on one small part, tell us  about that.

I was first fooled into thinking you wanted to show us how you solve the
majority of the entire problem, whatever it was so I wanted to hear things
like I show next.

An example would be to search two files for error matches of various kinds
and report if they contain any matches. Just report True versus False or
something.

Another goal might be to show the first match in some way then quit.

Another might be to do the same search in two files and report ALL the
matches in some format.

After being clear on the goal, you might specify the overall algorithm you
want to use. For example, do you process one file to completion and save
some results then process the other the same way then compare and produce
output? Or do you process both nearly simultaneously in one pass, or perhaps
multiple passes. Do you search for one error type at a time or all at once?
Can there be multiple errors on the same line of the same kind or different
ones? What does error even mean? Is it something like "Fail: 666" versus
"Warn: 42" or something where multiple errors share a part or ...

Once we have some idea of the goal, we could help you see if the approach
seems reasonable even before reading the code. And, when reading the code,
we might see if your implementation  seems to match the plan so perhaps we
can see where you diverge from it perhaps with a mistake.

If I just look at what you provided, you do some of what I asked. You are
not clear on what the two files contain other than they may have an error
that you can identify with a set of patterns. Can you tell us if you are
looking at one line at a time, assuming it is a text file? Your code shows
no evidence of a file at all. Your focus in what you share with us is mainly
on creating a list of compiled search patterns and applying it to one
uninitialized "st" and trying to figure out which one matched. 

You do not show any examples of the pattern but suggest something is
failing. For all we know one of your patterns just matched the presence of a
single common character or even was not formatted properly and failed to be
compiled.

My impression is you are not actually asking about the overall problem. Your
real question may be how to use a regular expression on a string and find
out what matched. If so, that would be the headline, not about two files.
And it may even be your entire approach could change. An example would be to
store your patterns as a text keyword in a dictionary with the value being
the compiled version so when you evaluate a line using the pattern, you know
which one you matched with. I am NOT saying this is a good solution or a
better one. I am asking you to think what you will need and what techniques
might make life easier in doing it.

So besides trying to alter some code based of the feedback, from others,
could you resubmit the question with a focus on what you are doing and what
exactly is not working that you want looked at. Specifics would be useful
including at least one pattern and a line of sample text that should be
matched by the pattern as an example and perhaps one that should not. And
any error messages are vital.

When you do, I am sure Steven and Alan and others might be able to zoom
right in and help you diagnose, if you don't figure it out by yourself first
by being able to see what your goal is and perhaps doing a little debugging.

-Original Message-
From: Tutor  On Behalf Of
Asad
Sent: Thursday, December 27, 2018 10:10 AM
To: tutor@python.org
Subject: [Tutor] Re Module

Hi All ,

  I trying find a solution for my script , I have two files :

file1 - I need a search a error say x if the error matches

Look for the same error x in other file 2

Here is the code :
I have 10 different patterns therefore I used list comprehension and
compiling the pattern so I loop over and find the exact pattern matching

re_comp1 = [re.compile(pattern) for pattern in str1]

for pat in re_comp1:
if pat.search(st,re.IGNORECASE):
x = pat.pattern
print x===> here it gives the expected output it correct
match
print type(x)



if re.search('x', line, re.IGNORECASE) is not None:  ===> Gives a wrong
match
  print line

Instead if I use :

if re.search(x, line, re.IGNORECASE) is not None: then no match occurs
  print line

Please advice where I going wrong or what can be done to make it better .

Thanks,


--
Asad Hasan
+91 9582111698
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:

Re: [Tutor] dangerous class neighborhood

[Avi Gross]

Sometimes when I post something I get back comments and evaluate them and
learn quite a bit. I then reply and debate every little point and it can
continue for a few rounds.

I don't seem to be in that mood today so let me simply restate my entire
post in a few sentences with no examples, no lectures, no advice on what
anyone else can do and very little for anyone to bother replying to. Here
goes:

Sometimes when I run up against a wall and find that a solution to a problem
does not work because things may not work as I expected, I pause. I
reconsider what I actually need to get done. Then I look to see if I can
come up with other ways to do it that will work while still getting the
important parts done. Failing that, I ask if perhaps there is another tool,
such as another programming language that is a better fit for the task. And,
if the work needed seems excessive, I ask if perhaps the problem does not
really need to be solved by me and I move on.

-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Thursday, December 27, 2018 5:11 PM
To: Python 
Subject: Re: dangerous class neighborhood

On Fri, Dec 28, 2018 at 8:47 AM Avi Gross  wrote:
> Question 2: Do you want the variables available at the class level or 
> at the instance level?

For constants, definitely put them on the class. They'll be available on
instances as well ("for free", if you like). For mutables, obviously you
need to decide on a case-by-case basis.

> Question 3: Which python variations on syntactic sugar, such as list 
> comprehensions, get expanded invisibly in ways that make the problem 
> happen by asking for variables to be found when no longer in the visible
range?

The oddities with comprehensions were tackled partly during the discussion
of PEP 572. If you want to know exactly why this isn't changing, go read a
few hundred emails on the subject. A lot of the main points are summarized
in the PEP itself:

https://www.python.org/dev/peps/pep-0572/

> There may be matters of efficiency some would consider but some of the 
> examples seen recently seemed almost silly and easy to compute. The 
> people asking about this issue wanted to define a bunch of CONSTANTS, 
> or things that might as well be constants, like this:
>
>
>
> def Foo():
>
> A = ("male", "female", "other")
>
> B = [ kind[0] for kind in A ]# First letters
> only
>
> # And so on making more constants like a dictionary 
> mapping each string to a number or vice versa.
>
>
>
> All the above can be evaluated at the time the class is defined but 
> unintuitive scope rules make some operations fail as variables defined 
> in the scope become unavailable to other things that SEEM to be 
> embedded in the same scope.

If you write simple and Pythonic code, these will almost always work
perfectly. The recent thread citing an oddity worked just fine until it was
written to iterate over range(len(x)) instead of iterating directly.

> If they are ONLY to be used within an instance of Foo or invoked from 
> within there, there may be a fairly simple suggestion. If you already 
> have a __init__ method, then instantiate the variables there carefully 
> using the self object to reference those needed.

But why? __init__ should initialize an instance, not class-level constants.
A Python class is not restricted to just methods, and there's no reason to
avoid class attributes.

> Create a function either outside the class or defined within. Have it 
> do any internal calculations you need in which all internal variables 
> can play nicely with each other. Then let it return all the variables 
> in a tuple like
> this:
>
> def make_sexual_constants():
>
> A = .
>
> B = .
>
> C = f(A,B)
>
> D = .
>
> def Foo():
>
> (A, B, C, D) = make_sexual_constants():

Lovely. Now you have to define your variables once inside the function, then
name them a second time in that function's return statement, and finally
name them all a *third* time in the class statement (at least, I presume
"def Foo():" is meant to be "class Foo:"). A mismatch will create bizarre
and hard-to-debug problems.
What do you actually gain? Can you show me real-world code that would truly
benefit from this?

> Can we agree that the class Foo now has those 4 variables defined and 
> available at either the class level or sub-class or instance levels? 
> But the values are created, again, in a unified safe environment?

Unified? No more so than the class statement itself. Safe? Definitely not,
because of the mandatory duplication of names.

> As noted in section 3, it would be good to know what python features 
> may be unsafe in this

Re: [Tutor] Defining variable arguments in a function in python

[Avi Gross]

I have my usual off the wall answer.

OK, seriously. Not exactly an answer but perhaps an experiment.

The question was how to have a non-named first argument to a function with
some form of default. 

As was pointed out, this does not fit well with being able to have python
gather all positional arguments after it as well as all keyword arguments.

But bear with me. Say I want to have a way to signal that I want a default
for the first argument?

An empty comma fails but try this:

def hello(a, *n, **m) :
if a == None: a=5
print(a)
print(*n)
print(**m)

The above says "a" is required. It can be followed by any number of
positional args gathered into "n" and any number of keyword args gathered
into "m"

But what if you define a sentinel to watch for such as None, in the above?

If the first and only arg is None, it switches to the default of 5.

>>> hello(None)
5

Add a few more args and it properly takes it.

>>> hello(1,2,3)
1
2 3

Switch the first to None:

>>> hello(None,2,3)
5
2 3

The keywords don't work for print but no biggie.

But is this only for None? What I say any negative arg is replaced by 5?

def hello(a, *n, **m) :
if a < 0: a=5
print(a)
print(*n)

Seems to work fine:

>>> hello(-666, 2, 3, 4)
5
2 3 4

And I wonder if we can use the darn ellipsis for something useful?

def hello(a, *n, **m) :
if a == ... : a=5
print(a)
print(*n)

>>> hello(1,2,3)
1
2 3
>>> hello(...,2,3)
5
2 3
>>> hello(...,2,...)
5
2 Ellipsis

OK, all kidding aside, is this helpful? I mean if you want a function where
you MUST give at least one arg and specify the first arg can be some odd
choice (as above) and then be replaced by  a default perhaps it would be
tolerable to use None or an Ellipsis.

Or on a more practical level, say a function wants an input from 1 to 10.
The if statement above can be something like:

>>> def hello(a, *n, **m) :
if not (1 <= a <= 10) : a=5
print(a)
print(*n)


>>> hello(1,2,3)
1
2 3
>>> hello(21,2,3)
5
2 3
>>> hello(-5,2,3)
5
2 3
>>> hello("infinity and beyond",2,3)
Traceback (most recent call last):
  File "", line 1, in 
hello("infinity and beyond",2,3)
  File "", line 2, in hello
if not (1 <= a <= 10) : a=5
TypeError: '<=' not supported between instances of 'int' and 'str'

As expected, it may take a bit more code such as checking if you got an int
but the idea may be solid enough. It is NOT the same as having a default
from the command line but it may satisfy some need.

Other than that, I fully agree that the current python spec cannot support
anything like this in the function definition.

Side note: To spare others, I sent Steven alone a deeper reply about ways to
select random rows from a pandas DataFrame. I am still learning how pandas
works and doubt many others here have any immediate needs.









-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Saturday, December 29, 2018 6:02 AM
To: tutor@python.org
Subject: Re: [Tutor] Defining variable arguments in a function in python

On Sat, Dec 29, 2018 at 11:42:16AM +0530, Karthik Bhat wrote:
> Hello,
> 
> I have the following piece of code. In this, I wanted to make 
> use of the optional parameter given to 'a', i.e- '5', and not '1'
> 
> def fun_varargs(a=5, *numbers, **dict):
[...]
> 
> fun_varargs(1,2,3,4,5,6,7,8,9,10,Jack=111,John=222,Jimmy=333)
> 
> How do I make the tuple 'number' contain the first element to be 1 and not
2?


You can't. Python allocates positional arguments like "a" first, and only
then collects whatever is left over in *numbers. How else would you expect
it to work? Suppose you called:

fun_varargs(1, 2, 3)

wanting a to get the value 1, and numbers to get the values (2, 3). And then
immediately after that you call 

fun_varargs(1, 2, 3)

wanting a to get the default value 5 and numbers to get the values (1, 2,
3). How is the interpreter supposed to guess which one you wanted?

If you can think of a way to resolve the question of when to give "a" 
the default value, then we can help you program it yourself:


def func(*args, **kwargs):
if condition:
# When?
a = args[0]
numbers = args[1:]
else:
a = 5  # Default.
numbers = args
...

But writing that test "condition" is the hard part.




--
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python

[Avi Gross]

Mary,

Mary,

It is often best to develop and test small parts of the project where you
can easily play with it, then move it into more complex configurations like
a function body

Here is your code:

def read_words(words_file):
return [word.upper() for line in open(words_file, 'r') for word in
line.split()]

I made a file on my local system and this works:

def read_words(words_file):
return [word.upper() for line in open(words_file, 'r') for word in
line.split()]

now you are returning an uppercase version of the current 'word" stored in
word. So what is the length of that word?

Here is the modified variation on your code:

>>> [word.upper() for line in open('TESTINK.txt', 'r') for word in
line.split()]
['THIS', 'IS', 'LINE', 'ONE', 'AND', 'THIS', 'IS', 'ANOTHER', 'LINE',
'JUST', 'TO', 'TEST', 'WITH.']

Here is yet another modification showing the length, instead:

>>> [len(word) for line in open('TESTINK.txt', 'r') for word in
line.split()]
[4, 2, 4, 3, 3, 4, 2, 7, 4, 4, 2, 4, 5]

By your rules, you want to only keep those words where "len(word) > 3"

So where in the list comprehension would you add this an if condition to get
this?

['THIS', 'LINE', 'THIS', 'ANOTHER', 'LINE', 'JUST', 'TEST', 'WITH.']

Since you read in all your data using the same function, you might even make
it take an optional value to cut at, defaulting with 3 or even 0.

-Original Message-
From: Tutor  On Behalf Of
Mary Sauerland
Sent: Thursday, December 20, 2018 10:49 AM
To: tutor@python.org
Subject: [Tutor] Python

Hi, 

I want to get rid of words that are less than three characters but I keep
getting errors. I tried multiple ways but keep getting errors. 

Here is my code:

f1_name = "/Users/marysauerland/Documents/file1.txt"
#the opinions
f2_name = "/Users/marysauerland/Documents/file2.txt"
#the constitution


def read_words(words_file):
return [word.upper() for line in open(words_file, 'r') for word in
line.split()]


read_words(f1_name)
#performs the function on the file
set1 = set(read_words(f1_name))
#makes each word into a set and removes duplicate words
read_words(f2_name)
set2 = set(read_words(f2_name))

count_same_words = 0

for word in set1:
if word in set2:
count_same_words += 1
#comparing the set1 (set of unique words in the opinions) with set2 (set of
unique words in the constitution) and adding 1 for each matching word found
which is just counting the words
print(count_same_words)


Best, 

Mary
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

Mike,

Excellent advice.

I find that many people are fairly uncomfortable with abstraction and tend to 
resist a pure top down approach by diving to any solutions they may envision. 
For example, if you say things like create a data structure that can hold as 
many kinds of information as will be needed. The data should be able to be 
viewed in several ways and adding a new item should be fast even if the number 
of items grows large ...

Some will have stopped reading (or creating) and will jump to deciding then 
need a dictionary. Others may want a deque. Some may insist they need a new 
class. 

But wait, if you continue reading or designing, it may be clear that some 
choices are not optimal. Heck, it may turn out some design elements are 
contradictory. As someone asked on another python list, is there a better way 
to get a random key for a dictionary. Well, not easily without expanding all 
keys into a list of perhaps huge length. Followed by a search of much of that 
list to get the nth index. So maybe a plain dictionary does not make that easy 
or efficient so do you give up that need or use some other data structure that 
makes that fast? Perhaps you need a hybrid data structure. One weird idea is to 
use the dictionary but every time you generate a new key/value pair you also 
store a second pair that looks like "findkey666": key so that a random key of 
the first kind can be found in constant time by picking a random number up to 
half the number of items, concatenate it to "findkey" and look up the value 
which is a key.

When you try to work bottom up with students, some see no point as they are 
missing the big picture. I used to work during graduate school writing PASCAL 
code for a company making flexible manufacturing systems and my job often was 
to read a man page describing some function that did something minor. I often 
had no clue why it was needed or where it would be used? I was sometimes told 
it had to FIT into a certain amount of memory because of the overlay technique 
used and if it was compiled to something larger, was asked to break the 
function down into multiple functions that were called alternately  
Sometimes an entire section had to be redesigned because it had to fit into the 
same footprint as another. That was the limit of the big picture. A shadow!

What I found works for me is a combination. I mean teaching. You give them just 
enough of the top-down view for motivation. Then you say that we need to figure 
out what kinds of things might be needed to support the functionality. This 
includes modules to import as well as objects or functions to build. But that 
too can be hard unless you move back into the middle and explain a bit about 
the subunit you are building so you know what kind of support it needs closer 
to the bottom.

I admit that my personal style is the wrong one for most people. I do top down 
and bottom up simultaneously as well as jump into the middle to see both ways 
to try to make sure the parts will meet fairly seamlessly. Does not always work.

How often have we seen a project where some function is designed with three 
arguments. Much later, you find out some uses of the function only have and 
need two but some may have additional arguments, perhaps to pass along to yet 
another function the second will conditionally invoke? It may turn out that the 
bottom up approach starting from one corner assumed that the function would 
easily meet multiple needs when the needs elsewhere are not identical enough. 
If they keep demanding one function to master all, you can end up with fairly 
awful spaghetti code. Of course python is not a compiled language like C/C++ 
and PASCAL and many others were. It is often fairly easy in python to have a 
variable number of arguments or for the same function to do something 
reasonable with multiple types and do something reasonable for each.

One thing I warn people about is mission creep. When asked to do something, try 
not to add lots of nice features at least until you have developed and tested 
the main event. I have seen many projects that did feel the need to add every 
feature they could imagine as there remained keys on the keyboard that did not 
yet invoke some command, even if no customer ever asked for it or would ever 
use it. Amazing how often these projects took too long and came to market too 
late to catch on ...

Some of the people asking questions here do not even tell us much about what is 
needed, let alone their initial design plan. It can take multiple interactions 
back and forth and I wonder how many give up long before as they just want an 
ANSWER. 

In case you wonder, I am reliably told the answer to life, the universe and 
everything  is 2*21.

-Original Message-
From: Mike Mossey  
Sent: Tuesday, December 25, 2018 9:49 PM
To: Avi Gross 
Subject: Re: [Tutor] decomposing a problem


> On Dec 25, 2018, at 4:00 PM, Avi Gross  wrote:
> 
> 

[Tutor] decomposing a problem

[Avi Gross]

>>> items8_5_3
[('f8', 'f5', 'f3'), ('g8', 'g5', 'g3')]

Or if you want them back as character with an underscore between:

>>> items8_5_3 = ['_'.join([h8, h5, h3]) for (h1,h2,h3,h4,h5,h6,h7,h8) in
splitsville]
>>> items8_5_3
['f8_f5_f3', 'g8_g5_g3']

The point is that we have oodles of little tools we can combine to solve
bigger problems, sometimes in a big complicated mess and sometimes a simple
step at a time. Not all can be easily chained the same way but then we have
a bit more complex topic like generators and queues that can be chained
together in ways even more complex than UNIX pipelines. Each generator would
only be called to produce one result when another needs it.

And I assume it might be possible to make a series of methods that are
placed in an object that extends an object type like "list" that maintain an
internal representation like a list of strings and changes it in place just
like .sort() does.

So to apply a UNIX-style pipeline may be as simple as:

mystdout = mystdin("LIST OF STRINGS TO
INITIALIZE).method1(args).method2(args)methodn(args)

The initializer will set the current "lines" and each method will loop on
the lines and replace them with the output it wants. Perhaps the initializer
or first method may actually read all lines from stdin. Perhaps the last
method will write to stdout. All methods will effectively follow in sequence
as they massage the data but will not actually run in parallel.

And you can even write a generic method that accepts any external function
designed to accept such a list of lines and return another to replace it.

My point to Mats is that the goal is to learn to divide and conquer a
problem. Using small and well defined methods that can fit together is
great. Many things in python can be made to fit and some need work. Dumb
example is that sorting something internally returns None and not the object
itself. So you cannot chain something like object.upper().sort() any
further. You may be able to chain this:

>>> list(reversed(sorted(lines))).pop()
'f1,f2,f3,f4,f5,f6,f7,f8'

Why the odd syntax? Because the developers of python in their wisdom may not
have chosen to enhance some methods to do things another way.

Object.sort() and Object.reverse() will change the internals and return
nothing. They are not designed to be piped. If there was a method that
performed a sort AND returned the object or performed a reverse and returned
the object, then we might see:

lines.sort(show=True).reverse(show=True).pop()

Or some other similar stratagem. Then we could write a fairly complex
sequence in a pipelined mode.

-Original Message-
From: Tutor  On Behalf Of
Mats Wichmann
Sent: Tuesday, December 25, 2018 11:04 AM
To: tutor@python.org
Subject: Re: [Tutor] look back comprehensively

On 12/24/18 5:45 PM, Avi Gross wrote:


> As for the UNIX tools, one nice thing about them was using them in a 
> pipeline where each step made some modification and often that merely 
> allowed the next step to modify that. The solution did not depend on 
> one tool doing everything.

I know we're wondering off topic here, but I miss the days when this
philosophy was more prevalent - "do one thing well" and be prepared to pass
your results on in a way that a different tool could potentially consume,
doing its one thing well, and so on if needed.  Of course equivalents of
those old UNIX tools are still with us, mostly thanks to the GNU umbrella of
projects, but so many current tools have grown so many capabilities they no
longer can interact with with other tools in any sane way.  "pipes and
filters" seems destined to be constrained to the dustbin of tech history.

I'll shut up now...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

Alan,

Your thoughts were helpful and gave me a hint.

Just an idea. What if you sub-classed an object type like list with a name
like chainable_list?

For most things it would be left alone. But if you isolated specific named
methods like sort() and reverse() you could over-ride them with the same
name or a new name.

If you override the function, you need to call list.sort() with whatever
arguments you had passed and then return this. If you choose a new name,
call this.sort() and then return this.

I tried it and it seems to work fine when I use a new name:

"""Module to create a version of list that is more chainable"""

class chainable_list(list):
"""Same as list but sort() can now be chained"""
def chainsort(this, *args, **kwargs):
this.sort(*args, **kwargs)
return this

Here it is on a list of ints:

>>> testink = chainable_list([3,5,1,7])
>>> testink
[3, 5, 1, 7]
>>> testink.chainsort()
[1, 3, 5, 7]
>>> testink.chainsort(reverse=True)
[7, 5, 3, 1]

Here it is on a list of strings that sort differently unless coerced back
into an int to show keyword arguments are passed:

>>> testink = chainable_list(["3","15","1","7"])
>>> testink.chainsort()
['1', '15', '3', '7']
>>> testink.chainsort(reverse=True)
['7', '3', '15', '1']
>>> testink.chainsort(key=int,reverse=True)
['15', '7', '3', '1']

I then tested the second method using the same name but asking the original
list sort to do things:

"""Module to create a version of list that is more chainable"""

class chainable_list(list):
"""Same as list but sort() can now be chained"""
def sort(this, *args, **kwargs):
list.sort(this, *args, **kwargs)
return this

>>> testink = chainable_list(["3","15","1","7"])
>>> testink.sort()
['1', '15', '3', '7']
>>> testink.sort().sort(reverse=true)
Traceback (most recent call last):
  File "", line 1, in 
testink.sort().sort(reverse=true)
NameError: name 'true' is not defined
>>> testink.sort().sort(reverse=True)
['7', '3', '15', '1']
>>> testink.sort().sort(reverse=True).sort(key=int)
['1', '3', '7', '15']

Again, it works fine. So if someone did something similar to many of the
methods that now return None, you could use the new class when needed.

This seems too simple so it must have been done. Obviously not in the
standard distribution but perhaps elsewhere. And, no, I do not expect a
method like pop() to suddenly return the list with a member dropped but it
would be nice to fix some like this one:

>>> testink.remove('7')
>>> testink
['1', '3', '15']

Meanwhile, I hear Beethoven is decomp..., well never mind! It was probably
Liszt!

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Tuesday, December 25, 2018 8:06 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On 26/12/2018 00:00, Avi Gross wrote:

> great. Many things in python can be made to fit and some need work. 
> Dumb example is that sorting something internally returns None and not 
> the object itself.

This is one of my few complaints about Python.
In Smalltalk the default return value from any method is self. In Python it
is None.

self allows chaining of methods, None does not.
Introducing features like reversed() and sorted() partially addresses the
issue but leads to inconsistent and ugly syntax.

Smalltalk uses this technique so much it has its own code layout idiom
(Pythonised as
follows):

object
   .method1()
   .method2()
   .method3()
   
   .lastone()

We can do this with some methods but not all.
And of course methods that return a different type of value require careful
handling (eg. an
index() call in the middle of a set of list operations means the subsequent
methods are being called on an int not a list - which if handled correctly
can be confusing and if not handled correctly produces errors! (The
idiomatic way says don't chain with methods not returning self!)

In practice I (and the Smalltalk community) don't find that an issue in real
world usage, but it may have been why Guido chose not to do it that way.
But I still curse the decision every time I hit it!

But as I said, it's about the only thing in Python I dislike... a small
price to pay.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Avi Gross]

[REAL SUBJECT: What's this?]

Steven,

I am afraid you are right. I was not selfish enough about this. I have done
object-oriented programming in many other languages and I am afraid today it
showed. Think C++ or Java. Part of me continues to think in every language I
ever used, including human languages. So since the name of this variable is
a suggestion, it was not enforced by the interpreter and I was not reminded.

Be happy I even used an English word and not  something like idempotent or
eponymous
.
P.S. just to confuse the issue, some in JavaScript confusingly use both this
and self near each other.
P.P.S. Please pardon my puns, especially the ones you did not notice.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Tuesday, December 25, 2018 11:39 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On Tue, Dec 25, 2018 at 10:25:50PM -0500, Avi Gross wrote:

> class chainable_list(list):
> """Same as list but sort() can now be chained"""
> def chainsort(this, *args, **kwargs):
> this.sort(*args, **kwargs)
> return this

In Python, it is traditional to use "self" rather than "this" as the
instance parameter.

Using "this" is not an error, but you can expect a lot of strange looks. 
Like a Scotsman in a kilt wandering down the middle of Main Street,
Pleasantville USA.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Avi Gross]

Asad,

I wonder if an import from __future__ happened, perhaps in the version of
collections you used. Later versions of 2.x allow optional use of the 3.x
style of print.

When you redefine print, the old statement style is hidden or worse.

-Original Message-
From: Tutor  On Behalf Of
Asad
Sent: Tuesday, December 11, 2018 10:38 AM
To: tutor@python.org
Subject: [Tutor] Increase performance of the script

Hi All,

  I used your solution , however found a strange issue with deque :

I am using python 2.6.6:

>>> import collections
>>> d = collections.deque('abcdefg')
>>> print 'Deque:', d
  File "", line 1
print 'Deque:', d
 ^
SyntaxError: invalid syntax
>>> print ('Deque:', d)
Deque: deque(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>>> print d
  File "", line 1
print d
  ^
SyntaxError: invalid syntax
>>> print (d)
deque(['a', 'b', 'c', 'd', 'e', 'f', 'g'])

In python 2.6 print statement work as print "Solution"

however after import collection I have to use print with print("Solution")
is this a known issue ?

Please let me know .

Thanks,


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Avi Gross]

Just for you Bob, I will make this short.

 

Here is the most bizarre solution as you might agree. 

 

Edit a file of python code. Use lines as long as it takes. Make sure it works.

 

Edit the file again and split lines in a way that does not mess up indentation. 
Meaning, use a normal text editor, not one that understands python. 

 

At the end of each line you cut, place a marker that makes sense. For example 
“!EOL” that may be easier to read or print (or harder) but not to edit.

 

Save all this to another plain file. It is not valid python.

 

Write a real python file that contains a function with a name like 
outclude(filename) that opens a file, reads in the entire file, replaces every 
place with the marker at the end of the line with NOTHING, effectively splicing 
it to the next line. It then returns a string containing your entire program 
with proper indentation.

 

Finally:

 

eval(outclude("filename"))

 

SKIPPING the other 98% this post could have had. Just an example:

 

DISCLAIMER: The above was posted as a sick form of humor, only. Any resemblance 
to reality is unintentional. But note, some languages do have a phase of 
scanning that would allow tricks like this in a pre-processor.

 

From: Bob Gailer  
Sent: Thursday, December 13, 2018 2:33 PM
To: Avi Gross 
Cc: tutor@python.org
Subject: Re: [Tutor] Long Lines techniques

 

On Dec 13, 2018 1:51 PM, "Avi Gross" mailto:avigr...@verizon.net> > wrote:
>
> Simple question:

Avi: when I see an email from you I tend to ignore it because it always seems 
to lead to something that is long, time consuming and complex. Would you 
consider finding ways to make your questions or comments a lot briefer?

I will be more inclined to read them if they are briefer.

You have correctly determined the conditions that will lead to continuation 
lines without backslash. I think we tend to use whatever is convenient.

In your example of a list comprehension over multiple lines there is no 
indentation. There is just a lot of white space. You might look at it this way: 
the compiler sees a left bracket with no corresponding right bracket on that 
line. So it assumes that there's more to the statement on the next line, it 
ignores the newline and just continues. Indentation is only significant if it 
starts at the beginning of a statement.

Hope this helps 

Bob gailer

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Avi Gross]

<[{SYNOPSIS: Many good answers. I am satisfied and we can move on.}]>

Steven,

I appreciate the many useful suggestions.

Many of them are what I already do. Some are in tension with other
considerations. Yes, it can be shorter and more efficient to not keep saying
module.this.that.something versus something like:

>From module.this.that import something as myname.

Of course you do that with care as you want to be careful about pulling too
many things into collisions in one namespace. Longer more descriptive names
are encouraged.

Based on reading quite a bit of code lately, I do see how common it is to
try to shorten names while not polluting the namespace as in the nearly
universal:

Import numpy as np, pandas as pd

The places I like to wrap lines tend to be, in reality, the places python
tolerates it. If you use a function that lets you set many options, it is
nice to see the options one per line. Since the entire argument list is in
parentheses, that works. Ditto for creating lists, sets and dictionaries
with MANY items at once. 

There are cases where it may make sense to have a long like connected by AND
or OR given how python does short-circuiting while returning the last thing
or two it touched instead of an actual True/False. For example, you may want
to take the first available queue that is not empty with something like
this:

Using = A or B or C or ... or Z
Handling = Using.pop()

Sure, that could be rewritten into multiple lines. 

I won't get sucked into a PERL discussion except to say that some people
love to write somethings so obscure they won't recognize it even a daylater.
PERL makes that very easy. I have done that myself a few times as I was an
early user. Python may claim to be straightforward but I can easily see ways
to fool people in python too with dunder methods or function closures or
decorators or ...

All in all, I think my question has been answered. I will add one more
concept.

I recently wrote some code and ran into error messages on lines I was trying
to keep short:

A = 'text"
A += "more text"
A += object
A+= ...

At one point, I decided to use a formatted string instead:

A = f"...{...}...{...}..."

Between curly braces I could insert variables holding various strings. As
long as those names were not long, and with some overhead, the line of code
was of reasonable size even if it expanded to much more.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Thursday, December 13, 2018 7:27 PM
To: tutor@python.org
Subject: Re: [Tutor] Long Lines techniques

On Thu, Dec 13, 2018 at 12:36:27PM -0500, Avi Gross wrote:

> Simple question:
> 
> When lines get long, what points does splitting them make sense and 
> what methods are preferred?

Good question!

First, some background:

Long lines are a potential code smell: a possible sign of excessively terse
code. A long line may be a sign that you're doing too much in one line.

https://martinfowler.com/bliki/CodeSmell.html
http://wiki.c2.com/?CodeSmell
https://blog.codinghorror.com/code-smells/

Related: 
https://www.joelonsoftware.com/2005/05/11/making-wrong-code-look-wrong/

Note that merely splitting a logical line over two or more physical lines
may still be a code-smell. Sure, your eyes don't get as tired reading
fifteen lines of 50 characters each, compared to a single 750 character
line, but there's just as much processing going on in what is essentially a
single operation.

Long lines are harder to read: your eyes have to scan across a long line,
and beyond 60 or 70 characters, it becomes physically more difficult to scan
across the line, and the error rate increases. 
[Citation required.]

But short lines don't include enough information, so the traditional
compromise is 80 characters, the character width of the old-school
green-screen terminals. The Python standard library uses 79 characters. 
(The odd number is to allow for scripts which count the newline at the end
of the line as one of the 80.)

https://www.python.org/dev/peps/pep-0008/


Okay, so we have a style-guide that sets a maximum line length, whether it
is 72 or 79 or 90 or 100 characters. What do you do when a line exceeds that
length?

The only firm rule is that you must treat each case on its own merits. 
There is no one right or wrong answer. Every long line of code is different,
and the solution will depend on the line itself. There is no getting away
from human judgement.


(1) Long names. Do you really need to call the variable
"number_of_characters" when "numchars" or even "n" would do?

The same applies to long function names: "get_data_from_database" is
probably redundant, "get_data" will probably do.

Especially watch out for long dotted names that you use over and over again.
Unlike static languages like Java, each dot represents a runtime lookup.
Long names like:

package.subpackage.module.object.method

requires fo

Re: [Tutor] Python function

[Avi Gross]

If you know what a CommaSeparatedValues file looks like, then reading ONE
LINE gives you enough info to answer the question about how many columns of
data it describes. 

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Thursday, December 13, 2018 7:47 PM
To: tutor@python.org
Subject: Re: [Tutor] Python function

On 13/12/2018 17:21, Sammy Lee wrote:
> How do I create a python function that opens a CSV file and determines 
> how many columns of data are in the file? The CSV files have been 
> randomly generated from https://www.mockaroo.com/
> 
> def csv_column_count(openfile):

You will find a bunch of stuff on creating functions in the Functions and
Modules topic of my tutorial.(see below)

You will find a bunch of stuff on handling files in the Handling Files topic
of my tutorial.(see below)

You will find a lot of help on using CSV files in the csv module
documentation in the standard library. (Or you could buy my book "Python
Projects" which has a section on the topic.)



--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Obfuscated Python [was Long Lines techniques]

[Avi Gross]

Steven,

There are dunderheads who will maliciously misuse things. Yes, we know what
__add__ is supposed to do. But if someone codes a class with one that
ignores addition to a collection if it has reached a maximum size, or does
addition modulo 16 or makes it play a happy birthday tune, while
subtracting, it may not be trivial for the innocent user of your code to
make much sense of it. 

There are documented IDEAS about what many of those dunder methods might
mean but no enforcement. 

I remember an Abstract Algebra course I took where we often talked about
operations like + and * in very abstract ways and came up with some
fascinating meanings to the operations. An example might be the various
definitions of multiplication for vectors or matrices. When you multiply,
are you multiplying corresponding parts or are you multiplying a row times a
column and summing the multiplications for each slot in the matrix? Or could
multiplication be a tad more complex and require also taking a transpose
first? Or perhaps getting a determinant or eigenvalues or eigenvectors? What
does it mean to add "1" to a complex number or quaternion or octonion? 

The reality is that two people can often try to make a similar class and
come up with different ideas and implementations. If I have a class
representing people who RSVP for a party, might a PLUS of a person result in
upping the party count by 2 since we assume they are bringing a date and if
their presence requires another table, upping that count and so on? A simple
__add__ can result in many things such as checking the added person for
dietary needs and adjusting more stuff just because adding a person can be
complex.

You re not going to sucker me into discussing obfuscation today. Some people
here got touchy last time.

I am not defending PERL nor slamming Python. I am saying people who behave
well will try to avoid writing code that is hard to read and may be
understood in multiple ways. There are cute ways to do things in many
languages.

Last comment, in what some might call stream of consciousness. Python allows
a function to return without an explicit return. Usually it returns None but
I not in some generators it may throw an exception instead. The problem is
that if you are not looking carefully at indentation, you might thing the
statements that follow may be part of the same function. So my personal
preference is to have an explicit return instead of letting it drop out of
the function or at least a comment saying this is the end of the function.
Similarly, many people won't close a file (I am not talking about in a with
statement) or delete variables no longer in use, and depend on these things
happening automatically eventually. There may be nothing wrong with that,
especially for smaller programs. But a part of me appreciates when the
scales are visibly balanced. But when overdone, as in a function where every
statement is in its own try/catch even for problems very unlikely to happen,
there is too muc detail to follow. Something in between seems more
comfortable. So interrupting an algorithm to del no longer needed variables
may not be best either. To each their own.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 14, 2018 12:36 AM
To: tutor@python.org
Subject: [Tutor] Obfuscated Python [was Long Lines techniques]

On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote:

> Python may claim to be straightforward but I can easily see ways to 
> fool people in python too with dunder methods or function closures or 
> decorators or ...

Dunder methods shouldn't fool anyone. Each dunder method has a
straightforward use, and most of the time you can guess that they do. 
__add__ implements the + operator, etc.

Closures and decorators can be abused, as can anything. One can write
obfuscated code in any language. Here's some amusing obfuscated Python:

https://www.reddit.com/r/Python/comments/i1qgp/reduce_and_lambda_two_great_t
astes_that_taste/

https://mail.python.org/pipermail/python-list/2013-October/658470.html

Anyone else want to share some obfuscated Python code?



--
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Avi Gross]

Steve[n],

Yes, I figured out what the problem was but while evaluating I also realized
that a format string was an ALTERNATIVE that automatically called for an str
or repr so in a sense it would work without debugging. 

I find that sometimes the long lines make it harder to see the skeleton of
the logic as you get bogged down by things that may be of lesser importance.

So I close with another idea. I asked about functions with lots of
arguments. An example might be making a graph using a library that lets you
specify literally hundreds of parameters all at once. I realized that a
perfectly valid alternative to make the main purpose more reasonable would
be to stuff all the extra arguments in a dictionary like

Other_args = { this: value,
That : value,
...
}
That is something in braces that can be wrapped for legibility.

Then the main function call can use the **  expansion like so:

Ret = function(from, to, **Other_args)

Of course, you can also use * with a list or other iterable for the
positional args when that makes it easier but at some point you no longer
have a feel as to what the function call is doing. But hiding less important
details this way seems to be good. Not sure about run-time efficiency, of
course.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 14, 2018 12:22 AM
To: tutor@python.org
Subject: Re: [Tutor] Long Lines techniques

On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote:

[...]
> There are cases where it may make sense to have a long like connected 
> by AND or OR given how python does short-circuiting while returning 
> the last thing or two it touched instead of an actual True/False. For 
> example, you may want to take the first available queue that is not 
> empty with something like
> this:
> 
> Using = A or B or C or ... or Z
> Handling = Using.pop()
> 
> Sure, that could be rewritten into multiple lines. 

using = (A or B
 or C or D 
 or E or F)


[...]
> I recently wrote some code and ran into error messages on lines I was 
> trying to keep short:
> 
> A = 'text"
> A += "more text"
> A += object
> A+= ...

Without knowing the error message, its impossible to say what the problem
is. My guess is that the object on line three wasn't a string. 
In which case, the obvious fix is:

A += str(object)



--
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] look back comprehensively

[Avi Gross]

There is linear thinking and then there is more linear thinking.

As Alan, Mats and others said, there are often choices we can make and many
approaches.

If your goal is to solve a problem NOW and right where you are, any method
you can find is great.

If your goal is to solve it repeatedly or in many possible places or more
efficiently or are trying to learn more about a particular language/method
like python, then you have constraints to consider.

The linear solution might be to solve the entire problem in one way. That
may be python or it may be some UNIX tool like AWK.

The flexible solutions may include doing it in stages, perhaps switching
tools along the way.

So for the logfile solution, some see two paradigms. One is to read in the
entire file, no matter how big. The other is to read in no more than a line
at a time.

Bogus choice.

In the logfile example, there are many other choices. If you can recognize
the beginning of a region and then the end, sure, you can buffer lines. But
how about a solution where you simply read the entire file (a line at a
time) while writing a second file on only the lines you might need. When
done, if the file is empty, move on.

If not, open that smaller file, perhaps reading it all in at once, and
process that small file containing perhaps a few error logs. Or, heck, maybe
each error region was written into a different file and you process them one
at a time with even less complex code.

Unless efficiency is paramount, many schemes like these can result in a
better division of labor, easier to understand and perhaps even code. And
some of these schemes may even have other advantages.

Here is yet another weird idea. REWIND. If the log file is a real file, you
can wait till you reach the end condition that tells you that you also need
earlier lines. Make a note of your position in the file and calculate an
estimate of how far back in the file you need to go, perhaps a very generous
estimate. Say you rewind a thousand bytes and start reading lines again,
perhaps discarding the first one as likely to be incomplete. You can read
all these lines into a buffer, figure out which lines you need starting from
the end, do what you want with it, toss the buffer, reset the file pointer,
and continue!

That solution is not linear at all. If you have a huge log file with a
sparse (or nonexistent) number of errors to process, this may even be faster
than a scheme which buffers the last hundred lines and is constantly rolling
those lines over for naught.

I am not criticizing any approach but suggesting that one good approach to
problems is to not close in on one particular solution prematurely. Be open
to other solutions and even think outside that box. There may be many ways
to look back. 

As for the UNIX tools, one nice thing about them was using them in a
pipeline where each step made some modification and often that merely
allowed the next step to modify that. The solution did not depend on one
tool doing everything.

Even within python, you can find a way to combine many modules to get a job
done rather than building it from scratch.

Which leads to the question of how you would design a log file if you knew
you needed to be able to search it efficiently for your particular
application.

I would offer something like HTML as an example. 

To some extent, the design of many elements looks like this:


...


The idea is you can write code that starts saving info when it reaches the
first tag, and when it reaches the second tag that ends it, you make a
decision. If you are in the right region, process it. If not, toss it and
just move on.

Of course, this scheme does not actually work for many of the tags in HTML.
Many tags allow an optional close but tolerate not having one. Some things
may be nested within each other.

But when it comes to log files, if some line says:

***

And that marks any error and you now wait till the end of the region to see
which error on a line like:

#ERRNO: 26

Then you can ask your code to ignore lines till it sees the first, marker.
Buffer any subsequent lines till you recognize the second marker and process
the buffer. Then go back to ignoring till ...

A real logfile may contain many sections for many purposes. They may even
have interleaved lines from different processes to the point where your
design may require all lines to start with something unique like the process
ID of the writer. This would make parsing such a file very hard, perhaps
requiring multiple passes sort of like described above. So sometimes a
better design is multiple log files that can be merged if needed.

Of course if you must use what exists, 

-Original Message-
From: Tutor  On Behalf Of
Mats Wichmann
Sent: Monday, December 24, 2018 10:15 AM
To: tutor@python.org
Subject: Re: [Tutor] look back comprehensively

On 12/24/18 2:14 AM, Alan Gauld via Tutor wrote:
> On 24/12/2018 05:25, Asokan Pichai wrote:
> 
>> That said, sometimes text processing at the 

[Tutor] Long Lines techniques

[Avi Gross]

Simple question:

 

When lines get long, what points does splitting them make sense and what
methods are preferred?

 

Details.

 

I am used to many languages where you can continue a statement across
multiple lines. What they share in common is the fact they do not use
indenting for the use Python makes of it. They often use something like
"{.}" to combine multiple statements into a single body and carriage returns
are generally ignored as multiple lines are parsed to search for thee end of
the compound statement at the "}" . In many languages, parentheses and
square brackets and even angle brackets can serve a similar function. Some
languages that insist on ending a statement with ";" are even looser. Many
treat a trailing comma or plus sign (and other symbols) as a hint to
continue reading another line.

 

So I would like some feedback on ways to deal with longer statements in
Python.

 

Yes, I am aware of ways to break up something long by breaking in into
multiple statements. It feels silly to do this:

 

A = "long string"

B = "another string"

.

Z = "last string"

 

When you meant to write one long string in the first place or connect a
bunch of them with adjacency or a "+"

 

By breaking them into snippets like above, and using short variable names,
you can combine them like this:

 

Combined = A + B + . + Z

 

Or even worse:

 

Combined = ""

Combined += A

Combined += B

.

Combined += Z

 

The above was ONE example of many where I bend my code in ways to make the
lines reasonably short so reading and editing are easy.

 

There are places you can break lines as in a comprehension such as this set
comprehension:

 

letter_set = { letter

   for word in (left_list + right_list)

   for letter in word }

 

The above is an example where I know I can break because the {} is holding
it together. I know I can break at each "for" or "if" but can I break at
random places? Here is a bizarre example that worked:

 

>>> [ x *

  y

  for x in

  range(1,5,2)

  for y

  in range(2,

   6

   ,2)

  if x + y <

  9

  ]

[2, 4, 6, 12]

 

So clearly when you use [.] it ignores indentation until it gets to the end.
I see that parentheses sometimes offer a similar protection as in the new
print function:

 

>>> print(1,

  2

  ,

  3)

1 2 3

 

But python has a few features like allowing a comma at the end of a tuple
that get in the way:

 

>>> x,y = 1,

Traceback (most recent call last):

  File "", line 1, in 

x,y = 1,

ValueError: not enough values to unpack (expected 2, got 1)

 

DUH! I was going to put the rest on the next line!

 

So I added parentheses and that does the trick here.

 

>>> x,y = (1,

   2

   )

 

Similarly, it is a syntax error if I simply end with something like a +

 

>>> x = 1 +

SyntaxError: invalid syntax

>>> x = (1 +

2)

>>> x

3

 

I AM NOT SAYING THIS IS HORRIBLE, just asking what pythonic ways are best to
use to get around things.

 

One that seems not to be favored is a backslash at the end of a line:

 

>>> x = 1 + \

2 + \

3

>>> x

6

 

Although my examples use short numbers, remember I might be using this with
long lines and not want to mess up the overall indentation. Strings have a
similar issue:

 

>>> x = "hello

SyntaxError: EOL while scanning string literal

>>> x = """hello

world"""

>>> x

'hello\nworld'

>>> x = "hello \

world!"

>>> x

'hello \tworld!''

 

As shown, the triple quotes sort of work but introduce and keep carriage
returns and formatting inside. The backslash at the end suppressed the
former but keeps the latter. Yes, there are routines you can pass the string
through afterward to remove leading whitespace or carriage returns.

 

I will stop here with saying that unlike many languages, parentheses must be
used with care in python as they may create a tuple or even generator
expression. Curly braces may make a set or dictionary. Options for splitting
the code without messing up indenting cues would be nice to understand.

 

 

 

 

 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PEP 572 -- Assignment Expressions

[Avi Gross]

Ivo,

One thing you could have done is explore with simpler code to see if you can
deduce what is happening.

 If you opened your Python interpreter and tried to see what happens with a
simplified variant like this, what do you get?

if (match = 5) is not None: pass

That might answer your question on why a := might introduce new
functionality.

Something similar would work fine in a language like C/C++. Heck, it is used
often as a concise side effect.

Both languages have an == to be used in expressions but with a meaning of
COMPARE, not set equal to.

I guess someone thought that situations like matching a regular expression
would benefit for the side effect. There are workarounds, of course. As an
example, in SOME cases, the value of the last expression can be found
momentarily in the special variable consisting of a single underscore. But
the expression above probably would evaluate to True, ...

Ovi


-Original Message-
From: Python-list  On
Behalf Of Ivo Shipkaliev
Sent: Tuesday, November 27, 2018 5:48 AM
To: python-l...@python.org
Subject: PEP 572 -- Assignment Expressions

   Hello.
   Maybe it's too late for a discussion, but I just couldn't resist.
   I just found out about this new ":=" operator. I need to ask:

   What is the need for this additional ":" to the "="?
   Why:
 if (match := pattern.search(data)) is not None:
   # Do something with match

   What is wrong with:
 if match = pattern.search(data) is not None:
   # Do something with match

   Assignment expression or assignment statement, it's an assignment, right?
It is very clear to everyone that it's an assignment! Can't it all just be a
"="?
   Thank you very much!


   Kind Regards
   Ivo Shipkaliev
--
https://mail.python.org/mailman/listinfo/python-list

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] writer overloaded

[Avi Gross]

 
identical and actually aren't.

You may have also missed my last point. Short circuit evaluation is old. That 
is not what I consider unusual in python. What I found to be different (and it 
is possible other languages had or now have it) is that it does not bother 
jumping into a mathematical Boolean mode. That is deferred indefinitely as not 
important. The result of quite a few operations is neither True nor False but 
just about ANYTHING. To explicitly make it force a Boolean result you need to 
call bool(A or B and not C) which otherwise simply return A or B or C, whatever 
they are and the truthiness of it is determined by either it having a length of 
zero, or being numerically 0 or whatever the __nonzero__ (pre 3.X) or __bool__ 
methods for that object return, perhaps recursively if they return another 
object.

It turns out, mathematically, to be valid and work in an amazing number of 
contexts and even where it might not, you can probably use the __bool__ method 
to make something that works. 

Sorry to be this long. I could have summed up my entire previous message in a 
few sentences but then you might have said I was too broad and showed no 
reasons or examples. As I said, life is short.

-Original Message-
From: Tutor  On Behalf Of Steven 
D'Aprano
Sent: Wednesday, November 28, 2018 6:15 PM
To: tutor@python.org
Subject: Re: [Tutor] user overloaded

On Wed, Nov 28, 2018 at 12:56:32PM -0500, Avi Gross wrote:

> try ...
> except ValueError, e:
> ...
> 
> You now need to do this format:
> 
> try ...
> except ValueError as e:
> ...
> 
> Why the change?

Because the first form was a trap and a bug magnet:

try:
block
except TypeError, ValueError, KeyError:
print "surprise! this will not catch key errors!"



[...]
> Back to your topic. "=" as a symbol has been used over the years in 
> many ways.  In normal mathematics, it is mostly seen as a comparison 
> operator as in tan(x) = sin(x)/cos(x)

That's not a comparison, that's an identity. The tangent function is (or at 
least can be) defined as sine divided by cosine.


> Many programming languages used it instead to sort of mean MAKE EQUALS.

That's usually called "assignment" or "binding".


> x = x + 5
> 
> is an example where it makes no mathematical sense. How can a number be
> equal to itself when 5 is added? 

There is no solution there in the normal real number system, but there 
are forms of arithmetic where it does, e.g. "clock arithmetic". If your 
clock only has values 0 through 4, then adding 5 to any x will give you 
x back again.

Although normally we write "congruent" rather than "equals" for clock 
arithmetic: 

x ≡ x + 5 (mod 5)


In any case, just because there is no solution doesn't mean that the 
question makes no sense:

x = x*5

makes perfect sense and has solution 0. Likewise:

x = x**2 + k

makes perfect sense too, and can have zero, one or two solutions 
depending on the value of k.



> The meaning clearly is something like x
> gets assigned a value by evaluating the current value of x and adding 5 to
> that. When done, the old value of x is gone and it has a new value.
> 
> For this reason, some languages like PASCAL used := so the above became 
> 
> x := x + 5
> 
> Yes, same symbol we are discussing and NOT what Python intends.

Actually, it is precisely what Python intends, except for two minor 
differences:

1. in Pascal, := assignment is a statement, not an expression, 
   while in Python it will be an expresion;

2. Python will discourage people from using the := expression
   form as a de facto statement, where = will do.


[...]
> And there are forms of equality that some languages need to distinguish.
> What does it mean if two (unordered) sets are equal? Must they be two
> reference to the same set or is it shorthand for them being strict subsets
> of each other in both directions? You can come up with additional
> abstractions like whether a list is a shallow or deep copy of another and
> how it effects your ideas about them being equal. You may want a way to say
> things are approximately equal. In math, something like ~= is used. Python
> has an operator called "is" that can be a variant on equality.

The "is" operator is in no way, shape or form an equality test. It is an 
*identity* test, which returns True if and only if the two operands are 
the same object.

Equality does not imply identity:

py> x = []
py> (x == []) and (x is not [])
True


nor does identity imply equality:

py> NAN = float(NAN)
py> (NAN is NAN) and (NAN != NAN)
True

Anyone using "is" when then test for equality is programming in a state 
of sin.


> Consider the concept of OR. Some languages spell it out when used in a
> Boolean context. 
> 
> a or b
> 
> But python does something new 

Re: [Tutor] online interactive Py resource?

[Avi Gross]

Mats,

This may not be a direct answer to your question but I have used more
generic tools in a way that might make some sense.

You can use a videoconference tool such as SKYPE to connect and have one of
you share your screen with the otgher so you can see whatever is open such
as an editor they are editing code in and another running python
interactively as well as any pop-up windows and so on. You can talk them
through things like change the next line ...

You can also have part of the screen on which you can do text chat such as
sending them some text to paste into their program or even push a download
of files or receive them.

Not quite the same shared editing experience but there are ways to use
something like Google Docs to collaborate and perhaps even set up to run
python directly on the remote file or just copy/paste it. Of course, the
shared editor would ideally not be a full-blown text processor or even
understand python code.

There are also ways to take over their computer (with permission) and have
it accept your commands as you demonstrate things.

I recall a web site I once used in a course that allowed me to crerate and
save python code that it ran remotely:

http://www.codeskulptor.org/

Each time you save a file, you get a URL like this:

http://www.codeskulptor.org/#user45_M9Oefx1vom_0.py

If you click on that URL it should show the minor code I made and let you
edit it and then I can click and see the change. This being an open forum,
multiple people may do so. 

On that screen, the leftmost small menu item near the top runs the code with
output on the right panel. The second saves the current code under the same
URKL that you maybe would bookmark. Other controls let you save a local
copy, etc.

Note there is a version 3 version to use at:

https://py3.codeskulptor.org/

Some of these techniques may be helpful for anyone wanting some interaction
but I do not claim they handle well having two people edit files at the same
time. You can always GIT.

Avi



-Original Message-
From: Tutor  On Behalf Of
Mats Wichmann
Sent: Wednesday, November 28, 2018 8:50 PM
To: tutor 
Subject: [Tutor] online interactive Py resource?


I'm going to end up tutoring someone (family member) learning Python in the
new year.  Look forward to it. It will be a remote arrangement.
Haven't trained anyone in the whole picture (as opposed to bits and bobs of
possibly helpful advice on demand, like here) since I wrote and delivered a
training course for a commercial training company, the last instance of
which was pre-historic in tech terms (2002 in calendar terms). That those
were very much hands-on, in-person.

It occurs to me it would be cool to work interactively in a distributed
Internet editing environment that could run Python code.  Are there such?
There are distributed editing environments, like Etherpad, Gobby,
(cough) Google Docs.  There are online interactive Python environments, such
as the ones from the nice folks at PythonAnywhere.

is there something that combines the two?  I'm not completely sure
PythonAnywhere doesn't have that capability... but I'm looking for some
wisdom and experiences (and maybe best practices) from the other tutors
around here.

cheers,

-- mats
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Copyleft

[Avi Gross]

Alan,

Just a reminder. I am NOT suggesting the first way to solve a problem is what I 
described. I am saying that sometimes a roadblock (real or imagined) can be 
gotten around using methods like that.

Agreed. There are copyrights that have to be considered, albeit often SMALL 
amounts are considered OK. Attribution is a good idea in general.

The problem that began this is when someone puts roadblocks in the way of what 
works.

If someone for example wants to be able to use something as trivial as math.pi 
but for some reason won't load the math module, you could suggest they look up 
the value of pi to umpteen decimal places and create an object called math 
Perhaps a class) containing nothing but a value called pi 

This might even satisfy other code they have imported that expects to be able 
to invoke math.pi. But if that other code does an import too, then you might 
need to create an (almost) empty file called math.py containing 
pi=3.1415926535...

Just to make the rest work.

Or, I suggest you can copy the real math module into the same place. You may 
want to edit it with a comment.

But, yes, bad example as pretty much any implementation of python will include 
the math module as it is standard.

Your post does remind me that there are many versions of copyright including 
some with the silly naming of "copyleft" that allow rather free use but with 
limits. They may charge large businesses or government agencies. They may 
demand you leave in some text as in comments or not make changes or notify them 
to get permission so they can keep track or ...

I brought up some questions a while ago that boiled down to how a program can 
know what is already available on the machine the code is being run on. Clearly 
if you use an unfamiliar package, one solution would be to bring the needed 
files with you in the distribution of the program. Yes, it may not be the 
latest or greatest, but you can imagine many ways where it can conditionally 
fall back on the copies included if a regular import fails. 

This may be one reason someone would be asked to limit their code to the 
standard distribution. 



-Original Message-
From: Tutor  On Behalf Of Alan 
Gauld via Tutor
Sent: Tuesday, December 4, 2018 6:43 PM
To: tutor@python.org
Subject: Re: [Tutor] Borrowing free code

On 04/12/2018 19:31, Avi Gross wrote:

> But some packages are simply python code that you can simply insert 
> into your own python files.

If they are fully public domain that's probably true.
If they are copyright (even if open/free) you would be potentially liable for 
prosecution since you are copying someone else's work.

Even if it is open source then at the very least you should include a comment 
to the effect that the code is based on, say, M Palin's file parrot.py or 
whatever.

> And, yes, this means you do not get updates if the module changes.

And this is a big drawback for any non trivial code unless you are a 
significantly better programmer than whoever wrote it in the first place. 
(Since understanding somebody else's code is much harder than understanding 
your own!)

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Avi Gross]

[[ Real SUBJECT: emulating decorators ]]

Steven,

I am not suggesting that a particular way of reusing a name is a good idea.
I am asking if some ways are encouraged and others are discouraged in the
python community. 

Just to clarify one point you asked about:

#  > I have often seen something like this done with methods, such as to
emulate
#  > decorator functionality where a method is created in an object with a
name
#  > and the very next method created has the same name with the same
function
#  > name as an argument to replace it.
#  
# I don't understand this. Can you show an example? Even if made up?

The answer is that I saw that mentioned in a Lutz book that I no longer
have. What I wrote above should be clear if I give an example:

Say you have a method in a class definition called do_this() that does
whatever this is.

You may also have other methods in the same or different classes with names
like do_that() and your programs run fine.

At a later point you decide you want to see how long it takes the method to
run. So you make or borrow a function called timing_is_everything() that
looks like this in pseudocode:

def timing_is_everything(fun, args):
save the current time
call fun with args
save the return value of calling fun
get the current time
calculate time elapsed and do whatever with it.
Return saved return value from fun.

The argument "fun" above can be replaced by any function such as "do_this"
or "do_that" and produce the same side-effect while also running the
function unharmed.

Then you decide you want a function that logs when another function is
called and with what arguments:

Again, in PSEUDOCODE, it looks like:

def logistics(fun, args):
with open(logfile):
 write info about the current date/time, name of the function and
arguments
return result of calling the function with those args

Now to the point. You could write each and every module you want
independently. At the top of the body you could insert any similar code to
begin timing it or to log it. Then you place the code needed for that
function followed by anything you want done after such an action and finally
return a value, if needed.

But if you already have written and tested something like the two modules
above, you can do something like this:

class classy(object):
...
# Original function first of this name
def recreational_fun(args):
...

# New function, second of this name
recreational _fun = timing_is_everything(recreational _fun, args)

# New function, third of this name
recreational _fun = logistics(recreational _fun, args)

AGAIN, the above is for illustration only and I am well aware of the proper
ways to do the pseudocode such as using notations like *args, **kwargs and
so on. We are talking about an IDEA that perhaps predated the introduction
of decorators way back when. The point is that you could have used three
names here for the various functions but realistically, only the last was
needed so why carry around extra methods.

With "decorators" as a python feature allowing you to wrap multiple
functions around the one being defined, assuming you built the above helper
functions properly so they can be used as decorators, the above might look
like this:

class classified(object):
...
# Original function first and only of this name 
# and highly decorated bottom to top

@ logistics
@ timing_is_everything
def recreational_fun(args):
...

Both ways of doing this, again if PROPERLY done, should have the same
result, albeit there may be subtle changes such as hidden within the dunder
properties. There will be a single function name visible to any user. When
called, it will log an entry, note what time it is, do whatever the
re-created method is supposed to do, then calculate how much time it took
and return some result.

And, yes, I know there is overhead added as you end up with layered function
calls and so on.

This was an answer to a request. I hope I answered it. I do not wish to now
continue on this topic. The topic was not decorators. But the decorator
functionality used above likely does not confuse anybody as to what the name
of the method is. That variable name is only seen once as compared to my
first example when it refers to three completely different functions and
keeps giving up a connection to what effectively becomes an anonymous
function that can only be remembered and called within the enclosing
function.

I tend to agree with the other points Steve made.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Avi Gross]

I left the "subject" above to be the same, as requested.

The actual subject might have been "Pythonic variable name use"

Alan says he had a challenge evaluating code (below) because the same
variable names were reused and it made me wonder if the python community has
general ideas about name re-use.

I am keeping this short. In languages without garbage collection, reusing
the same name, like "index" repeatedly might save some small amount of
space. With garbage collection, reuse may be one way to hint it can be used
again but if that is important, you can use "del var" to remove var.

For short programs, or in a context like a "normal" function that goes away
when completed, it probably is best to make the code very readable. PLEASE
do not mention the many exceptions as I do not intend much of what I say to
be taken as completely accurate.

But so much code I see in python does not only reuse the same variable names
but in a confusing way.

file =  "some name"
file = open(file, "r")
file = some_wrapper(file)

mylist = [ ... ]
mylist = sorted(mylist)

for index in range(...):
  stuff

for index in open(...)
  more stuff

for index, index2  in enumerate(...)
  yet more stuff

I have often seen something like this done with methods, such as to emulate
decorator functionality where a method is created in an object with a name
and the very next method created has the same name with the same function
name as an argument to replace it.

So is there a guide on when reuse is good and when it just obfuscates? What
is good practice?

-Original Message-
From: Tutor  On Behalf Of
Alan Gauld via Tutor
Sent: Thursday, December 6, 2018 7:26 PM
To: tutor@python.org
Subject: Re: [Tutor] Regarding Python api script

On 06/12/2018 14:17, Ravi Kumar wrote:

> 1)The for loops that have written I am able to access all the 
> networks,able to loop through  all access points(Devices) in the 
> network,able to loop through and get all clients in each access points 
> but when it comea to client log events I am able to loop through and 
> get only  the last row of accesspoints  list and get those particular 
> and clients and its log events

I don't know the cause, but did take a quick look at the code.
One thing that makes it very difficult to follow is that you use the same
variable names over and over making it hard to keep track of what any given
'item' or 'json_string' or 'r' actually holds at any given time.

Buy using  a different name for each json_string reflecting the expected
data - such as json_networks or json_clients - it would be very much easier
to follow the flow of the code.

> I assume I am going wrong in the last for loop code and output as 
> shown below

Probably but I confess I couldn't spot it from a casual read through.

> for client in json_string:
> 
> hostname = client.get("dhcpHostname")
> description = client.get("description")
> ipaddress = client.get("ip")
> macaddress = client.get("mac")
> 
> usage = client.get("usage")
> sentbytes = usage.get("sent")
> recvbytes = usage.get("recv")
> 
> print ('{0:20}   {1:20}   {2:20}{3:30}   {4:20}
> {5:20}'.format(hostname, description, ipaddress, macaddress,sentbytes,
> recvbytes))
> 
> 
> 
> for  item in serialnumlist:
> print ("\nQuerying Meraki for clientlogevents on Devices: (" +
> item.get("mac") + ")")
> 
> for item  in json_string:
> config_url = "/networks/"+"/N_63***1050/"+"/clients/"+ 
> item.get("mac")
> + "/events?perPage=1000"
> r = requests.get(base_url + config_url, headers=headers)
> 
> print ('{}  '.format(r.content))
> **>

The 'item's in the last loop appear to be the same as the 'client's in the
previous loop? Since 'json_string' never changes...

Also the setialnumlist only prints the result it never stores anything.

Then the last loop uses item.get(mac) which will not be the one printed
earlier since item is now read from json_string(back tome earlier confusion
over names!)

I suspect this lack of continuity may be the root of your problem but the
name collisions are twisting my brain and its late at night...


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Borrowing free code

[Avi Gross]

David,

What does it mean when someone says they cannot install a module? I can see how 
a school assignment might require using only some limited set of functionality. 
I note some installations add more than others or let you designate optional 
components to include.

Some companies may have security concerns or portability considerations. The 
finished product may be used on machines lacking what is needed or even cut off 
from the ability to get them.

But some packages are simply python code that you can simply insert into your 
own python files. If they have few external dependencies and no need to compile 
C code, can you copy them without violating rules?

I am thinking for example of a module that defines a class you may want to 
subclass. You could find the source code you would have imported and take 
relevant parts into your main code file or into any local files you import. 

I do not know if the module being discussed meets the criteria or if this would 
not  be considered honest. But it is a step to be considered. Another thread I 
have seen related to a user with no internet connection for downloading. But 
sneakernet is trivial if the only thing needed for an install might be copying 
a file or three into any valid location on the machine including the directory 
your main program is in.

And, yes, this means you do not get updates if the module changes.  

Just a thought.


Sent from AOL Mobile Mail
On Tuesday, December 4, 2018 David Rock  wrote:

> On Dec 4, 2018, at 12:05, Asad  wrote:
> 
> Hi All ,
> 
>          I am not allowed to import pexcept  .Therefore only option I
> have is to implement a solution using the standard libraries in python .

I’m only suggesting it because it’s an easier way to interface with expect.  If 
you aren’t allowed to install it, then we’ll stop talking about it.

>            However I am using subprocess.Popen for sftp I am unable to
> pass the password.
> 

You still are not understanding my point.  using sftp in batch mode CAN’T USE a 
password.  If you look at the manpage for sftp, batch mode prohibits it.

    -b batchfile
            Batch mode reads a series of commands from an input batchfile 
instead of stdin.
            Since it lacks user interaction it should be used in conjunction 
with non-inter‐
            active authentication.  <<—

“non-interactive authentication” means you must use an ssh key to authenticate 
instead of a password.  In other words, trying to figure out how to send a 
login to the subprocess is not the issue; even if you can, it won’t work 
because sftp *in batch mode* will not accept it.  If you _can’t_ set up an ssh 
key for this access, then you MUST stop trying to use batch mode, because it 
will never work.

>  Here I am unable to proceed , I am sure someone would have cracked this
> problem earlier if they can share the code

There is no code for this because it is not possible to do within the 
constraints you have proposed up to this point.

1. Can you set up ssh key passwordless authentication?
  if yes, then do it and what you have now will start working
  if no, then you can’t use sftp the way you are currently trying (the -b 
“batch mode”) and find a different solution (expect, here docs, etc).


— 
David Rock
da...@graniteweb.com




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Borrowing restricted code

[Avi Gross]

I don't want to start (and continue) another sideways discussion. Alan and
Steven (not Stephen) both made very good points and helped clarify my
understanding.

I am NOT advocating copying code. Not even "free" code.

I am saying there may be times you want to package the code for special
purposes.

Perhaps someone can enlighten me on a subtle aspect here.

What does it mean to include something?

If we return to the days of yesteryear, albeit much of it exists this year,
a major programming paradigm was to compile a language like C or PASCAL into
some machine language or a binary form. But you only wrote a part of the
final program and inserted parts from what had been written, mainly, by
others.

One inclusion in languages like C was variants of lines like this:

#include "stdio.h"

What would happen was when a phase of the compile process read your file and
saw a request to include, it would pause, try to find the file in the usual
places, and then copy the text and continue. This could be recursive if that
file included another. It is in one sense similar to how "import" works in
python or "library" in R and similar features in other languages.

What was different was that one large file was created which was a flattened
version of this tree of code. Another pass of the compiler would then
process it. Nothing was RUN until later and only if there were no
compilation errors. Along the way, another form of inclusion had to happen.
If your code (directly or indirectly) called functions not defined in your
own code, and not already compiled in, it had to search assorted libraries
of such code, find the segments needed, and link them in. Eventually, it got
even worse and shared libraries would be mounted in memory and when your
code ran, it would link in what was needed.

The above is not meant to be a precise description but indicates that your
code was rarely all your own. You could literally set it up so if someone
else left their code unguarded, you could arrange to grab text or library
code into your finished executable.

In python, you can generally have access to the python code of what you can
import or at least to the byte code. But there may be different rules
attached to several categories.

If you use "import" you don't so much copy as USE the code. I mean the
interpreter pauses evaluating the current file and opens the one you asked
for and reads it as if it were your code. Some actions are transient. A line
like "5+3" evaluates to 8 and you ignore it. But many and perhaps most lines
end up creating instantiations of objects (yes, classes are also objects)
which then sit in memory. Functions are also objects and pretty much contain
within them everything you need to reconstruct the function if you know
where to look. Is what is copied really different than the original text
used? 

If you import the base modules that come with python, can you assume it is
freely given with few strings other than the kind discussed? If it is
outside but available from a standard download location, perhaps you need to
see the documentation for each item. If it is sold to you, I suspect you
need to be very careful about including any.

Back to the main topic, for me. What kind of uses might be considered legal
or at least not worth prosecuting? The law in various countries likely
differs, Alan and I are definitely using different legal systems but
possibly not as different as where Asad is. (His phone number would be in
India.) Clearly if you make a product and sell it, then borrowing code can
be a serious thing. If you are a student doing a homework assignment and
will never use it again, few might care especially if you use small amounts
and don't claim it is your own work.

But the question I would like answered is DOES IT MAKE A DIFFERENCE how you
borrow the use of the code? 

We talked about the normal intended method:

Load the file into a place on your local machine.
Write one or more python files and one of them imports it.

A second method I suggested WHEN NEEDED AND ALLOWED is to find the code you
need, perhaps on another machine where you can download it, or in some other
public repository. Copy the minimum you need into your own code, with
perhaps enough attribution and explanation.

Now what about a third way? No edit needed. Simply use a program like "cat"
to concatenate multiple files including the one with your code into a bigger
lump. Feed that to the python interpreter.  Included would be the code you
want. True, the code would now all be in one namespace. But is this form of
copying all that much different than was intended? 

I am not going to continue in any further discussion. But I do think this
could be useful to people learning python as often the best way to find out
how to do something is to study the code of someone who did it. You might
want to play with it and see if you can improve it or vary some part ...

So it would be good to know what can be done legally and what to avoid.

As 

[Tutor] Born to be free

[Avi Gross]

Alan,

Yes, I meant that many things are effectively free these days. Some things are 
required to be distributed free by CopyLeft. But you can pay a nominal fee for 
say a CD of the software. You can pay for a bundle like you describe, perhaps 
including some consulting or warranties to keep you updated or who knows what?

I have been using a slew of programs lately in work for someone else who gave 
me access to software they paid for under some license. I mean statistical 
tools that are sold or licensed and have been around for ages. But I find I can 
cobble together my own programs and often find packages for free to do some 
parts. But as noted, I often have to process data that came from sources like 
SAS, MPLUS, STATA, SPSS and others. I often have to return the results of the 
calculations back in those formats. I have not felt the urge to buy the 
software but the day may come. My goal is to use a variety of tools that 
include R and Python unless the easiest path is ...

I am not in the market for buying or selling such things so I simply admit my 
lack of encyclopedic memory and experience.  I suggested that based on my 
knowledge, to date, I gather Python is mostly available for free and this might 
impact companies that wish to bend it to their needs rather than paying someone 
to give them an expensive tool. 

Many languages and other shared goods have ISO committees or Standards Bodies 
that document it or regulate changes. In the past, I often found companies like 
AT and HP attending all kinds of such bodies to influence the direction they 
take or be informed where they are headed. 

Some people have a photographic memory while some have a pornographic memory. 
Anyone with both is beyond dangerous 

-Original Message-
From: Tutor  On Behalf Of Alan 
Gauld via Tutor
Sent: Thursday, November 22, 2018 7:54 PM
To: tutor@python.org
Subject: Re: [Tutor] origins bootstrapped.

On 22/11/2018 06:05, Steven D'Aprano wrote:

> I don't know of any non-free (free as in beer, or free as in speech) 
> implementations of Python. Can you elaborate?

There are several commercial distributions (as opposed to
implementations) of Python, that may be what Avi has in mind.
Some of these are commercial IDEs that include python as part of an integrated 
bundle - I think Blackadder is one such - and others are just uber distros like 
Enthought Entropy(?) which is a "supported" distro for scientific work - rather 
like Anaconda.

Others are in the Movie industry where it is either tied to a particular tool 
or again to a support arrangement.

The implementations are the standard open source code but the distribution is 
paid for, with the value add either in the toolset, the packaging or the 
support.

But maybe Avi means something different...

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] A required question

[Avi Gross]

Just to be different, and perhaps return to the purpose of this group, I have a 
question.

 

Is there some functionality available in Python that you could put in the 
beginning of a program so it aborts with a message if the version of R being 
run is not acceptable?

 

Let me elaborate. There seem to be new features added as you move up. I mean 
2.2, 2.3, … 3.0 and so on.

 

But there also seem to be features deprecated then removed.

 

To make it crazier, some features are back-ported into later releases of 2.X 
while some features are activated only in you import the right things from 
__FUTURE__ .

 

Add to that the fact that various modules may exist or not exist on your 
machine in your search PATH and they may import yet others, and it gets crazy. 
Even weirder is that you can probably download a missing file as the program 
runs and then import it and probably many other scenarios where you own program 
writes a module and then imports it!

 

What I was thinking was the ability to do something like this:

 

import ReChoir as require

 

require.version(condition, before=True, after=False)

require.modules(module list, recursive=True)

require.os([“Eunuchs”, “Windblows”])

require.functionality(“print3”)

 

I hope you get the idea. Any one of the statements above might be made anywhere 
in your program but is best near the top of the main program perhaps after any 
imports from the dunder FUTURE.

 

I am not designing it. I am giving a very silly example and also suggesting a 
possible scenario like this:

 

if ! require( <> ):

print(“Sorry, your system is too primitive to use this snazzy 
software.”)

print(“In a moment, you will be re-directed to a dumber version 
written”)

print(“stone age tools. Please upgrade son.”

print(“running program paleo.py for you with same arguments.”

sleep(5)

exit( os.system(<<<“call to python to run paleo.py with same 
args using argv …”>>>) )

 

So in some cases, you quit with a message saying why you quit. In the example 
at the end, you may call alternate functionality, perhaps a shell-script rather 
than python, that does something for them. In an extreme case, the script would 
download a later version of Python as a sort of bootstrap. 

 

Yes, tons of holes in here and implementation details. For example, some 
features are only needed if the command line or data lead you down some path. 
No need to abort most of the time and not easy or even possible to see if you 
might need it.

 

So is there anything interesting out there already, designed to not so much 
make python portable, but to prevent things from running and then dying in 
middle of something important. 

 

Yes, I know there is an assert command you can use and there are system 
variables you can query to see what version of Python you are running on what 
OS and so on. No doubt there are tools you can use to see if everything 
imported exists (maybe not the right file) somewhere in the path. The latter 
alone is a huge issue as there have been changes in how you import including 
some really odd ones where they broke old functionality. 

 

There may come a day when 2.X is as extinct as polio (stubbornly hanging in 
there as some refuse to vaccinate) but I do not foresee the march of changes 
slowing in 3.x, let alone 4.x^2 and beyond.

 

Such a tool might make it easier for people to use new features with less fear.

 

One more thought. Such a function set might simply try new features inside a 
“try” and catch the error so they can gracefully annoy the user with a snide 
remark and even tell them what is missing. For example a line with a := in it 
should fail everywhere now but might work later. An f”string” is not that old. 
A line using open() in an iterator context would fail somewhere back in 2.x. In 
theory, you can make a list of testable new features and even give them names 
you can ask for such as the “print3” above.

 

Why do I ask? I had a great day today except it followed a horrible day 
yesterday. I did a bunch of work in R but the main point could just as easily 
happen in Python. I wanted various packages to do various things. Some had 
broken for me in the last year or so and been replaced with something that 
refused to work. In English, it was supposed to take a linear model generated 
from data and calculate some predicted info from it and make a nice graph with 
lines showing the impact of another variable at the mean and +/- a standard 
deviation. The problem is that I had the latest version of R installed but some 
of the many packages required were not yet available or their dependencies were 
either not available or of the wrong date. This happens with Python modules 
too. With lots of research, I loaded some things from alternate places and 
backed up to older versions and then forward and eventually had a brainstorm of 
figuring out what to 

Re: [Tutor] A required question

[Avi Gross]

ne 29, in 
var = imp.find_module(modulosity)
  File 
"C:\Users\avid2016\AppData\Local\Programs\Python\Python37-32\lib\imp.py", line 
297, in find_module
raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'koala'

Amazingly it failed!

Calling it a night as I find errors even after I rewrite it like this;

modulosity = "SimPy"

print(" importing " + modulosity + "By hook or by crook.")

import imp
try:
var = imp.find_module(modulosity)
print ("FOUND: " + modulosity + " at " + str(var))
except:
print("NOT FOUND")
print("Module needs loading: " + modulosity)
import os
os.system("python -m pip install " + modulosity)

exec(f"import {modulosity}")

I am sure something is doable and can be made into a proper function. Yes, I 
cheated by using exec. But it is now after Spanksgiving.

-Original Message-
From: Tutor  On Behalf Of Alan 
Gauld via Tutor
Sent: Friday, November 23, 2018 10:35 AM
To: tutor@python.org
Subject: Re: [Tutor] A required question

On 23/11/2018 05:34, Avi Gross wrote:
> Just to be different, and perhaps return to the purpose of this group,

Actually I think the higher level debate of Python's direction as a teaching 
language is entirely appropriate for this group. It kind of defines the group 
and and its current and future role.

Whereas...

> What I was thinking was the ability to do something like this:
> 
> import ReChoir as require
> 
> require.version(condition, before=True, after=False) 
> require.modules(module list, recursive=True) require.os([“Eunuchs”, 
> “Windblows”])
> require.functionality(“print3”)

I can see the logic but suspect discussion of new features is probably the 
preserve of the main Python list. If you can get traction there somebody might 
actually go ahead and write one!

On the tutor list such matters are usually adequately covered by virtual 
environments etc.

--
Alan G


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A required question

[Avi Gross]

David,

As I suspected. Yes, I am aware how to do those things. Just wondered if anyone 
automated the process so a fairly simple interface worked. 

I am dropping the request.

Avi

-Original Message-
From: Tutor  On Behalf Of David 
Rock
Sent: Friday, November 23, 2018 9:28 PM
To: Tutor Python 
Subject: Re: [Tutor] A required question


> On Nov 23, 2018, at 09:35, Alan Gauld via Tutor  wrote:
> 
> On 23/11/2018 05:34, Avi Gross wrote:
>> What I was thinking was the ability to do something like this:
>> 
>> import ReChoir as require
>> 
>> require.version(condition, before=True, after=False) 
>> require.modules(module list, recursive=True) require.os([“Eunuchs”, 
>> “Windblows”])
>> require.functionality(“print3”)
> 
> I can see the logic but suspect discussion of new features is probably 
> the preserve of the main Python list. If you can get traction there 
> somebody might actually go ahead and write one!

discussion of a “require” library probably isn’t necessary.  It’s pretty 
straightforward to include the logic using existing methods.

For the version of python, test against sys.version_info For the modules, put 
your import calls in a try block and handle exceptions For the OS version, test 
against os.name or sys.platform The last one, “functionality,” is a bit vague.  
Probably another candidate for a try block.


—
David Rock
da...@graniteweb.com




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Pythonic way

[Avi Gross]

This is not a question or reply. Nor is it short. If not interested, feel free 
to delete.

 

It is an observation based on recent experiences.

 

We have had quite a few messages that pointed out how some people approach  
solving a problem using subconscious paradigms inherited from their past. This 
includes people who never programmed and are thinking of how they might do it 
manually as well as people who are proficient in one or more other computer 
languages and their first attempt to visualize the solution may lead along 
paths that are possibly doable in Python but not optimal or even suggested.

 

I recently had to throw together a routine that would extract info from 
multiple SAS data files and join them together on one key into a large 
DataFrame (or data.frame or other such names for a tabular object.). Then I 
needed to write them out to disk either as a CSV or XLSX file for future use.

 

Since I have studied and used (not to mention abused) many programming 
languages, my first thought was to do this in R. It has lots of the tools 
needed to do such things including packages (sort of like modules you can 
import but not exactly) and I have done many data/graphics programs in it. I 
then redid it in Python after some thought.

 

The pseudocode outline is:

 

*   Read in all the files into a set of data.frame objects.
*   Trim back the variables/columns of some of them as many are not needed.
*   Join them together on a common index using a full or outer join.
*   Save the result on disk as a Comma Separated Values file.
*   Save the result on disk as a named tab in a new style EXCEL file.

 

I determined some of what I might us such as the needed built-in commands, 
packages and functions I could use for the early parts but ran into an 
annoyance as some of the files contained duplicate entries. Luckily, the R 
function reduce (not the same as map/reduce) is like many things in R and takes 
a list of items and makes it work. Also, by default, it renames duplicates so 
if you have ALPHA in multiple places, it names them ALPHA.x and ALPHA.x.x and 
other variations.

 

df_joined <- reduce(df_list, full_join, by = "COMMON")

 

Mind you, when I stripped away many of the columns not needed in some of the 
files, there were fewer duplicates and a much smaller output file.

 

But I ran into a wall later. Saving into a CSV is trivial. There are multiple 
packages meant to be used for saving into a XLSX file but they all failed for 
me. One wanted something in JAVA and another may want PERL and some may want 
packages I do not have installed. So, rather than bash my head against the 
wall, Itwisted and used the best XSLX maker there is. I opened the CSV file in 
EXCEL manually and did a SAVE AS …

 

Then I went to plan C (no, not the language C or its many extensions like C++) 
as I am still learning Python and have not used it much. As an exercise, I 
decided to learn how to do this in Python using tools/modules like numpy and 
pandas that I have not had much need for as well as additional tools for 
reading and writing files in other formats.

 

My first attempts gradually worked, after lots of mistakes and looking at 
manual pages. It followed an eclectic set of paradigms but worked.  Not 
immediately, as I ran into a problem in that the pandas version of a join did 
not tolerate duplicate column names when used on a list. I could get it to 
rename the left or right list (adding a suffix a suffix) when used on exactly 
two DataFrames. So, I needed to take the first df and do a df.join(second, …) 
then take that and join the third and so on. I also needed to keep telling it 
to set the index to the common value for each and every df including the newly 
joined series. And, due to size, I chose to keep deleting df no longer in use 
but that would not be garbage collected.

 

I then looked again at how to tighten it up in a more pythonic way. In English 
(my sixth language since we are talking about languages  ) I did some things 
linearly then shifted it to a list method. I used lists of file names and lists 
of the df made from each file after removing unwanted columns. (NOTE: I use 
“column” but depending on language and context I mean variable or field or axis 
or many other ways to say a group of related information in a tabular structure 
that crosses rows or instances.)

 

So I was able to do my multi-step join more like this:

 

join_with_list = dflist[1:]

current = df1

suffix = 1

 

for df in join_with_list:

current = current.join(df, how='outer', rsuffix='_'+str(suffix))

suffix += 1

current.set_index('ID')

 

In this formulation, the intermediate DataFrame objects held in current will 
silently be garbage collected as nothing points to them, for example. Did I 
mention these were huge files?

 

The old code was much longer and error prone as I had a df1, df2, … df8 as well 
as other intermediates and was easy to copy and paste then 

Re: [Tutor] how to print lines which contain matching words or strings

[Avi Gross]

d is there any obvious
advantage?

If you care about efficiency, some final notes.

The order of the searches might matter. The most commonly found ones should
be tested first and the rarest ones last. Since the algorithm does not want
to print a line multiple times, it will stop evaluating when it finds what
it wants.

Regular expressions are powerful but also not cheap. If many or all the
things you are searching for are simple text, consider using normal string
functions as was discussed earlier. One approach would be to have two
modules and something like this:

If string_search(...):
Print it
elif re_search(..):
Print it
Else
Skip it

And be warned you may make spurious matches. If you search for "and" you
will match sand and ampersand. You need to make the regular expression
understand you want a word boundary before and after or whatever your need
is. You can do very powerful but expensive things like only matching if you
find the same word at least three times on that line, no matter what the
word is. You may need to tell it whether a match should be greedy and many
other considerations such as ignoring case. 

Have fun.

Avi




-Original Message-
From: Tutor  On Behalf Of
Asad
Sent: Monday, November 19, 2018 10:15 PM
To: tutor@python.org
Subject: Re: [Tutor] how to print lines which contain matching words or
strings

Hi Avi Gross /All,

 Thanks for the reply. Yes you are correct , I would like to to
open a file and process a line at a time from the file and want to select
just lines that meet my criteria and print them while ignoring the rest. i
have created the following code :


   import re
   import os

   f3 = open(r'file1.txt',r)
   f = f3.readlines()
   d = []
   for linenum in range(len(f)):
if re.search("ERR-1" ,f[linenum])
   print f[linenum]
   break
if re.search("\d\d\d\d\d\d",f[linenum])   --- > seach for a patch
number length of six digits for example 123456
   print f[line]
   break
if re.search("Good Morning",f[linenum])
   print f[line]
   break
if re.search("Breakfast",f[linenum])
   print f[line]
   break
...
further 5 more hetrogeneus if conditions I have

===
This is beginners approach to print the lines which match the if conditions
.

How should I make it better may be create a dictionary of search items or a
list and then iterate over the lines in a file to print the lines matching
the condition.


Please advice ,

Thanks,


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to print lines which contain matching words or strings

[Avi Gross]

Asad,

As others have already pointed out, your request is far from clear. 

Ignoring the strange use of words, and trying to get the gist of the
request, would this be close to what you wanted to say?

You have a file you want to open and process a line at a time. You want to
select just lines that meet your criteria and print them while ignoring the
rest.

So what are the criteria? It sounds like you have a list of criteria that
might be called patterns. Your example shows a heterogenous collection:

[A ,"B is good" ,123456 , "C "]

A is either an error or the name of a variable that contains something. We
might want a hint as searching for any old object makes no sense.

The second and fourth are exact strings. No special regular expression
pattern. Searching for them is trivial using normal string functionality.
Assuming they can be anywhere in a line:

>>> line1 = "Vitamin B is good for you and so is vitamin C"
>>> line2 = "Currently nonsensical."
>>> line3 = ""
>>> "B is good" in line1
True
>>> "B is good" in line2
False
>>> "B is good" in line3
False
>>> "C" in line1
True
>>> "C" in line2
True
>>> "C" in line2
True

To test everything in a list, you need code like for each line:

for whatever in [A ,"B is good" ,123456 , "C "]
If whatever in line: print(line)

Actually, the above could print multiple copies so you should break out
after any one matches.

123456 is a challenge to match. You could search for str(whatever) perhaps.

Enough. First explain what you really want.

If you want to do a more general search using regular expressions, then the
list of things to search for would be all the string in RE format. You could
search multiple times or use the OR operator carefully inside one regular
expression. You have not stated any need to tell what was matched or where
it is the line so that would be yet another story.

-Original Message-
From: Tutor  On Behalf Of
Asad
Sent: Sunday, November 18, 2018 10:19 AM
To: tutor@python.org
Subject: [Tutor] how to print lines which contain matching words or strings

Hi All ,

   I have a set of words and strings :

like :

p = [A ,"B is good" ,123456 , "C "]

I have a file in which I need to print only the lines which matches the
pattern in p

thanks,

--
Asad Hasan
+91 9582111698
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] the fivefold path

[Avi Gross]

 having people get lost counting 
parentheses at a time that editors did not easily tell you what matched what.

But keeping track of multiple threads in a GUI can be a modern version of 
nested complexity. 

I used to observe the Obfuscated C contests years ago. Anyone for obfuscated 
python. Well, not here in the tutor group.

https://www.ioccc.org/
http://wiki.c2.com/?ObfuscatedPython

I have played in the past with tricks to write code mostly in a language like 
Hungarian using a pre-processor that flipped the key words back into the usual 
English style and of course used variable names in yet other languages like 
transliterated Aramaic. Hard to find people who could even figure out what the 
program did. Throw that into a language like Python where you can create 
classes and overload operators and it can get to look like an Abbot and 
Costello routine.

But if you want clear, self-documenting, code, I prefer that.

Time to get back to normal questions on this forum?

-Original Message-
From: Tutor  On Behalf Of Mark 
Lawrence
Sent: Tuesday, November 20, 2018 3:45 PM
To: tutor@python.org
Subject: Re: [Tutor] Pythonic way

On 20/11/2018 18:08, Avi Gross wrote:

> 
> We have two completely separate ways to format strings that end up 
> with fairly similar functionality. Actually, there is an implicit 
> third way 
> 

You could argue five ways :-)

1. C printf style formatting
https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting
2. New style string formatting
https://docs.python.org/3/library/string.html#string-formatting
3. f-strings
https://docs.python.org/3/reference/lexical_analysis.html#f-strings
4. String templates
https://docs.python.org/3/library/string.html#template-strings
5. String methods
https://docs.python.org/3/library/stdtypes.html#string-methods

Any advance on five anybody?

--
My fellow Pythonistas, ask not what our language can do for you, ask what you 
can do for our language.

Mark Lawrence

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] evolutionary drift

[Avi Gross]

Steve,

You may be right. It often happens that someone has a (small) idea, perhaps 
very focused, and others chime in and try to apply it more widely, perhaps by 
making it more general, and it grows. Over the years, the earlier adopters may 
be seen almost as co-creators or even become the lead in the story. Tell me who 
seems to be associated with Apple and who did much of the technical work? I am 
not saying that Jobs did not have vision and marketing talent and an eye for 
style and so on. I am not even sure who came up with ideas back then. Another 
such person, Bill Gates, did do some programming of BASIC early on and so forth.

So, whatever the history of early Python (and predecessors) was, it may have 
begun as some enhancements and improvements of what came before and perhaps new 
paradigms. Others who saw it may have seen something that looked easier to 
teach. An obvious example, if it was there way back then, was removing lots of 
brackets used in other languages (such as {([])} ) and using indentation. Feels 
more natural to not-so-mathematical types.

But over the years the brackets have returned and worse. Like many languages, 
Python used what symbols it could find on the keyboard and then overloaded them 
horribly. Parentheses are used for grouping but also for tuples and to force 
invocation of a function and to hold the argument tuple (albeit no trailing 
comma is needed for a single argument as in other tuples) and presumably in 
other contexts such as within regular expressions. Periods can mean quite a few 
things as can asterisk and so on. There are few matched sets of characters on 
the keyboard. () is now used for tuples, among other things. [] is used for 
lists except when it is used for dictionary access as in cards[key] versus 
text[5] and {} is used for dictionaries except when you use [] but also for 
sets ...

Heck, they ran out of symbols. {} is an empty dictionary and you say set() for 
an empty set. Ditto for tuple. 

<> is not currently used as a matched set as it has many other uses like in 
comparisons. Some languages even use <> as the same as != or ~= to mean not 
equals. "" and '' and even `` are sort of used as if they are a matched set I 
some languages (`` is in R, not Python) but are typographically identical as 
compared to text processing that creates two versions.

EBCDIC had a few other symbols but many languages now use only ASCII symbols 
and have to combine them to make complex and even non-intuitive combinations as 
symbols. Try teaching a child in say C++ that:

X++==++Y is valid and even meaningful if read as:

X++ == ++Y

because it asks you to get the current value of X to compare to the NEW value 
of Y incremented and return a Boolean result and immediately thereafter, 
increment X. Heck, you can write (XY) and so on. I have seen languages with 
an = and == and even === alongside := and ::= and -> and --> and <- and <-- and 
more all to mean variations on a theme.

If we had started with many more symbols, in some ways it would be harder but 
in other ways easier. Mathematicians borrow symbols from lower case, upper case 
and script letters from languages like Greek and Latin but also from Hebrew as 
in, well not easy to include in a file contain normal text but aleph-null (and 
aleph-one and infinitely more levels of infinity.)

A simple teaching language that uses English words children know might either 
be verbose in places or long as in making you spell out DELETE instead of del. 
But quite a few languages are simple if you leave out most of the 
functionality. Yes, you need to explain why some things must end in a semicolon 
or be indented a certain way or require parentheses or ...
What you don't want to teach is a complex language like English. I was teaching 
my Dad as he prepared for Citizenship and he balked when told there were at 
least seven distinct ways you pronounce OUGH in common English words.

So, perhaps python can be used to teach basic programming for several reasons 
including no need to declare variables and their "types" in advance and having 
relatively intelligent basic types that easily get converted in the background 
as in various kinds of numbers. But when you use more and more features, it 
expands into areas where, frankly, there is no one right answer and choices are 
often made by fiat or by a majority. Mathematically, if you view a topic like 
multiple inheritance in classes and the kludges made to get around logical 
issues, you see what a mess it really is and very hard to teach. Look at the 
double underscore notation(prefix-only) for variables that renames them to be 
unique within a certain scope to avoid collisions in the mythical searched name 
space. 

I am not against such drift but at times wonder if a one-size-has-to-fit-all 
mentality is wise.

I had  a thought on an earlier topic by Asad. He wanted to write in Python what 
is effectively a member of the UNIX grep family. Specifically, fgrep 

Re: [Tutor] Writing the right code rite

[Avi Gross]

Steve,

I appreciated your letter and it helped point me in the direction this group
may be designed for. For some it is sort of like forms of therapy where the
goal is to help them reach an insight rather than toss a diagnosis at them
and maybe write a prescription. The ultimate goal is to learn how to use
resources by yourself. I am so used to doing my own work and getting a real
understanding, that it is not easy to adjust to how some others are like the
Help Vampires in the article you quoted.

Having read it, I do recognize how some people I know may fit into that
category and I have often after a while automatically set up not enabling
them in some of the ways you describe. Amusingly enough, I have lived in
Transylvania (border area where Romania and Hungary meet) and never met any
(nonexistent) Vampires. But they seem to be everywhere in books and on TV so
why not HELP Vampires. 

Avi
-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Tuesday, November 27, 2018 5:40 AM
To: tutor@python.org
Subject: Re: [Tutor] Writing the right code rite

On Mon, Nov 26, 2018 at 07:18:46PM -0500, Avi Gross wrote:

> What kind of answers have people provided?

The archives of this mailing list go back to 1994. If you spend some time
browsing randomly at the answers people provide, you should get a feel for
what sort of answers we tend to give.

https://mail.python.org/pipermail/tutor/



> I know it slows things down, especially with a moderator, but often 
> the best answer is to ask some questions before trying to supply an
answer.

Indeed. Learning how to ask good questions is part of the learning process,
and when people ask poor questions, asking questions in return will either:

- teach them by example what sort of information they need to solve their
problem; or

- discourage the lazy help-vampires from draining the energy out of this
group.

http://www.skidmore.edu/~pdwyer/e/eoc/help_vampire.htm



> Perhaps a
> rapid email exchange directly with a student, perhaps moving on to 
> instant messaging or phone or video forms of communication would work 
> better for those interested.

Of course you are an adult in a free country and you have the right to do
whatever you like, but I don't recommend this. In fact I would consider it
anti-social and hostile to the rest of the community.

I think I speak for most of us when I say we're looking to help and educate
the entire community. We're not the personal servants of individual posters,
or private teachers. We post so that *everyone* can learn from the answers,
not just the person asking the question, and the
*process* of reaching the answer is just as important as the final code. 

Taking that process off-list is, in my opinion, not helping the community.
(Here, I'm not speaking for others.)

Its also likely to get annoying fast, for both parties: you, when the poster
starts bombarding you with question after question, and the poster
themselves, when you are too slow to respond. At least if they email the
list, there is the opportunity for others to reply in your stead.

Of course there are circumstances where it is appropriate to take discussion
off-list:

- you're being paid to help;

- the discussion wanders off-topic;

- or it becomes irrelevant and of no interest to the rest of the community
(perhaps because it is too specialised to interest anyone but yourself and
the original poster);

- or moves into confidential/private areas of discussions that shouldn't be
posted to the list (were you asked to sign an NDA?);

etc. Some subjective judgement may be required.


> When done, you might post a summary if appropriate for others 
> interested but note showing a full solution can be unfair if other 
> students working on the same problem just latch on to that.

The community standard here is that we don't do homework for others. 
We'll answer concrete questions about Python the language itself, of
course: "how do I use the zip function?". We'll help them debug their code,
if they've written some code.

If they haven't written even a single line of code, we generally don't do
much more than say "Show us what you've tried."

We'll help guide people towards solutions, but not hand them the solution on
a platter.

That's for students in school, of course. We're a bit more flexible when it
comes to self-learners or even professional programmers asking for help. And
we usually take it on trust if they say "this isn't homework".

But even then, we're not the personal slave of the poster, and we have no
obligation to solve their problem for them. We're not obliged to hand over a
complete solution, and in fact doing so goes against our stated aim to
*teach* people. (How will they learn effectively if we do their work for
them?)

Again, some subjective judgement is required. I wouldn't hesitate to answer
minor or trivial questions in full, e.g. "how do I count the number of
dig

[Tutor] origins bootstrapped.

[Avi Gross]

Alan has been involved with Python for a long time so he has more to offer
historically.

I don't see some things as either/or. You can start with one major
motivation and it morphs from a one-celled creature like an Amoeba to a
complex vertebrate like a Python which needs modules added so it can walk
around better.

OK, horrible analogy but interesting naming. So some say Guido started with
learning his ABC and then became educated enough to understand Monty Python
and reach for the holy grail.

OK, even worse. Time to get serious.

I have seen this on many projects, not just programming languages and
environments. Something fairly simple is imagined then perhaps prototyped.
Someone may notice that what was created may be used in another way if
perhaps expanded a bit. Someone then realizes they now have functionality
that can be used to create something else, in a form of bootstrapping. After
a while they have a collection of tools that can be combined to make
something more complex. The biological analogy above can be an example. No,
I am not saying that a distant ancestor of a snake like a python was an
amoeba. But they do share common ancestors they have both diverged from with
the amoeba remaining a single celled organism and the python descending from
something that became multi-cellular then differentiated into having
different kinds of cells in tissues and organs and became a somewhat
integrated whole that is possibly more than the sum of its parts. The ABC
analogy is also obvious. Once an alphabet is chosen and provisional meanings
given to each letter, it can grow and even adjust to making words and
sentences and even seemingly endless streams of consciousness like some of
my messages.

Python was built on top of other achievements that some people were learning
from. There were many steps along the way from building machines programmed
one byte at a time in binary (I hated a class that made me do that as one
error means start over) to various levels where a compiler and then an
interpreter would parse things. We have been discussing using regular
expressions. Much of a language like python is having bits and pieces of
code written in ASCII or Unicode be parsed using hopefully unambiguous rules
into tokens that can be made into decision trees or whatever data structure.
That deepens on being able to look for and find some sort of pattern in
strings. I am not sure what python and others use, but it may be tools
similar to string search or regular expressions that allows them to
bootstrap.

Back when my group was programming in C, I was sent to Denver for a class in
Lex/Yacc to learn how to use C libraries that now look primitive. One was a
lexical analyzer and the other sort of a parser somewhat rudely named as Yet
Another Compiler-Compiler.

But today, what do most people use? Our tools improve, often by being a
wrapper to older tools and so on for multiple levels. New functionality is
added too.

Can I ask a question that I really want an opinion on? As a preface, I see
some think python as a formal language is being pushed by industry in
directions that may not meld as well for its use in other contexts like for
teaching students. How much of that is due to it being a relative open and
free product? There are plenty of other applications that you pay for and
thus have to be responsive to the buyers to remain in business. Python has
many implementations including some freer than others. Yet is has gone
through a bit of a bifurcation and many would like to see 2.X retained and
others wish everyone should migrate. Is there room for a smaller core
language that remains good for teaching purposes and that is small enough to
fit in a Rasberry pi, while other versions are of industrial strength? Do we
already sort of have some of that?

I was thinking of how many languages and environments have been looking at
working using parallelism. Most people simply have no need for the
complication. When you add the ability to do multiprocessing within an
application using something like threads, you spend lots of time making sure
you literally lock down shared resources so they are used serially. You need
to make sure race conditions do not lock up all your threads at once. Lots
of added overhead is only worth it if you gain in the process. Add multiple
cores in your CPU, and you may need to handle more complications as they are
actually running in parallel, perhaps still sharing a common memory. Allow
it to use multiple processors around the world, and you need even more
elaborate control structures to synchronize all that.

It definitely is worth doing but does everyone need it especially for
teaching an intro class?

I was thinking about the little project I mentioned the other day. Should
some of it be done in parallel using methods available? One part of the
problem was to read in N files into N pandas DataFrame objects. I knew that
I/O tends to be fairly slow and most programs take a nap while waiting. 

[Tutor] seniority

[Avi Gross]

Steven,

Good question. I do not know how long and in what capacity many people have 
been involved with anything, including Python in some form. I do know a bit 
about Alan and plenty about myself. I was saying that I would take seriously 
what was said by people LIKE Alan because they had more experience and perhaps 
more internal knowledge than someone like e who took an introductory course in 
Python maybe five years ago then went on to many other languages and only came 
back recently and is doing some serious studying. 

The main difference between me and others here with questions is that I am in 
the general category Alan mentioned of someone with a mindset that has been 
very hospitable to mathematics and an algorithmic way of thinking. But that 
does not give me much knowledge in an experiential sense other than what I have 
read.

And, some of my resources are out of date. Last night I watched several talks 
by David Beazley that Mats pointed out and plan on doing some more. He kept 
talking about revolutionary changes in release 3.6 and I note I am now using a 
3.7 variant. He suggested you use these such as the f-string or depending on 
dictionaries to be in the order things were added and so on. My thought was 
that this would disrupt anyone running not just 2.X but any older version of 
3.X but that may have been his point. You may think you know the language, but 
it is like layers of the onion and some of your favorite features were 
definitely not in place way back when the people with seniority got into it.

Watching him rapidly improvise code live, I also saw how some people can make 
ever deeper abstractions come to life in ways that even sort of add features to 
the language if used right. He did an entire talk on using classes and 
metaclasses and decorators to change the useless annotations added to the 
language that don't actually check if arguments to functions (or in classes) 
are of the type or condition expected, such as a positive integer, to one where 
the invisible wrappers created will check implicitly and return errors.

It is very possible to write programs that others may not easily understand 
just using existing features, let alone as it keeps evolving.

Rushing out, as usual. Today is Thanksgiving in my part of the world so will 
reply to the rest of the message another time.

Avi
-Original Message-
From: Tutor  On Behalf Of Steven 
D'Aprano
Sent: Thursday, November 22, 2018 1:06 AM
To: tutor@python.org
Subject: Re: [Tutor] origins bootstrapped.

On Wed, Nov 21, 2018 at 11:31:59AM -0500, Avi Gross wrote:

> Alan has been involved with Python for a long time so he has more to 
> offer historically.

I've been involved with Python for a long time too. What exactly are you trying 
to say?

<<>>>
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] learning languages

[Avi Gross]

--- 
From: Tutor  On Behalf Of Steven 
D'Aprano
Sent: Thursday, November 22, 2018 1:06 AM
To: tutor@python.org
Subject: Re: [Tutor] origins bootstrapped.

On Wed, Nov 21, 2018 at 11:31:59AM -0500, Avi Gross wrote:

...

> Can I ask a question that I really want an opinion on? As a preface, I see
> some think python as a formal language is being pushed by industry in
> directions that may not meld as well for its use in other contexts like for
> teaching students.

I think there is always going to be tension between the needs of 
different users. Beginners need simplicity; expert, experienced 
programmers need power; both have very different ideas of what 
"readable code" means.

I don't think Python is being pushed in any direction by "industry". It 
is evolving according to the needs of the programmers who use it, some 
of whom may work for some industry or another.

> How much of that is due to it being a relative open and
> free product? There are plenty of other applications that you pay for and
> thus have to be responsive to the buyers to remain in business. Python has
> many implementations including some freer than others.

I don't know of any non-free (free as in beer, or free as in speech) 
implementations of Python. Can you elaborate?


> Yet is has gone
> through a bit of a bifurcation and many would like to see 2.X retained and
> others wish everyone should migrate. Is there room for a smaller core
> language that remains good for teaching purposes and that is small enough to
> fit in a Rasberry pi, while other versions are of industrial strength? Do we
> already sort of have some of that?

Standard CPython is light enough to run on fairly low-powered devices, 
including Raspberry Pi. For an even smaller footprint, you can use 
Micropython, which will run on embedded devices, although μPy does make 
some comprompises that means that it's not a fully compliant Python 
implementation.

There are, or were, other small implementations:

- Pippy, Python for Palm (probably unmaintained by now...)
- Python for S60, for the Nokia S60 platform (likewise...)
- Pythonce, for Windows CE (who still uses WinCE?)
- PyMite for embedded devices
- Python-iPod
- Py4A and QPython (Android)
- TinyPy
- PyPad for the iPad
- Pycorn, Python running on bare hardware with no OS


> I was thinking of how many languages and environments have been looking at
> working using parallelism.
[...]
> It definitely is worth doing but does everyone need it especially for
> teaching an intro class?

Who teaches threading and parallelization in introductory classes?


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A retired question

[Avi Gross]

After saying I dropped my request, short replies keep coming.

Let me reply in this context:

Would there be BUILT-IN tools that can be expected to already be everywhere
which can be used at the top of a program you write and distribute to
quickly check if the rest of the program  has the resources it
needs to run probably.

I am not currently planning on designing or implementing anything. It was an
academic question.

I was thinking along the lines of slightly advanced ways functions can start
with a group of assertions testing what is expected and quit if not set
right. They use assert().

Some of what is being suggested may have useful pieces you can use in doing
something like this. But I note that the ability to package things is not
something most users need. You need a kernel of guaranteed software you can
count on in order to test if other required elements and conditions can be
met.

Realistically, there are so many variations on python and how you can get
copies distributed that I cannot assume that if I write a routine that uses
numpy or pandas, as I have been doing, will be installed before a user makes
use of the software. And, you can even build your own versions of python
while leaving out or including things. It is nice to be flexible. But then
you may need to stock to some version from prehistoric times using
relatively few features AND being careful to not use features that were
later changed or removed.

Why bother?

OK, here is the solution. At the beginning of the script put out a big
warning telling the user that if they do not have all the required software
they are proceeding at their own risk and take full responsibility if
something weird happens before the software dies. Hitting ENTER will be
considered acceptance of the terms!

Again. Dropped. Moving on. 

If the goal here is to tutor, no need to share complex solutions but rather
help students learn how to look at their problems and see what kinds of data
and logic to apply in what order and then how to debug the inevitable
mistakes like a missing comma.

-Original Message-
From: Tutor  On Behalf Of
Mats Wichmann
Sent: Monday, November 26, 2018 10:23 AM
To: paso...@gmail.com
Cc: tutor@python.org
Subject: Re: [Tutor] A required question

On 11/25/18 8:54 PM, Asokan Pichai wrote:
> On Sat, Nov 24, 2018, 14:33 Avi Gross  
>> David,
>>
>> As I suspected. Yes, I am aware how to do those things. Just wondered 
>> if anyone automated the process so a fairly simple interface worked.
>>
> Does the requirements.txt file (associated with pip IIRC) does most of 
> what you want?

If so, then also worth looking at the pipfile and pipenv which are aiming to
improve on the experience.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Writing the right code rite

[Avi Gross]

I am here to learn but also to help out when I already know something that
is asked.

 

What has been of some concern to me is how to reply when the asked has not
shared with us lots of info.

 

If I see them using "print" without parentheses, I can assume they are not
using 3.x and perhaps emulate their usage. Showing them samples of code that
won't work in their environment is a bit of a waste of time, theirs and
mine.

 

But in watching, I conclude that a subset of the requests come from people
who are some form of early students and only aware of selected aspects of
the language. For some of them uses of words and phrases is a tad loose so
when they say "list" they might mean something that can range from what
others might call tuples or lists or dictionaries or sets or reaching a bit
beyond to other words like vectors, arrays, matrices, bags, collections,
name spaces and so on.

 

What many want is for someone to help them write some kind of simple loop.
Ideally, they would state the problem clearly enough and try to write code
and ask help when it does not seem to work. If they answer a few questions
to clarify, they can often get a simple answer they can use or understand.
But it might help to steer them otherwise.

 

We have had recent requests for example for what sounded like a need to read
text into words and then keep adding them to a 'list" if not already there.

 

What kind of answers have people provided?

 

What many of the answers would have in common is to break the lines up into
tokens we consider words. But even that can be a complex task that can be
done many ways depending on what the file might contain and whether words
can include an apostrophe or hypen. Some solutions might tokenize a line at
a time and some do the entire file at once using some regular expression
that matches anything after one or more non-word characters till the next
series of one or more non-word characters. Others might just split a line on
a single space which likely won't work on normal English text containing
punctuation and so on.

 

Similarly, some may suggest tossing all the tokens into a set to get unique
results. Others may suggest using a dictionary and couniting how many times
each word appears (even if that was not required.) Others may use lists and
introduce use of whys to check if something is already in the list or extend
the list. Some may suggest using a data structure like a dequeue for reasons
I am not able to explain. 

 

Others may want to use a list and carefully explain how to search in an item
is already in a list and how to extend the list. Some may even want them to
keep the list in alphabetical order. Some may want to dump all words in list
1 then iterate on it by adding only words not already in list2 to list2.
Some may want to use a binary tree that finds thing in O(log(N) or
something. And yes, some would like you to make a long list then sort it and
apply a function such as an iterator that returns the next item after
finding all consecutive items that are the same.

 

There are an amazing number of fairly reasonable ways to solve problems,
with the usual tradeoffs.

 

But if someone is new to this, what answer meets their immediate needs? For
most school projects, unless they are told to use some module, the
expectation is for something fairly simple without worrying that the small
dataset used will run very long or use much memory.

 

And, the solution probably should not be deeply nested in ways hard to read.
I recently wrote a function for someone (not on this group) with about a
dozen lines of code that did things step by step using variables with
somewhat meaningful names. In English, the goal was to take in a DataFrame
and only keep those rows where some subset of the columns all had valid
data. The long version was easy to understand. It did things like make a
narrow copy of the data using just that subset of the columns. Then it
applied a function that returned a vector of True/False if the narrow rows
were all OK or had anything missing. Then that vector was used to index the
original table to select only rows where it was true. But all that could be
collapsed into a single line with a single but somewhat nested statement as
each step sort of feeds into being an index of another step. But I would not
want that kind of code around without paragraphs of comments and in this
case, what difference does it make if temporary variables had a name before
being garbage collected when the function returns?

 

Since I find I am more interested in examining different ways to solve a
problem with some thought on what is better or just easier or more elegant
and so on, I am not always a best fit for instructing new students unless
they are sitting in a class I am teaching and mostly at the same level. This
is an open forum where we often have no clue what the student already knows
or has tried and we do not encourage them to post more than a minimal
example. But we can 

[Tutor] user overloaded

[Avi Gross]

cts means whatever you want
it to mean. I recently (for fun) created a sort of auto-increment (as in the
C/C++ symbol ++) so that if you had a counter called x, then adding ANYTHING
to x using the += notation just incremented it by 1.

X = odd_object(6) # X initialized to 6
X += 1  # X is now 7
X += 12 # X is now 8 as the __iadd__ method ignores the 12 and adds
just 1

The above abomination does not replace ++ as an autoincrement but can give
you some functionality but many users will have no idea what just happened.

I note it can be worse in languages that allow you to create your own
operators. I often use %>% in R for example and can make my own by placing
almost any nonsense between percent signs and binding it to a function.

As I understand it, back to Python, if adding a := to be used in expressions
does not break any existing code, it may not be so bad. I mean nobody is
supposed to be using that symbol now, so what does it hurt?

Ah, but wait. If I had the previously discussed notion of checking whether
the string I wanted to dynamically evaluate contained possibly dangerous
code, would it be looking for a ":=" ? Heck, if I am parsing something and
looking for a colon that indicates the beginning of a BODY, might it get
confused by finding this colon in a wrong context?

I conclude by saying life is complicated and then you die. I mean there are
many places where similar but not identical things happen in languages as
well as life.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Wednesday, November 28, 2018 4:13 AM
To: tutor@python.org
Subject: Re: [Tutor] PEP 572 -- Assignment Expressions

On Tue, Nov 27, 2018 at 08:26:04PM -0500, Avi Gross wrote:
> Ivo,

You've replied to the wrong mailing list. Ivo wrote to Python-List, not
Tutor.

But now that you've raised the issue...


[Ivo asked]
>Why:
>  if (match := pattern.search(data)) is not None:
># Do something with match
> 
>What is wrong with:
>  if match = pattern.search(data) is not None:
># Do something with match
> 
>Assignment expression or assignment statement, it's an assignment,
right?
> It is very clear to everyone that it's an assignment! Can't it all 
> just be a "="?

It seems that Ivo didn't read the PEP because that is discussed:

https://www.python.org/dev/peps/pep-0572/#id32

Despite Ivo's statement that it is "clear to everyone", the problem is that
whenever you see an assignment x=y in an assignment, there's no obvious way
of telling whether they meant assignment or it was a mistyped equality test
x==y. So it is not clear at all.

In C, this confusion between = and == is a frequent source of serious bugs,
including one attempt at inserting a back-door into the Linux
kernel:

https://freedom-to-tinker.com/2013/10/09/the-linux-backdoor-attempt-of-2003/

https://lwn.net/Articles/57135/


Requiring the colon x:=y makes it less likely to mess up the assignment or
comparison.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I need help with my project

[Avi Gross]

I suggest starting at the beginning when asking a question to people who
have no way of knowing what you have not told them.
Your sentence is completely in middle or even near the end of something that
has to be larger:

" If the user selected a sandwich, french fries, and a beverage, reduce the
total cost of the order by $1.00."

It sounds like you are being asked to emulate a place you can buy food. So I
suspect you need to display a menu, ask the user to enter info, keep a
running total and so on.

So how do you think you would store the order as well as whatever you need
to know about past sales or the non-discounted cost of the current order?
How do you see if the current order has all the parts needed for the
discount. Do you need to apply the discount just once or multiple times if
they ordered food for lots of people? 

Sure, we could write some code for you, but perhaps you need to think it
through first and maybe if you get stuck, ask if someone can tell you what
is wrong with the code.

Given what you sent, the answer is easy.

RECOGNIZE they placed an order that meets the criterion:
SUBTRACT 1.00 from their order.

But that is not written in Python for obvious reasons.


-Original Message-
From: Tutor  On Behalf Of
Treyton Hendrix
Sent: Tuesday, November 27, 2018 7:30 PM
To: tutor@python.org
Subject: [Tutor] I need help with my project

If the user selected a sandwich, french fries, and a beverage, reduce the
total cost of the order by $1.00.

This is what I have to do and I don't know where to start.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding "IDLE Subprocess Didn't Make Connection" Error

[Avi Gross]

I was wondering if I was the only one who felt the urge to apply a tad of humor 
and suppressed most of the thoughts about idol/IDLE worship and other puns so I 
am glad to see Steve do just a little of the same.

It seems that despite how portable Python is touted to be, quite a few people 
report problems in getting the basics working when installing. And if you can't 
get started, that is obviously a barrier. My suggestion earlier was based on 
the fact that IDLE is an add-on and they should first check if Python itself 
works. Any text editor can be used that just produces plain text and scripts 
can be run in other ways.

But I have a thought. An old an often effective method to solve a problem is to 
search in the source code. Yes, you did not write IDLE. 

 I am still not clear on how IDLE aborts on startup but I recall that IDLE may 
be written in Python with source code available. I went into python to see 
where to look by seeing the path it would search.

>>> import sys
>>> print(sys.path)

I then followed a trail and what I found was idlelib with many components and 
when I import that library in the IDLE main unit I see some of the results:

>>> import idlelib
>>> dir(idlelib)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', 
'__package__', '__path__', '__spec__', 'autocomplete', 'autocomplete_w', 
'calltip_w', 'calltips', 'config', 'debugger', 'debugger_r', 'debugobj', 
'debugobj_r', 'hyperparser', 'iomenu', 'macosx', 'multicall', 'pyparse', 'rpc', 
'run', 'scrolledlist', 'stackviewer', 'testing', 'tree', 'windows', 
'zoomheight']

I could potentially follow along, and for a while I did. I think that after a 
few steps, on my machine, it hits pyshell in 
C:\Users\avid2016\AppData\Local\Programs\Python\Python37-32\Lib\idlelib and 
then runs code starting with:

#! /usr/bin/env python3

import sys

try:
from tkinter import *
except ImportError:
print("** IDLE can't import Tkinter.\n"
  "Your Python may not be configured for Tk. **", file=sys.__stderr__)
raise SystemExit(1)

And it goes on and on. It imports all kinds of things and creates many classes 
and if called directly, runs a big function called main. I searched for 
"subprocess" and "IDLE" to see if the file had whatever error message you saw 
and I think I found it in the definition of a function:

def display_no_subprocess_error(self):
tkMessageBox.showerror(
"Subprocess Startup Error",
"IDLE's subprocess didn't make connection.  Either IDLE can't "
"start a subprocess or personal firewall software is blocking "
"the connection.",
parent=self.tkconsole.text)

Correction, the first argument of "self' was a hint and that is actually a 
method in: 
class ModifiedInterpreter(InteractiveInterpreter):

Now if that is the error message, you work backwards to see when it might have 
been called/triggered.

Searching for a call to that method shows two places:

Here is one snippet:

self.spawn_subprocess()
#time.sleep(20) # test to simulate GUI not accepting connection
# Accept the connection from the Python execution server
self.rpcclt.listening_sock.settimeout(10)
try:
self.rpcclt.accept()
except socket.timeout:
self.display_no_subprocess_error()
return None

Here is the other:

self.spawn_subprocess()
try:
self.rpcclt.accept()
except socket.timeout:
self.display_no_subprocess_error()
return None

I am going to stop now and suggest it happens when IDLE creates a subprocess to 
run in parallel and plays games with sockets and fails.


FWIW, the method called before is:

def spawn_subprocess(self):
if self.subprocess_arglist is None:
self.subprocess_arglist = self.build_subprocess_arglist()
self.rpcsubproc = subprocess.Popen(self.subprocess_arglist)

Presumably something goes wrong in there but I suspect the Popen failed.

Why would it fail? Well, it could be many things including whatever actual info 
is in the arglist ...

That might take plenty of additional digging including suggestions about 
something not being set right in the path or other registry components or some 
rogue library component being imported rather than the intended one or so much 
more. But, in theory, you can even take the source code and run it more 
carefully and debug this. Not something for a novice.

Not sure if that helps but if I actually had to read all that code and do 
serious debugging, it might take me days and I have a funeral and a party 
(unrelated) to go to today alone 


-Original Message-
From: Tutor  On Behalf Of Steven 
D'Aprano
Sent: Saturday, November 17, 2018 6:18 AM
To: tutor@python.org
Subject: Re: [Tutor] Regarding "IDLE Subprocess Didn't Make Connection" Error

On Thu, Nov 15, 2018 at 03:38:28PM -0800, Break fast wrote:

> "IDLE subprocess didn't make 

Re: [Tutor] Finding the largest gap in tuple between two lists.

[Avi Gross]

Harry,

Your code may not have come through as intended as the formatting often
combined lines.

It may help to think about the question before giving deeper answers.

You have two places and two sets of goods with prices. You want the
intersection of those two to focus on what goods are in both. There are
various ways to do that ranging from using sets to creating an empty object
(such as a list or dictionary) then iterating over all items in group A and
adding only what is also IN group B.

You then want to maximize the difference in price for a particular item in
group A versus B. If you have the right data structures, that is a snap.

Suppose you made two dictionaries holding the keys in A and the keys in B
alongside values for each. You could iterate over the keys in either
dictionary and calculate A[key] = B[key] and if it is higher than the
previous high, replace the previous high while also saving the value of key.

Is that roughly what is needed? You buy one item with the maximum "profit"?

I can imagine having a fixed dollar amount to spend and a similar problem
where you can buy as many as you can afford of that item as a variant or
even a knapsack version where you might buy cheaper ones with any money
left, ...

If you post again, see if you can find a way to have the text formatted
properly as we have seen examples where Python programs get different
results, or fail completely, with changes in indentation.

-Original Message-
From: Tutor  On Behalf Of
Harry Oneill
Sent: Saturday, November 17, 2018 8:55 PM
To: tutor@python.org
Subject: [Tutor] Finding the largest gap in tuple between two lists.

Hello there everyone i hope you are all doing fantastic.

Im currently working on a little program that is designed to do a basic
function to help me to trade efficiently within the game Star Citizen:

#1 take two inputs ( location and destination )
#2 Find what product is best to buy at location to sell at destination
#3 Print the results

currently i have figured out how to get python to take the two inputs and
provide me with the lowest buy price and highest sell price.

However the information i want to produce is what has the largest profit
margin possible between the two locations so i need to compare what items
are available in both locations and then see what one has the largest price
difference then display that items name.

Below is  the code i have currently made for this.


Current_Location = input("Where are you currently located? :").lower()
Current_Destination = input("Where are you planning to travel? :").lower()

def Find_Lowest_Buy_Olisar():
print("Finding lowest buy price for OLISAR...")
print(*Sort_Buying_Olisar[:1], sep='\n') def Find_Highest_Sell_Olisar():
print("Finding highest sell price for OLISAR...")
print(*Sort_Selling_Olisar[:1], sep='\n') def Find_Lowest_Buy_Levski():
print("Finding lowest buy price for LEVSKI...")
print(*Sort_Buying_Levski[:1], sep='\n') def Find_Highest_Sell_Levski():
print("Finding highest sell price for LEVSKI...")
print(*Sort_Selling_Levski[:1], sep='\n')

Buying_Olisar = [('Medical Supply', 17.01), ('Waste', 0.005),]
Sort_Buying_Olisar = sorted_by_second = sorted(Buying_Olisar, key=lambda
tup: tup[1])

Selling_Olisar = [('Agricium', 25.60), ('Aluminum', 1.25), ('Beryl', 4.26),
('Chlorine', 1.57), ('Corundum', 2.53), ('Diamond', 6.90), ('Distilled
Spirits', 4.95), ('Fluorine', 2.80), ('Gold', 6.07), ('Hydrogen', 1.02),
('Iodine', 0.41), ('Laranite', 28.91), ('Processed Food', 1.39), ('Quartz',
1.44), ('Scrap', 1.67), ('Stims', 3.40), ('Titanium', 8.27), ('Tungsten',
3.90),] Sort_Selling_Olisar = sorted(Selling_Olisar, key=lambda
tup:(-tup[1], tup[0]))

Buying_Levski = [('Agricultural Supply', 1.11), ('Aluminum', 1.20),
('Hydrogen', 0.98), ('Iodine', 0.38), ('Quartz', 1.37), ('Waste', 0.005),]
Sort_Buying_Levski = sorted_by_second = sorted(Buying_Levski, key=lambda
tup: tup[1])

Selling_Levski = [('Agricium', 25.60), ('Altruciatoxine', 11.63), ('Beryl',
4.25), ('Chlorine', 1.56), ('Corundum', 2.53), ('Diamond', 6.07),
('Distilled Spirits', 4.95), ('Fluorine', 2.80), ('Gold', 6.07),
('Laranite', 28.25), ('Medical Supply', 18.00), ('Processed Food', 1.38),
('Scrap', 1.68), ('Stims', 3.40), ('Titanium', 8.27), ('Tungsten', 3.90),
('Widdow', 24.00),] Sort_Selling_Levski = sorted(Selling_Levski, key=lambda
tup:(-tup[1], tup[0]))


if Current_Location == "olisar":
Find_Lowest_Buy_Olisar()
elif Current_Location == "levski":
Find_Lowest_Buy_Levski()
else:
print("Unknown location please try again.")

if Current_Destination == "olisar":
Find_Highest_Sell_Olisar()
elif Current_Destination == "levski":
Find_Highest_Sell_Levski()
else:
print("Unknown location please try again.")


Any input would be hugely appreciated.

Kind regards,

Harry O'Neill
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:

Re: [Tutor] Defining variable arguments in a function in python

[Avi Gross]

 There
are general functions where you must tell it in excruciating detail what to
do. What character (or sequence) separates the fields? Is it a comma as in a
CSV file? Is it a tab or some other character? Or are the fields of
specified constant widths with no separator? Are all rows the same or is
there a header row? Should the program make an educated guess about the
number of columns of data or will you tell it how many or which ones you
want? Ditto for the types of data in each. If the file might have characters
embedded such as in quotes containing a comma, how is that recognized or
dealt with? If a column read in has repeated values, should it be read in as
a more compressed categorical form such as a factor in R. The list of
possibilities is HUGE.

So instead of using a function like read.table() people make available lots
of others like read.csv() that is a wrapper that merges your supplied
arguments with a set of defaults that make sense when reading a Comma
Separated value file. 

But in the above example, to be clear, the defaults are all done in a normal
way. You do not use silly placeholders.

Steve writes:

" But at least with sentinels like None, or Ellipsis, it is *obvious* that
the value is probably a placeholder. With a placeholder like 11 or 999, it
isn't. They look like ordinary values."

I partially agree. I can think of cases where None really means None as in I
want None for Dessert, not the gentle substitution of sugar-free Jell-O.

Steve then provides an example of a more subtly bug. To be succinct, it is a
class of arguments that boils down to localized knowledge fails when
evaluated elsewhere or worse when a substitution is evaluated elsewhere.
That is also true in more normal cases but is indeed a problem.

Back to my CSV example, if the wrapper function ASSUMES that there will
normally be a header line in a CSV file and calls the wrapped function
appropriately with that added request, then mistakes can happen. If it makes
the opposite assumption, mistakes can happen. If it makes no assumption and
specifies nothing, mistakes can happen!

So I ask the question of how to deal with real life scenarios? There are
many no-win situations out there. Do you try to give warnings? Or maybe a
customized warning that says you should call the function again and this
time specify one of several alternatives as an explicit argument? I have
seen that done.

Steve is right that the scenario where a function changes things quietly can
be nefarious especially downstream. A good example is when dealing with
missing data your program may use a placeholder like Nan or NA or 999 or a
blank string. But if you call a function to take the mean of a sample, it
may not see the marker as a suggestion to skip it. It may treat nonsense as
a 0 and get a wrong mean. It may skip it but count it when dividing by the
number of items. Or, it may need an argument like na.rm=TRUE and perhaps
even a second argument specifying what an 'na' looks like. Worse, I often
save the results of a calculation in a file using some format like a CSV. I
then have it re-read into perhaps a program in another programming language.
Some expect strange things like a single period to mean not available. So
you may want to be careful how you write a file containing NOT AVAILABLE
items. Even worse, you often cannot export some things reliably as the
receiver cannot handle any form. Can you save a value of Inf that is
meaningful even to read back in? I don't mean from using something like
pickle, but from some storage formal like CSV or EXCEL.

I see Steve wrote a bit more and will just say I agree. God programming
style tries to avoid surprises when it can.

On Sun, Dec 30, 2018 at 12:07:20AM -0500, Avi Gross wrote:

[...]
> Or on a more practical level, say a function wants an input from 1 to 10.
> The if statement above can be something like:
> 
> >>> def hello(a, *n, **m) :
>   if not (1 <= a <= 10) : a=5
>   print(a)
>   print(*n)
> 
>   
> >>> hello(1,2,3)
> 1
> 2 3
> >>> hello(21,2,3)
> 5
> 2 3
> >>> hello(-5,2,3)
> 5
> 2 3


This design is an example of "an attractive nuisance", possibly even a "bug
magnet". At first glance, when used for mickey-mouse toy examples like this,
it seems quite reasonable:

hello(999, 1, 2)  # I want the default value instead of 999

but thinking about it a bit more deeply, and you will recognise some
problems with it.

First problem: 

How do you know what value to pass if you want the default? Is 999 out of
range? How about 11? 10? Who knows? If you have to look up the docs to know
what counts as out of range, you might as well read the docs to find out
what the default it, and just pass that:

hello(5, 1, 2)  # I want the default value 5

but that kind of defeats the purpose of a default. The whole point of a
default is that you shouldn't need to 

Re: [Tutor] Interpreter pasting Question

[Avi Gross]

Steven,

Thanks. You are right. I am no longer going to talk IDLE worship!

When I open up a command window and run what I assumed was the same version
of python, no problems. Should have tried that but frankly that means I am
stuck with having to then do other things differently like reloading an
edited file. Oh well.

It sounds like the problem is that IDLE is (invisibly) intercepting what I
type or paste and then passing it on to a slaved interpreter. There likely
are other side effects and another editor/environment may not have this
issue. I will try some.

As one of my programming styles includes lots of hands-on incremental
analysis of bits and pieces to get them working before combining them,  I
have no time for an idyllic experience.

OK, just kidding. Nice to have choices and I particularly want to move on to
using notebooks in which I can have batches of code I can run inline and
even interleave bits of multiple languages. 

Yes, I am aware that my experiments with exec are not testing the same
thing. They do indicate ways to get around the issue even using IDLE and I
am now thinking of ways to do more dynamic things by building code and then
executing it. So, not really time wasted. The fact that python relies on
indentation means you can have problems if things are not aligned just
right.

Here is my work-around if I happen to be using IDLE and want to copy in
something huge from say a web page.

Type this:

Cmd = """

Do the paste.

Type on a line by itself:

"""

Now type

exec(Cmd)

May not work all the time, but perhaps a reasonable work around.

-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 28, 2018 1:58 AM
To: tutor@python.org
Subject: Re: [Tutor] Interpreter pasting Question

On Fri, Dec 28, 2018 at 12:58:00AM -0500, Avi Gross wrote:

[...]
> Copying and pasting multiple lines into the interpreter fails in 
> mysterious ways, unless they are a logical single entity.
> 
> Is there a way to change this behavior, or perhaps an 
> editor/environment that feeds multiple lines more carefully to the
interpreter?

Which interpreter are you using?

If it is the regular Python REPL (Read Eval Print Loop), then pasting
multiple lines should work, with some limitations. For example, I can
paste:

x = 5
y = 6

as two lines, and it works fine under Linux. I see no reason why it should
be different under Windows.

If you just run "python" in the Windows shell (cmd.exe or whatever its
called), you should get an interactive interpreter. What happens when you
paste multiple lines in that?



[...]
> When copied in looks like this:
> 
> >>> X=5
> 
> Y=6
> 
> SyntaxError: multiple statements found while compiling a single 
> statement

That looks like a bug. Are you using IDLE? Perhaps it has been fixed in
newer versions of Python, and if not, you should report it as a bug on the
bugtracker

https://bugs.python.org/

but:

(1) try searching for similar bug reports first;
(2) read this first: 

http://www.sscce.org/

(3) and do try to keep your bug report short and to the point.


It looks like others have this problem with IDLE too:

https://duckduckgo.com/?q=idle+paste+multiple+lines


IPython/Jupyter allows pasting of multiple lines; I expect that bpython will
too.

https://ipython.org/

https://bpython-interpreter.org/


But as I said, the vanilla Python REPL ought to work. How are you starting
the interpreter? My guess is that you're using IDLE.



[...]
> Python has an eval() and an exec() and I would assume the latter would be
a
> way to see what works.

No, exec() executes Python code, it doesn't try to simulate a REPL.


> Here are three lines using \n:
> 
> >>> exec("x=6\ny=7\nprint(x+y)")
> 13
>
> 
> That seems to work. Again, if I copied and pasted the same as three lines,
> it fails.

Without knowing the system you are using, and how you copy and paste, it 
is hard to comment except to say "Works for me". Here are three lines:

x = 6
y = 7
print(x + y)


If I select those three lines with the mouse, copy, then paste into a 
standard Python interactive interpreter, I get this:


py> x = 6
py> y = 7
py> print(x + y)
13


exactly as expected. But if I do it in IDLE, I get this:

>>> x = 6
y = 7
print(x + y)

SyntaxError: multiple statements found while compiling a single 
statement
>>> 

That *really* sounds like a bug to me. But perhaps I just don't 
understand IDLE.




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Defining variable arguments in a function in python

[Avi Gross]

Steve,

I had the same thoughts and many more when I played with these ideas last 
night. I thought I stated clearly that this was an EXPLORATION and not a 
serious solution. So if that is accepted, we can certainly discuss merits 
without giving demerits as well as the many flaws and dangers lurking in any 
such design.


To begin with, anyone trying any variants of a design like this needs to be 
sure it is well documented. I have seen functions that do something vaguely 
similar and often with unexpected results.

Consider a NORMAL function with no external gimmicks. Say it accepts an 
argument (positional or by name does not matter) and processes it by checking 
they argument type, the specific values against a range expected, and so on. It 
then unilaterally makes changes. If it expects a single value of type integer, 
ang gets a floating point number, it may round or truncate it back to an 
integer. If it gets a complex number, it drops the imaginary part. If it gets a 
list or tuple or iterator of any sort, it just takes the first entry and 
coerces it into an integer. If it sees a matrix or other challenging object, it 
flattens it then ... If it sees a character string, it tries to convert 
something like "one" to an integer. If your choice is in some sense wrong, or 
just not convenient, it replaces it with another choice. For example, it may 
move your integer to the nearest prime integer. It may reject your suggestion 
to make a window on your screen a trillion pixels wide and drop it to 1024. 

Sometimes it prints a warning about changes it has made. Sometimes it won't run 
but print out a message telling you what value it might have accepted.

Again, discussing a different scenario. Would you agree that kind of design 
does happen and sometimes is seen as a smart thing but also as full of pitfalls 
even sometimes for the wary?

In that spirit, any design like the ones I played with is equally scary. Even 
worse, I have a QUESTION. Let me remind readers of the initial idea behind 
this. You have a function in which you want to communicate with the python 
ruler that it have a positional argument that will be switched to a default. 
But the current rules are that the only way to make something optional is by 
making it a keyword parameter and declare the default right there. If the 
function template is:

def hello(a=5)

then you can do any of the following with reasonable results:

hello()
hello(7)
hello(a=7)

The default is only applied in the first version.

The problem is if you want a non-key word to be used as a default and then also 
want to gather up additional positional arguments and so on. There is no way 
designed to do that and possibly there should not be.

So what I am trying to understand is this. When someone types in a function 
invocation and it is evaluated, when does the programmer get a chance to 
intervene? Something in what I will call the interpreter seems to look at what 
the programmer typed just now and sees:
hello()
hello(1)
hello(1,2)

and so on. It looks up the object that encapsulates the function hello is 
pointing to. It examines the many parts hidden within and determines what the 
designer asked for. If it sees that the actual call does not match, it geerally 
produces an error message like this:

>>> def hello(n): print(n)

>>> hello()
Traceback (most recent call last):
  File "", line 1, in 
hello()
TypeError: hello() missing 1 required positional argument: 'n'

That is a reasonable thing to do. But in my idiotic design, I would like the 
user to be told a tad more. I want them told that if they want the default, use 
an n consisting of an ellipsis as in
hello(...)

to get the default.

Is there a way to create a function and set it up so you have some control over 
the error message? Without that, this kind of function is even more dangerous.

This message is long enough. I will reply to Steven's specific points in a bit. 
Still on vacation 


-Original Message-
From: Tutor  On Behalf Of Steven 
D'Aprano
Sent: Sunday, December 30, 2018 5:39 AM
To: tutor@python.org
Subject: Re: [Tutor] Defining variable arguments in a function in python

On Sun, Dec 30, 2018 at 12:07:20AM -0500, Avi Gross wrote:

[...]
> Or on a more practical level, say a function wants an input from 1 to 10.
> The if statement above can be something like:
> 
> >>> def hello(a, *n, **m) :
>   if not (1 <= a <= 10) : a=5
>   print(a)
>   print(*n)
> 
>   
> >>> hello(1,2,3)
> 1
> 2 3
> >>> hello(21,2,3)
> 5
> 2 3
> >>> hello(-5,2,3)
> 5
> 2 3


This design is an example of "an attractive nuisance", possibly even a "bug 
magnet". At first glance, when used for mickey-mouse toy examples like this, it 
seems quite reasonable:

hello(999, 1, 2)  # I want the default value instead of 999

but thinking about it 

Re: [Tutor] decomposing a problem

[Avi Gross]

in data and manipulate it. R has
multiple sets of tools including one in what they call the tidyverse. In
English, given such a data structure with any number of rows and columns,
you have names for the columns and optionally the rows. The tools allow you
to select any combination of rows and columns based on all kinds of search
and matching criteria. You can re-arrange them, add new ones or create new
ones using the data in existing ones, generate all kinds of statistical info
such as the standard deviation of each column or apply your own functions.
All this can be done in a pipelined fashion.

What you often do is read in a data.frame from a Comma Separated Values file
(CSV) or all kinds of data from other programs including EXCEL spreadsheets,
Stata and so on, including the Feather format python can make, and massage
the data such as removing rows with any NA (not available) values, or
interpolate new values, split it into multiple dataframes as discussed and
so on. You can do many statistical analyses by feeding entire dataframes  or
selected subsets to functions to do many things like linear and other forms
of regression and it really shines when you feed these data structures to
graphics engines like ggplot2 letting you make amazing graphs. Like I said,
R is designed with vectors and data.frames as principal components.

But once python is augmented, it can do much of the same. Not quite sure how
much is ported or invented. Some data types like "formulas" seem to be done
differently. It will take me a while to study it all.

I can point to resources if anyone is interested but again, this is a python
forum. So it is of interest to me that it is possible to combine bits and
pieces of R and python in the same programming environment. I mean you can
use one to do what it does best, have the data structures silently be
translated into something the other one understands and do some more
processing where you have software that shines, then switch back and forth
as needed. This kind of duality may mean it is not necessary to keep
changing one language to be able to do what the other does, in some cases.
And, amusingly, much of the underlying functionality accessed is in C or C++
with some data structures being translated to/from the compiled C/C++
equivalents as you enter a function, them translated back at exit. 

This is not very deep so just making a point since Alan asked. You can find
strengths and weaknesses in any language. I love how python consistently
enough has everything being object-oriented. R started off without and has
grafted on at least a dozen variations which can be a tad annoying.



-Original Message-
From: Tutor  On Behalf Of
Steven D'Aprano
Sent: Friday, December 28, 2018 8:04 PM
To: tutor@python.org
Subject: Re: [Tutor] decomposing a problem

On Fri, Dec 28, 2018 at 03:34:19PM -0500, Avi Gross wrote:

[...]
> You replied to one of my points with this about a way to partition data:
> 
> ---
> The obvious solution:
> 
> keys = list(mydict.keys())
> random.shuffle(keys)
> index = len(keys)*3//4
> training_data = keys[:index]
> reserved = keys[index:]
> ---
> 
> (In the above, "---" is not python but a separator!)
> 
> That is indeed a very reasonable way to segment the data. But it sort 
> of makes my point. If the data is stored in a dictionary, the way to 
> access it ended up being to make a list and play with that. I would 
> still need to get the values one at a time from the dictionary such as 
> in the ways you also show and I omit.

Yes? How else do you expect to get the value given a key except by looking
it up?


> For me, it seems more natural in this case to simply have the data in 
> a data frame where I have lots of tools and methods available.


I'm not sure if your understanding of a data frame is the same as my
understanding. Are you talking about this?

http://www.r-tutor.com/r-introduction/data-frame

In other words, a two-dimensional array of some sort?

Okay, you have your data frame. Now what? How do you solve the problem
being asked? I'm not interested in vague handwaving that doesn't solve
anything. You specified data in a key:value store, let's say like this:


mydict = {'spam': 25, 'ham': 2, 'eggs': 7, 'cheddar': 1, 'brie': 14,
  'aardvark': 3, 'argument': 11, 'parrot': 16}

Here it is as a data frame:

df = [['spam', 'ham', 'eggs', 'cheddar', 'brie', 'aardvark', 'argument',
'parrot'],
  [25, 2, 7, 1, 14, 3, 11, 16]]

Now what? How do you randomly split that into randomly selected set of
training data and reserved data?

Feel free to give an answer in terms of R, provided you also give an
answer in terms of Python. Remember that unlike R, Python doesn't have a
standard data frame type, so you are responsible for building whatever
methods you need.




-- 
Steve
___
Tutor maillist  -  Tutor@python.o

Re: [Tutor] decomposing a problem

[Avi Gross]

Steve,

I am going to just respond to one part of your message and will snip the
rest. I am not is disagreement with most of what you say and may simply
stress different aspects. I will say that unless I have reason to, I don't
feel a need to test speeds for an academic discussion. Had this been a real
project, sure. Even then, if it will need to run on multiple machines using
multiple incarnations of python, the results will vary, especially if the
data varies too. You suggest that discussions backed by real data are
better. Sure. But when a discussion is abstract enough, then I think it
perfectly reasonable to say "may be faster" to mean that until you try it,
there are few guarantees. Many times a method seems superior until you reach
a pathological case. One sorting algorithm is fast except when the data is
already almost fully sorted already.

So why do I bother saying things like MAY? It seems to be impossible to
please everybody. There are many things with nuance and exceptions. When I
state things one way, some people (often legitimately) snipe. When I don't
insist on certainty, other have problem with that. When I make it short, I
am clearly leaving many things out. When I go into as much detail as I am
aware of, I get feedback that it is too long or boring or it wanders too
much. None of this is a problem as much as a reality about tradeoffs.

So before I respond, here is a general statement. I am NOT particularly
interested in much of what we discuss here from a specific point of view.
Someone raises a question and I think about it. They want to know of a
better way to get a random key from a dictionary. My thought is that if I
needed that random key, maybe I would not have stored it in a dictionary in
the first place. But, given that the data is in a dictionary, I wonder what
could be done. It is an ACADEMIC discussion with a certain amount of hand
waving. Sometimes I do experiment and show what I did. Other times I say I
am speculating and if someone disagrees, fine. If they show solid arguments
or point out errors on my part or create evidence, they can change my mind. 

You (Steve) are an easy person to discuss things with but there are some who
are less. People who have some idea of my style and understand the kind of
discussion I am having at that point and who let me understand where they
are coming from, can have a reasonable discussion. The ones who act like TV
lawyers who hear that some piece of evidence has less than one in a
quadrillion chance of happening then say BUT THERE IS A CHANCE so reasonable
doubt ... are hardly worth debating.

You replied to one of my points with this about a way to partition data:

---
The obvious solution:

keys = list(mydict.keys())
random.shuffle(keys)
index = len(keys)*3//4
training_data = keys[:index]
reserved = keys[index:]
---

(In the above, "---" is not python but a separator!)

That is indeed a very reasonable way to segment the data. But it sort of
makes my point. If the data is stored in a dictionary, the way to access it
ended up being to make a list and play with that. I would still need to get
the values one at a time from the dictionary such as in the ways you also
show and I omit.

For me, it seems more natural in this case to simply have the data in a data
frame where I have lots of tools and methods available. Yes, underneath it
all providing an array of indices or True/False Booleans to index the data
frame can be slow but it feels more natural. Yes, python has additional
paradigms I may not have used in R such as list comprehensions and
dictionary comprehensions that are conceptually simple. But I did use the
R-onic (to coin a phrase nobody would ironically use) equivalents that can
also be powerful and I need not discuss here in a python list. Part of
adjusting to python includes unlearning some old habits and attitudes and
living off this new land. [[Just for amusement, the original R language was
called S so you might call its way of doing things Sonic.]]

I see a balance between various ways the data is used. Clearly it is
possible to convert it between forms and for reasonable amounts of data it
can be fast enough. But as you note, at some point you can just toss one
representation away so maybe you can not bother using that in the first
place. Keep it simple.

In many real life situations, you are storing many units of data and often
have multiple ways of indexing the data. There are representations that do
much of the work for you. Creating a dictionary where each item is a list or
other data structure can emulate such functionality and even have advantages
but if your coding style is more comfortable with another way, why bother
unless you are trying to learn other ways and be flexible.

As I have mentioned too many times, my most recent work was in R and I
sometimes delight and other times groan at the very different ways some
things are done when using specific modules or libraries. But even within a
language and 

Re: [Tutor] Defining variable arguments in a function in python

[Avi Gross]

Mark,

I will be happy to make a somewhat less brief reply to a reasonable enough 
question.

I admit I have not studied the charter for the group. The standard library is 
not something I consider as many distributions automatically add libraries like 
np and pandas. I do accept your point. I still think it reasonable to MENTION 
the existence of some functionality that the person might use as some people 
just want to solve a problem, but clearly detailed examples and tutorials are 
way beyond that scope. I have unfortunately had people who got a private reply 
tell me to put it on the forum where clearly many are not interested.

On to the main point about recent posts that you replied to. Someone asked a 
question on how to do something using python.

Some responses boiled down to you can't do that.

One suggested something similar  in another way by asking what could be done in 
a context to make it happen, implying there was no answer.

I took the bait and in a very limited way suggested a context. Some may think 
the answer is not much of an answer or not useful and they probably have a 
point.

What I call an ACADEMIC exercise will vary. It may simply mean that I do not 
perceive any personal need to do things that way BUT since you asked, here is 
something ...

The language at any particular time and in a particular implementation allows 
you to do some things that are not necessarily a good idea. It is fair to 
suggest people avoid doing that but unfair to say it is WRONG as in it should 
not work. A dumb example is features that are deprecated but still exist. The 
goal is to switch, not keep using it till the last minute. But for a program 
you will run today then toss, it works!

I have to approach my role here carefully and appreciate that some choices will 
not be met well by some, including of course you. I can ignore some requests, 
and mainly do. There are others who can reply, often better. But when something 
catches my interest, it more often is not about an aspect of very basic python. 
I can reply and suggest the person not bother doing that, or perhaps not that 
way, or perhaps not use python. That might indeed teach them some things. Or, I 
can give a measured reply helping them see what can legally be done and maybe 
point out some tradeoffs including why it may not be the suggested way.

Last point. I promise! We often debate efficiency here. In retrospect, some 
people just want to solve a problem so all they may want is to find their error 
and let them do it as planned. Others ask if there is a more efficient way. 
Their question invites another approach. Unfortunately, that is when I often 
enter into an academic discussion. Maybe I should go back into teaching and 
bore people elsewhere.  I am sure that would make you happy for as long as 5 
minutes.


-Original Message-
From: Tutor  On Behalf Of Mark 
Lawrence
Sent: Sunday, December 30, 2018 4:35 PM
To: tutor@python.org
Subject: Re: [Tutor] Defining variable arguments in a function in python

On 30/12/2018 17:26, Avi Gross wrote:
> Replying to Steve's points. Again, it was not a serious design and 
> said so but was an ACADEMIC exploration of what could be done. I fully 
> agree with Steve that it is probably not a great idea to do this but 
> note the original request might not have been a great one in the first place.
> 

The usual massive snip, but what has an academic exploration got to do with 
learning Python on the tutor mailing list?  From 
https://mail.python.org/mailman/listinfo/tutor "This list is for folks who want 
to ask questions regarding how to learn computer programming with the Python 
language and its standard library. "  Also note the latter, nothing in there 
about pandas dataframes.

Now will you please go away.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Wrap a pipe to get text mode

[Avi Gross]

Brief(er) message on topic:

 

Someone was having a problem that was traced to their use of a pipe to
communicate between tasks that normally passes data in binary mode.

 

They were offered solutions on how to convert the data that should work
fine. I am offering a different solution from the Lutz Book I am currently
reading:

 

Programming Python: Powerful Object-Oriented Programming

By Mark Lutz

 

This is the snippet on-line:

 

https://books.google.com/books?id=q8W3WQbNWmkC

=PA226=PA226=python+wrap+a+pipe=bl=Y8ddhRkA2E=NW2Yy
gRxI9qUtJjMQx77Xhwfy88=en=X=2ahUKEwj-sZygzv_eAhUCnFkKHbbxBu8Q6AEwC
noECAQQAQ#v=onepage=python%20wrap%20a%20pipe=false

 

The key is to take one end of the pipe and wrap a file around it that then
provides the interface that processes and returns raw data as text:

 

In the parent, take the side of the pipe you read from, called pipein as in:

 

pipe, pipeout = os.pipe

 

and do this:

 

pipein = os.fdopen(pipein)

 

NOTE that their method does not use Popen() to sun a command. Your child
process would need to start that program on their own. But the text received
would not require further processing from binary.

 

Many other solutions exist, such as on some systems using named pipes which
can be opened in text mode. Even simpler is to run the external process with
an argument like ">file" and have python open that file in text mode after
it completes. These solutions do require having permission to create files
and perhaps require additional effort to create a unique temp file.

 

 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor